Content uploaded by Alper Erdogan

Author content

All content in this area was uploaded by Alper Erdogan on Mar 23, 2015

Content may be subject to copyright.

1

A Convolutive Bounded Component Analysis

Framework for Potentially Non-Stationary

Independent and/or Dependent Sources

Huseyin A. Inan, Student Member,IEEE, Alper T. Erdogan, Senior Member,IEEE

Koc University

Istanbul, Turkey

Abstract— Bounded Component Analysis (BCA) is a recent

framework which enables development of methods for the sep-

aration of dependent as well as independent sources from their

mixtures. This article extends a recent geometric BCA approach

introduced for the instantaneous mixing problem to the convo-

lutive mixing problem. The article proposes novel deterministic

convolutive BCA frameworks for the blind source extraction and

blind source separation of convolutive mixtures of sources which

allows the sources to be potentially non-stationary. The global

maximizers of the proposed deterministic BCA optimization

settings are proved to be perfect separators. The article also

illustrates that the iterative algorithms corresponding to these

frameworks are capable of extracting/separating convolutive

mixtures of not only independent sources but also dependent

(even correlated) sources in both component (space) and sample

(time) dimensions through simulations based on a Copula dis-

tributed source system. In addition, even when the sources are

independent, it is shown that the proposed BCA approach have

the potential to provide improvement in separation performance

especially for short data records based on the setups involving

convolutive mixtures of digital communication sources.

Index Terms— Bounded Component Analysis, Independent

Component Analysis, Convolutive Blind Source Separation, De-

pendent Source Separation, Finite Support, Frequency-Selective

MIMO Equalization.

I. INTRODUCTION

Blind Source Separation (BSS) is a major area of research

with a large scope of applications in signal processing and

machine learning [1]. BSS aims to extract individual compo-

nents (or sources) from their mixture samples where there is

no, or very limited, prior information about their nature or the

mixing process. The blindness which refers to using only the

observations with the absence of information on the mixing

system is a striking feature which leads to its widespread use.

However, the blindness feature also rises difﬁculties to the BSS

problem where the challenge created by the lack of training

data and relational statistical information is in general dealt

with exploiting some side information/assumptions about the

system.

Huseyin A. Inan is with Electrical-Electronics Engineering Department of

Koc University, Sariyer, Istanbul, 34450, Turkey. Contact Information: Email:

hinan@ku.edu.tr

Alper T. Erdogan is with Electrical-Electronics Engineering Department of

Koc University, Sariyer, Istanbul, 34450, Turkey. Contact Information: Email:

alper@stanfordalumni.org, Phone:+90 (212) 338 1490, Fax: +90 (212) 338

1548

The BSS problems are mostly addressed by means of

Independent Component Analysis (ICA) [1]–[3] exploiting the

assumption that the original sources are mutually statistically

independent. ICA has been the most popular BSS solution

approach, due to the applicability of independence assumption

in wide span of applications, as well as the mathematical

tractability of the corresponding framework. A variety of other

BSS methods have also been emerged from different data

model assumptions such as time structure (e.g., [4], [5]),

sparsity (e.g., [6]) and special constant modulus or ﬁnite

alphabet structure of communications signals (e.g. [7]–[9]).

In most practical BSS applications the sources take values

from a compact set. Puntonet et.al.’s contribution in [10] can be

considered as the pioneering work taking advantage of the ge-

ometric structure based on such compactness of sources. ICA

framework has an important branch where source boundedness

has been exploited as an additional asset in addition to the

founding independence assumption. An in inﬂuential work in

this branch is by Pham [11] where the mutual information cost

function is approximated in terms of quantile function which

can be estimated from the order statistics. In this work, Pham

show that when the sources are bounded, the proposed cost

function can be formulated in terms of the ranges of separator

outputs. In a similar context, Cruces and Duran extend the

deﬁnition of the Renyi’s Entropy in [12], leading to support

length minimization to separate sources from their mixtures.

Additionally, Vrins et.al. proposed BSS algorithms exploiting

range minimization approach in references [13]–[15]. In a sim-

ilar direction, Erdogan extend the blind equalization approach

in [16], [17] and propose BSS algorithms based on inﬁnity

norm minimization for bounded source signals in [18], [19],

[20]. where the approaches assume peak symmetry regarding

the bounded source signals which is later abandoned in [21].

The aforementioned approaches utilize the assumption of

source boundedness within the ICA framework. In a recent

work [22], Cruces proved that when the sources are known

to be bounded, the hypothesis of the statistical independence

of sources can be replaced with a weaker domain separability

assumption. This new framework, referred as Bounded Com-

ponent Analysis (BCA), enables development of methods for

the separation of independent and/or dependent sources from

their mixtures. The weaker domain separability assumption

can be stated as follows: (the convex hull of the) the sup-

port of the joint density of sources can be written as the

2

cartesian product of (the convex hulls of the) supports of the

individual source marginals. We note that this is a necessary

condition for the independence, however, the independence

assumption additionally requires the joint pdf separability on

top of domain separability. Therefore, under the boundedness

property ICA becomes a special case of BCA and by replacing

the independence assumption with the domain separability

assumption, it is possible to separate both independent and

dependent sources. Hence, BCA provides a more general and

ﬂexible framework than ICA when the sources are bounded.

In this new framework, Cruces introduced a blind source

extraction algorithm in [22]. A deﬂationary algorithm is pro-

posed for BCA in [23]. In [24], total output range minimization

based BSS approach is positioned as a BCA method for the

separation of uncorrelated sources. Furthermore, a stationary

point analysis for the proposed algorithms based on symmet-

rical orthogonalization is provided. More recently, Erdogan

introduced a geometric BCA framework and a family of

BCA algorithms in [25], which can separate both independent

and dependent (even correlated) sources from their mixtures

where the mixing system is instantaneous. In this approach,

two geometric objects regarding the separator outputs are

introduced, i.e., principal hyperellipsoid and bounding hy-

perrectangle where the separation problem is based on the

maximization of the relative sizes of these objects. When the

volume is used for the representation of the size of bounding

hyperrectangle, a generalized form of Pham’s objective in

[11], which was derived by modifying the mutual informa-

tion objective in ICA framework, is obtained. Based on the

similar geometric treatment, a convolutive BCA approach for

wide sense stationary (dependent or independent) sources was

introduced in [26], [27]. A variety of different geometric

approaches are introduced for the context of hyperspectral

imaging in [28], [29] and [30] by considering a minimum

volume of a simplex that circumscribes the data space.

In this article, we extend the instantaneous or memoryless

BCA approach introduced in [25] for the convolutive BCA

problem. Contrary to the stochastic convolutive framework in

[26], [27] based on the stationary source assumption, we pro-

pose deterministic frameworks for the convolutive blind source

extraction and blind source separation problems which allows

the sources to be potentially non-stationary. We point out that

the sources could be stationary or non-stationary and we do

not exploit non-stationary property of sources. However, the

proposed scheme works for both stationary and non-stationary

sources. We show that the algorithms corresponding to these

frameworks are capable of extracting/separating convolutive

mixtures of not only independent sources but also dependent

(even correlated) sources where the correlation can be in

both space and time dimensions. We can highlight the novel

contributions of the article as:

1) The article provides convolutive BCA approaches which

can be used to generate algorithms that are capable

of extracting/separating dependent sources as well as

independent sources from their convolutive mixtures.

2) The article proposes a set of convolutive BCA objectives

which are directly deﬁned in terms of mixture samples

rather than some stochastic measures or their sample

based estimates allowing the source samples to be drawn

from a potentially non-stationary process. The intro-

duced framework does not exploit any non-stationarity

feature, therefore it is applicable to both stationary/non-

stationary sources.

3) The article proves that the global optima of pro-

posed BCA objectives correspond to perfect extrac-

tors/separators.

4) The article illustrates the capability of proposed algo-

rithms regarding the extracting/separating convolutive

mixtures of dependent (even correlated) sources. The

article further compares the performance of proposed

algorithms with the state of the art convolutive ICA

approaches and through a digital communications ex-

ample, the article shows the potential for the signiﬁcant

performance improvement offered by the proposed BCA

approach, especially for short data records.

The organization of the article is as follows: Section II

describes the convolutive BCA setup assumed throughout the

article. The blind source extraction approach is provided in

Section III and the blind source separation approach is pro-

vided in Section IV. We illustrate the separation performances

of the BCA algorithms through the numerical examples in

Section V. Finally, Section VI is the conclusion.

Notation: Let A∈Cp×qand a∈Cp×1be arbitrary. The

notation used in the article is summarized in Table I.

Notation Meaning

Am,:(A:,m)mth row (column) of A

R{A}(I{A}) The real (imaginary) part of A

kakrUsual r-norm given by (Pp

m=1 |a|r)1/r.

diag(a)Diagonal matrix whose diagonal entries

starting in the upper left corner are a1, ..., ap.

Q(a)a1a2...ap, i.e. the product of the

elements of a.

˜aM(k)Stacks the vectors

a(k),a(k−1),...,a(k−M+ 1)

into a single Mp size vector ˜aM(k).

emStandard basis vector pointing in the mth direction.

TABLE I

NOTATIO N USED I N THE ART ICLE .

Indexing: mis used for (source, output) vector components,

kis the sample index and iis the algorithm iteration index.

II. CON VOLU TIV E BOUN DED COMPONENT ANA LYSIS

SET UP

In this article, the components of the convolutive BCA setup

that we consider are as follows:

•We consider a deterministic setup with preal sources

represented by the vector s= [s1s2. . . sp]Tand

we assume that the sources are bounded in magnitude.

•The source signals are mixed by a convolutive MIMO

channel and produce the mixtures as:

y(k) =

L−1

X

l=0

H(l)s(k−l), k ∈Z,

3

where {H(l); l∈ {0, . . . , L −1}} are the impulse

response coefﬁcients of dimension q×p. We as-

sume that the mixing system is equalizable having or-

der of L−1[31]. We also assume that q≥p,

therefore, we consider the (over)determined BSS prob-

lem. Deﬁning ˜

H=H(0) H(1) . . . H(L−1)

as the mixing coefﬁcient matrix and ˜sL(k) =

sT(k)sT(k−1) . . . sT(k−L+ 1) T, we can

also write

y(k) = ˜

H˜sL(k), k ∈Z.

•We investigate two related problems corresponding to this

scenario:

– Blind Source Extraction : The mixtures are passed

through an extractor system and produce the single

output as:

o(k) =

M−1

X

l=0

wT(l)y(k−l), k ∈Z.

where {w(l); l∈ {0, . . . , M −1}} are the impulse

response coefﬁcients of dimension q×1and M−1

is the order of the extractor system. Deﬁning ˜w=

wT(0) wT(1) . . . wT(M−1) Tas

the extractor coefﬁcient matrix and ˜yM(k) =

yT(k)yT(k−1) . . . yT(k−M+ 1) T,

we can also write

o(k) = ˜wT˜yM(k), k ∈Z.

– Blind Source Separation : In this case, the outputs

of the separator system are produced as:

o(k) =

M−1

X

l=0

W(l)y(k−l), k ∈Z.

where {W(l); l∈ {0, . . . , M −1}} are the impulse

response coefﬁcients of dimension p×qand M−1

is the order of the separator system. Similarly, we

deﬁne ˜

W=W(0) W(1) . . . W(M−1)

as the separator coefﬁcient matrix which yields

o(k) = ˜

W˜yM(k), k ∈Z.

•The overall mapping from sources to the outputs:

– Blind Source Extraction : The impulse response

coefﬁcients of the overall system are represented as

{g(l); l∈ {0, . . . , P −1}} where the dimension is

p×1. Note that,

gT(k) =

P−1

X

l=0

wT(l)H(k−l),

where P−1 = L+M−2is the order of the overall

system. Therefore, the sources {s(k)∈Rp;k∈Z}

and the single extractor output {o(k)∈R;k∈Z}

are related by

o(k) =

P−1

X

l=0

gT(l)s(k−l), k ∈Z.

Deﬁning ˜g=gT(0) . . . gT(P−1) Tand

˜sP(k) = sT(k). . . sT(k−P+ 1) T, we

have o(k) = ˜gT˜sP(k)for k∈Z.

– Blind Source Separation : In this case, the impulse

response coefﬁcients of the overall system are rep-

resented as {G(l); l∈ {0, . . . , P −1}} where the

dimension is p×p. Note that,

G(k) =

P−1

X

l=0

W(l)H(k−l),

where P−1 = L+M−2is the order of the overall

system. Therefore, the sources {s(k)∈Rp;k∈Z}

and the separator outputs {o(k)∈Rp;k∈Z}are

related by

o(k) =

P−1

X

l=0

G(l)s(k−l), k ∈Z.

Deﬁning ˜

G=G(0) G(1) . . . G(P−1) ,

we have o(k) = ˜

G˜sP(k), for k∈Z.

We assume a ﬁnite set of observations corresponding to the

mixture samples represented by Y={y(1),y(2),...,y(N)}.

The main goal in BSS problems is to adapt the extrac-

tor/separator system based on these observations. Since the

mixing channel ˜

His convolutive having order of L−1, the

corresponding set of unobservable source samples could be

denoted by S={s(−L+ 2),...,s(0),s(1),s(2),...,s(N)}

such that

Y={˜

H˜sL(1),˜

H˜sL(2),..., ˜

H˜sL(N)}.

Depending on the system, for a given convolutive extractor

channel ˜wor a separator channel ˜

Whaving order of M−

1with a corresponding convolutive overall channel ˜gor ˜

G

having order of P−1, the convolutive nature of channel

generates N−M+ 1 outputs and we illustrate the generated

set of extractor output o={o(1), o(2), . . . , o(N−M+ 1)}

or separator outputs O={o(1),o(2),...,o(N−M+ 1)}as

o={˜wT˜yM(M),˜wT˜yM(M+ 1),..., ˜wT˜yM(N)}

={˜gT˜sP(M),˜gT˜sP(M+ 1),...,˜gT˜sP(N)},

or

O={˜

W˜yM(M),˜

W˜yM(M+ 1),..., ˜

W˜yM(N)}

={˜

G˜sP(M),˜

G˜sP(M+ 1),..., ˜

G˜sP(N)}.

We deﬁne the ranges of the sources as

ˆ

R(sm) = max(sm)−min(sm),

for m= 1,2, . . . p, where max(sm)(min(sm)) is the maxi-

mum (minimum) value of smin the set S. It follows that:

s(k) = Λs(k)for k=−L+ 2, . . . , N,

where Λ = diag(ˆ

R(s1),ˆ

R(s2),..., ˆ

R(sp)) is the range matrix

of sand sis the corresponding unit range source vector.

In this article, we extend the deterministic instantaneous

or memoryless BCA approach introduced in [25] for the

4

convolutive BCA problem. The objective functions are directly

deﬁned in terms of mixture samples rather than some stochas-

tic measures or their sample based estimates which allows the

sources to be drawn from either stationary or non-stationary

processes. Regarding the bounding hyper-rectangle, deﬁning

the set SP={˜sP(M),˜sP(M+1),...,˜sP(N)}, we introduce

the following assumption:

Assumption: SPcontains the vertices of its (non-degenerate)

bounding hyper-rectangle (A1).

III. BLI ND SOURCE EXTRACTION

In this section, we ﬁrst introduce the objective function for

the blind source extraction of real signals. We then prove that

the global maxima of the introduced objective function corre-

spond to perfect extractors. We provide the iterative algorithm

corresponding to the objective function. We conclude with the

complex sources extension of the proposed approach.

A. Criterion

We introduce the objective function for the blind source

extraction method as

Je(˜w) = q1

N1PN1

l=1 (o(l)−ˆµo)2

ˆ

R(o),(1)

where ˆµo=1

N1PN1

l=1 o(l),N1=N−M+ 1 and ˆ

R(o)is

the range of the single output o. We note that this objective

function is deduced from the instantaneous BCA objectives

introduced in [25].

We deﬁne

ˆµ˜sP=1

N1

N

X

l=M

˜sP(l),

ˆ

R˜sP=1

N1

N

X

l=M

(˜sP(l)−ˆµ˜sP)(˜sP(l)−ˆµ˜sP)T,

as the sample covariance matrix of ˜sP. If sources are station-

ary, then ˆ

R˜sPis a block Toeplitz matrix. However, sources

are allowed to be non-stationary, therefore, ˆ

R˜sPmay not

be a block Toeplitz matrix. Note that this approach does

not exploit any structure on ˆ

R˜sP(i.e., the sources can be

stationary or non-stationary). Under the condition ˆ

R˜sP0,

the following theorem shows that maximizing the proposed

objective function (1) achieves the goal of blindly extracting

a source from a given convolutive mixture for the setup is

outlined in Section II.

Theorem 1: Assuming the setup in Section II, ˜

His

equalizable by an FIR extractor matrix of order M−1and

under the validity of (A1), the set of global maxima for Jein

(1) is equal to the set of perfect extractors.

Proof: The proof is provided in Appendix A.

B. Algorithm

In this section, we provide the iterative algorithm corre-

sponding to the optimization setting presented in the previous

section.

Rather than maximizing Je, we maximize its logarithm

since with the logarithm operation, we utilize the conversion of

ratio expression to the difference expression since it simpliﬁes

the update components in the iterative algorithm. Therefore,

the new objective function is modiﬁed as

¯

Je(˜w) = log (Je(˜w))

=1

2log ˜wTˆ

R˜yM

˜w−log ˆ

R(o),(2)

where ˆ

R˜yM

is the sample covariance matrix of ˜yM.

Note that the derivative of the ﬁrst part of ¯

Je(˜w)with

respect to ˜wis

∂log ˜wTˆ

R˜yM

˜w

∂˜w=2ˆ

R˜yM

˜w

˜wTˆ

R˜yM

˜w.

Following the similar steps as in [25] for the derivative

of log ˆ

R(o), the subgradient based iterative algorithm for

maximizing objective function (2) is provided as

˜w(i+1) =˜w(i)+µ(i)ˆ

R˜yM

˜w

˜wTˆ

R˜yM

˜w−

1

ˆ

R(o(i))˜yM(lmax(i))−˜yM(lmin(i)),(3)

where µ(i)is the step-size at the ith iteration (see e.g., [17]

for a discussion on step sizes) and lmax(i)(lmin(i)) is the

sample index for which the maximum (minimum) value for

the extractor output is achieved at the ith iteration.

C. Extension to Complex Signals

In the complex domain, both mixing and extractor coefﬁ-

cient matrices are complex matrices, i.e., ˜

H∈Cq×pL and

˜w∈CqM ×1. The set of source vectors Sis a subset of Cp,

the set of single extractor output ois a subset of Cand the

set of mixtures Yis a subset of Cq.

In this section, we extend the approach introduced in the

Section III-A to the complex signals. We modify the objective

function as

Jce(˜w) = q1

N1PN1

l=1 (R{o(l)} − R{ˆµo})2

ˆ

R(R{o}),(4)

where R{ˆµo}=1

N1PN1

l=1 R{o(l)}and ˆ

R(R{o})is the range

of real parts of output o.

We similarly deﬁne ˆ

R`sPas the sample

covariance matrix of `sPwhere `sP(k) =

[R{sT(k)}I{sT(k)}. . . R{sT(k−P+1)}I{sT(k−

P+ 1)}]T. Under the condition ˆ

R`sP0, the following

theorem shows that maximizing the modiﬁed objective

function (4) achieves the blind source extraction of convolutive

mixtures of complex signals.

Theorem 2: Assuming the setup in Section II, ˜

His

equalizable by an FIR extractor matrix of order M−1and

under the validity of (A1), the set of global maxima for Jce

in (4) is equal to a subset of perfect extractors.

Proof: The proof is provided in Appendix B.

5

In the iterative algorithm, we maximize the logarithm of

Jce, therefore, the objective function is modiﬁed as

¯

Jce(`w) = log (Jce (`w))

=1

2log `wTˆ

R`yM

`w−log ˆ

R(R{o}),(5)

where

`w= [ R{wT(0)} − I{wT(0)}. . . −I{wT(M−1)}]T,

and ˆ

R`yM

is the sample covariance matrix of `yM. Following

similar steps, the iterative algorithm for maximizing objective

function (5) is provided as

`w(i+1) =`w(i)+µ(i)ˆ

R`yM

`w

`wTˆ

R`yM

`w−

1

ˆ

R(R{o}(i))`yM(lmax(i))−`yM(lmin(i)),(6)

where µ(i)is the step-size at the ith iteration and lmax(i)

(lmin(i)) is the sample index for which the maximum (mini-

mum) value of the real part of the extractor output is achieved

at the ith iteration. Finally, we can obtain ˜wfrom `wusing

a simple transition ˜wmq+1:(m+1)q=`w2mq+1:2(m+1)q−q−

j`w2(m+1)q−q+1:2mq for m= 0,1, . . . , M −1.

D. Complexity Analysis

The complexity of the introduced algorithm can be speciﬁed

by 4 parts.

•Calculating the single separator output o=˜wT˜yM: The

complexity is O(qM N ).

•Calculating the range of real components of the single

separator output ˆ

R(R{o}): The complexity is O(N).

•Calculating the sample covariance matrix ˆ

R`yM

: The

complexity is O(M2q2N)

•Performing the iterative algorithm (6): The complexity is

O(M2q2).

Therefore, the overall complexity can be determined as

O(M2q2N). The required number of iterations for the con-

vergence depends on the system settings. We will provide

examples in Section V.

IV. BLI N D SOURCE SEPARATI O N

In this section, we ﬁrst introduce an objective function

for the blind source separation of real signals. We then

prove that the global maxima of the introduced objective

function correspond to the perfect separators. We next provide

a family of alternative objective functions. After producing the

iterative algorithms corresponding to the introduced objective

functions, we conclude with the complex extension of the

proposed approaches.

A. Criteria

In order to deﬁne the ﬁrst objective function, we use a

similar geometric setting introduced in [25]. Deﬁning the set

OK={˜oK(K),˜oK(K+ 1),...,˜oK(N−M+ 1)}, we

introduce the following objects corresponding to the sets of

output samples OKand O:

•P(OK): This is the hyper-ellipsoid whose center is

given by the sample mean of the set OK, its principal

semiaxes directions are determined by the eigenvectors

of the sample covariance matrix ˆ

R˜oKcorresponding to

OKand its principal semiaxes lengths are equal to the

principal standard deviations, i.e., the square roots of the

eigenvalues of ˆ

R˜oK.

•B(O): This is the bounding hyper-rectangle which is

deﬁned as minimum volume box covering all the samples

in Oand aligning with the coordinate axes.

The ﬁrst objective function that we introduce for blind

source separation is

Js1(˜

W) = qdet( ˆ

R˜oK)1/K

Qp

m=1 ˆ

R(om),(7)

where

ˆµ˜oK=1

N2

N1

X

l=K

˜oK(l),

ˆ

R˜oK=1

N2

N1

X

l=K

(˜oK(l)−ˆµ˜oK)( ˜oK(l)−ˆµ˜oK)T,

N2=N1−K+ 1 such that ˆ

R˜oKis the sample covariance

matrix of ˜oK.ˆ

R(om)is the range of the m’th component of

the output vectors in the set Oand we choose K≥Pwhere

Pis the order of the overall system.

We note that, as deﬁned in [25],

•qdet( ˆ

R˜oK)refers to the scaled volume of principal

hyper-ellipse for the extended output vector ˜oK.

•Qp

m=1 ˆ

R(om)is the volume of the bounding hyper-

rectangle for the output vector o.

Under the condition ˆ

R˜sK+P−10, the following theorem

shows that maximizing the objective function (7) achieves the

blind source separation of convolutive mixtures whose setup

is outlined in Section II.

Theorem 3: Assuming the setup in Section II, ˜

His

equalizable by an FIR separator matrix of order M−1and

under the validity of (A1), the set of global maxima for Js1

in (7) is equal to the set of perfect separator matrices.

Proof: The proof is provided in Appendix C.

We can propose different alternatives for the denominator of

the objective function (7) (measure of the size of the bounding

hyperrectangle for the output vectors). We can choose the

length of the main diagonal of the bounding hyperrectangle

as a measure of the size instead of its volume. As a result, we

obtain a family of alternative objective functions in the form

Js2,r(˜

W) = qdet( ˆ

R˜oK)1/K

|| ˆ

R(o)||p

r

,(8)

6

where r≥1. We provide the results of analysing this family

of objective functions, for some special rvalues (i.e., r=

1,2,∞) in Appendix D.

B. Algorithms

In this section, we provide the iterative algorithms corre-

sponding to the optimization settings presented in the previous

section.

•Objective Function Js1(˜

W):

Similar to the approach in blind source extraction, rather

than maximizing Js1(˜

W), we maximize its logarithm.

Therefore, the new objective function is modiﬁed as

¯

Js1(˜

W) = log Js1(˜

W)

=1

2Klog det ΓK(˜

W)ˆ

R˜yK+M−1ΓK(˜

W)T

−log p

Y

m=1

ˆ

R(om)!,(9)

where ˆ

R˜yK+M−1

is the sample covariance matrix of

˜yK+M−1. Note that the derivative of the ﬁrst part of

¯

Js1(˜

W)with respect to ˜

Wis

∂log det ΓK(˜

W)ˆ

R˜yK+M−1ΓK(˜

W)T

∂˜

W=

2

K−1

X

l=0

Alp+1:(l+1)p,lq+1:(l+M)q

where A=ΓK(˜

W)ˆ

R˜yK+M−1ΓK(˜

W)T−1ΓK(˜

W)

ˆ

R˜yK+M−1

. Following the similar steps as in [25] for the

derivative of log Qp

m=1 ˆ

R(om), the subgradient based

iterative algorithm for maximizing objective function (9)

is provided as

˜

W(i+1) =˜

W(i)

+µ(i)1

K

K−1

X

l=0

Alp+1:(l+1)p,lq+1:(l+M)q−

p

X

m=1

1

eT

mˆ

R(o(i))em˜yM(lmax(i)

m)−˜yM(lmin(i)

m)T,

(10)

where µ(i)is the step-size at the ith iteration and lmax(i)

m

(lmin(i)

m) is the sample index for which the maximum

(minimum) value for the mth separator output is achieved

at the ith iteration.

•Objective Function Js2,r(˜

W):

We note that for the family of objective functions (8), the

update equation is similar to (10) where the change is in

the derivative of logarithm of the denominator depending

on the selection of r. For r= 1,2, we can write the

update equation as

˜

W(i+1) =˜

W(i)+

µ(i)1

K

K−1

X

l=0

Alp+1:(l+1)p,lq+1:(l+M)q−

p

X

m=1

pˆ

Rm(o(i))r−1

|| ˆ

R(o(i))||r

r

em˜yM(lmax(i)

m)−˜yM(lmin(i)

m)T.

For r=∞, the update equation has the form

˜

W(i+1) =˜

W(i)+

µ(i)1

K

K−1

X

l=0

Alp+1:(l+1)p,lq+1:(l+M)q−X

m∈M(oi)

pβ(i)

m

|| ˆ

R(o(i))||∞

em˜yM(lmax(i)

m)−˜yM(lmin(i)

m)T

where M(o(i))is the set of indexes for which the peak

range value is achieved, i.e.,

M(o(i)) = {m:ˆ

Rm(o(i)) = kˆ

R(o(i)k∞},(11)

and β(i)

ms are the convex combination coefﬁcients.

C. Extension to Complex Signals

In the complex domain, both mixing and separator coefﬁ-

cient matrices are complex matrices, i.e., ˜

H∈Cq×pL and

˜

W∈Cp×qM . The set of source vectors Sand the set of

separator outputs Oare a subset of Cp, the set of mixtures Y

is a subset of Cq.

In this section, we extend the approach introduced in the

Section IV-A to the complex signals. We modify the ﬁrst

objective function for the blind source separation of complex

signals as

Jcs1(˜

W) = qdet( ˆ

R`oK)1/K

Q2p

m=1 ˆ

R(`om),(12)

where ˆ

R`oKis the sample covariance matrix of `oKwhere

`oK(k)=[R{oT(k)}I{oT(k)}. . . R{oT(k−K+

1)}I{oT(k−K+ 1)}]Tand Q2p

m=1 ˆ

R(`om)is the product

of ranges of real and imaginary parts of all separator outputs.

Under the condition ˆ

R`sK+P−10, the following theorem

shows that maximizing the modiﬁed objective function (12)

achieves the blind source separation of convolutive mixtures

of complex signals.

Theorem 4: Assuming the setup in Section II, ˜

His

equalizable by an FIR separator matrix of order M−1and

under the validity of (A1), the set of global maxima for Jcs1

in (12) is equal to a subset of perfect separator matrices.

Proof: The proof is provided in Appendix E.

7

In the iterative algorithm, we maximize the logarithm of

Jcs1, therefore, the ﬁrst objective function is modiﬁed as

¯

Jcs1(˜

W) = log Jcs1(˜

W)

=1

2Klog det Γ2K(`

W)ˆ

R`yK+M−1Γ2K(`

W)T

−log 2p

Y

m=1

ˆ

R(`om)!,(13)

where `

W=

R{W0} −I{W0}. . . R{WM−1} −I{WM−1}

I{W0}R{W0}. . . I{WM−1}R{WM−1}

and ˆ

R`yK+M−1

is the sample covariance matrix of `yK+M−1.

The corresponding iterative update equation of W(n)for

n= 0,1, . . . , M −1can be written as

W(i+1)(n) = W(i)(n)

+µ(i)C1:p,2nq+1:(2n+1)q+Cp+1:2p,(2n+1)q+1:2(n+1)q

+jCp+1:2p,2nq+1:(2n+1)q−C1:p,(2n+1)q+1:2(n+1)q−

2p

X

m=1

1

2ˆ

R(`o(i)

m)vm˜yM(lmax(i)

m)−˜yM(lmin(i)

m)H,(14)

where C=1

2KPK−1

l=0 F2lp+1:2(l+1)p,2lq+1:2(l+M)q,F=

Γ2K(`

W)ˆ

R`yK+M−1Γ2K(`

W)T−1Γ2K(`

W)ˆ

R`yK+M−1

and

vm=emm≤p,

iem−pm > p. (15)

Similar to the complex extension of Js1, we can extend the

Js2family by modifying

Jcs2,r(˜

W) = qdet( ˆ

R`oK)1/K

|| ˆ

R(`o)||2p

r

.(16)

The update equation is similar to (14) where the change is in

the derivative of logarithm of the denominator depending on

the selection of r, e.g.,

•r= 1,2Case: In this case

∂log ||R(`o)||2p

r

∂˜

W=

2p

X

m=1

pˆ

Rm(`o(i))r−1

|| ˆ

R(`

o(i))||r

r

vm˜yM(lmax(i)

m)−˜yM(lmin(i)

m)H

where vmis as deﬁned in (15).

•r=∞Case: In this case

∂log ||R(`o)||2p

r

∂˜

W=X

m∈M(`

o(i))

pβ(i)

m

|| ˆ

R(`

o(i))||∞

vm˜yM(lmax(i)

m)−˜yM(lmin(i)

m)H

where vmis as deﬁned in (15),

M(`

o(i)) = {m:ˆ

Rm(`

o(i)) = kˆ

R(`

o(i))k∞},

and β(i)

ms are the convex combination coefﬁcients.

D. Complexity Analysis

Similarly, we determine the complexity of the introduced

algorithms by considering 4 parts.

•Calculating the separator output o=˜

W˜yM: The com-

plexity is O(pqM N ).

•Calculating the ranges of separator outputs ˆ

R(o): The

complexity is O(pN).

•Calculating the sample covariance matrix ˆ

R`yK+M−1

: The

complexity is O((K+M−1)2q2N)

•Performing the iterative algorithm (14): The complexity

is O(pK(K+M−1)2q2).

Therefore, the overall complexity can be determined as

O((K+M−1)2q2N)+O(pK(K+M−1)2q2). The required

number of iterations for the convergence depends on the

system settings. We will provide examples in Section V.

V. EX AMP LES

In this section, we illustrate the extraction/separation capa-

bility of the proposed algorithms for the convolutive mixtures

of both independent and dependent sources.

A. Blind Source Extraction

We ﬁrst consider the following scenario to illustrate the

performance of the proposed blind source extraction algorithm

regarding the convolutive mixtures of space-time correlated

sources: In order to generate space-time correlated sources,

we ﬁrst generate a samples of a τp size vector, d, with

Copula-t distribution, a perfect tool for generating vectors

with controlled correlation, with 4 degrees of freedom whose

correlation matrix parameter is given by R=Rt⊗Rs

where Rt(Rs) is a Toeplitz matrix whose ﬁrst row is

1ρt. . . ρτ−1

t(1ρs. . . ρp−1

s). Each sam-

ple of dis partitioned to produce source vectors, d(k) =

s(kτ )s(kτ + 1) . . . s((k+ 1)τ−1) . Therefore, we

obtain the source vectors as samples of a wide-sense cyclosta-

tionary process whose correlation structure in time direction

and space directions are governed by the parameters ρtand

ρs, respectively.

In the simulations, we consider a scenario with 7sources

and 20 mixtures, an i.i.d. Gaussian convolutive mixing system

with order 7and a extractor of order 8. We set ρs= 0.5,ρt=

0.5and τ= 5. We note that the sources are non-stationary

in this case (we will cover stationary sources in the digital

communication sources scenario).

Figure 1 shows the output total Signal energy to total Inter-

ference+Noise energy (over all outputs) Ratio (SINR) obtained

for the proposed BCA algorithm ( ¯

Je) for various sample

lengths under 45dB SNR. SINR performance of Minimum

Mean Square Error (MMSE) ﬁlter of the same order, which

uses full information about mixing system and source/noise

statistics, is also shown to evaluate the relative success of the

proposed approach. A comparison has also been made with

a gradient maximization of the criterion (kurtosis) of [32]

(KurtosisMax.) and Alg.2 of [33] where we take kmax = 50

and lmax = 20. We have obtained these methods from [2],

[34]. As we did not encounter any convolutive BSS algorithm

8

with correlated source separation capability, we compared our

algorithm with some well known convolutive ICA approaches.

0 5 10 15 20 25 30 35 40 45 5023 4 5 7.5

−20

−10

0

10

20

30

40

50

Signal to Interference-plus-Noise Ratio (dB)

Sampl e Lengt h (x103)

MMSE

BCA (

¯

Je)

Alg.2 of [33]

KurtosisMax. of [3 2]

Fig. 1. Result of the proposed blind source extraction algorithm performance

for the convolutive mixtures of dependent sources (ρsand ρtis set as 0.5)

for various sample lengths under SNR = 45dB.

For the same setup, Figure 2 shows the output total Signal

energy to total Interference+Noise energy (over all outputs)

Ratio (SINR) obtained for the proposed BCA algorithm ( ¯

Je),

gradient maximization of the criterion (kurtosis) of [32] (Kur-

tosisMax.), Alg.2 of [33], and MMSE for various sample

lengths under 20dB SNR.

0 5 10 15 20 25 30 35 40 45 5023 4 7.5

−20

−15

−10

−5

0

5

10

15

20

25

Signal to Interference-plus-Noise Ratio (dB)

Sampl e Length (x103)

MMSE

BCA (

¯

Je)

Alg.2 of [33]

KurtosisMax. of [3 2]

Fig. 2. Result of the proposed blind source extraction algorithm performance

for the convolutive mixtures of dependent sources (ρsand ρtis set as 0.5)

for various sample lengths under SNR = 20dB.

These results demonstrate that the performance of the

proposed blind source extraction algorithm is approaching fast

to its MMSE counterpart as the sample length increases. On

the other hand, the performance of gradient maximization of

the criterion (kurtosis) of [32] (KurtosisMax.) and Alg.2 of

[33] is far from the performance of MMSE ﬁlter even when

the sample length is increased (Figure 1) or they require more

sample lengths to reach the same SINR performance (Figure 2)

since in the correlated case, independence assumption simply

fails. Therefore, we observe that the proposed BCA approach

is capable of blind source extraction of convolutive mixtures

of space-time correlated sources.

B. Blind Source Separation

We ﬁrst consider a similar scenario as in the blind source ex-

traction examples to illustrate the performance of the proposed

blind source separation algorithms regarding the separability

of convolutive mixtures of space-time correlated sources.

Here, we consider a scenario with 5sources and 15 mix-

tures, an i.i.d. Gaussian convolutive mixing system with order

5and a separator of order 6where the sample size is 50000.

Figure 3 shows the output total Signal energy to total Inter-

ference+Noise energy (over all outputs) Ratio (SINR) obtained

for the proposed BCA algorithms ( ¯

Js1,¯

Js2,1,¯

Js2,2¯

Js2,∞) for

various space correlation parameters under 45dB SNR. The

performances of MMSE, gradient maximization of the crite-

rion (kurtosis) of [32] (KurtosisMax.) and Alg.2 of [33] are

also plotted for comparison. We note that the algorithm ¯

Js1

yields better performance than the other algorithms.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

0

5

10

15

20

25

30

35

40

45

50

Signal to Interference-plus-N oise Ratio (dB)

Correl ation C oeﬃcient ρs

MMSE

BCA ( ¯

Js1)

BCA ( ¯

Js2,1)

BCA ( ¯

Js2,2)

BCA ( ¯

Js2,∞)

KurtosisMax. of [32]

Alg.2 of [33]

Fig. 3. Results of the proposed blind source separation algorithms’ perfor-

mances for the convolutive mixtures of dependent sources for various space

correlation parameters under SNR = 45dB.

In Figure 4, we present a typical convergence graph of

the BCA algorithms when ρs= 0.3by plotting SINR

performances and the objective functions versus the number

of iterations.

0 250 500 750 1000 1250 1500

−20

−10

0

10

20

30

40

50

Iterat ions

SINR (dB)

Algorithm (

¯

Js1)

Algorithm (

¯

Js2,1)

Algorithm (

¯

Js2,2)

Algorithm (

¯

Js2,∞)

0 250 500 750 1000

−26

−24

−22

−20

−18

−16

−14

−12

−10

−8

−6

Iterat ions

log(

¯

Js1), log(

¯

Js2,1), log(

¯

Js2,2), log(

¯

Js2,∞)

log(

¯

Js1)

log(

¯

Js2,1)

log(

¯

Js2,2)

log(

¯

Js2,∞)

Fig. 4. Convergence graph of the introduced BCA algorithms.

For the same setup, Figure 5 shows the output total Signal

energy to total Interference+Noise energy (over all outputs)

Ratio (SINR) obtained for the BCA algorithm ( ¯

Js1), gradient

maximization of the criterion (kurtosis) of [32] (Kurtosis-

Max.), Alg.2 of [33], and MMSE for various space correlation

parameters under 20dB SNR.

For the same setup, we ﬁnally choose 5dB SNR and Figure

6 shows the corresponding SINR results of the algorithms.

These results demonstrate that the performance of proposed

blind source separation algorithms closely follow its MMSE

counterpart for a wide range of correlation values. Therefore,

we obtain a convolutive extension of the BCA approach

introduced in [25], which is capable of separating convolutive

mixtures of space-time correlated sources.

Also note that the proposed blind source separation algo-

rithms maintain high separation performance for various space

9

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

0

5

10

15

20

25

Signal to Interference-plus-N oise Ratio (dB)

Correl ation C oeﬃcient ρs

MMSE

BCA ( ¯

Js1)

KurtosisMax. of [3 2]

Alg.2 of [33]

Fig. 5. Results of the proposed blind source separation algorithms’ perfor-

mances for the convolutive mixtures of dependent sources for various space

correlation parameters under SNR = 20dB.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

−6

−4

−2

0

2

4

6

8

10

Signal to Interference-plus-Noise Ratio (dB)

Correl ation C oeﬃcient ρs

MMSE

BCA ( ¯

Js1)

BCA ( ¯

Js2,1)

BCA ( ¯

Js2,2)

KurtosisMax. of [3 2]

Alg.2 of [33]

Fig. 6. Results of the proposed blind source separation algorithms’ perfor-

mances for the convolutive mixtures of dependent sources for various space

correlation parameters under SNR = 5dB.

parameters. However, the performance of gradient maximiza-

tion of the criterion (kurtosis) of [32] (KurtosisMax.) and

Alg.2 of [33] degrades substantially with increasing correlation

since the independence assumption does not hold. We point

out that when ρs= 0 the sources are independent, yet

BCA algorithms still outperforms other ICA algorithms. This

result can be attributed to the ﬁnite sample effects. In other

words, although the sources are stochastically independent,

ﬁnite samples may not reﬂect this behaviour and the sources

may even have non-zero sample correlation. BCA algorithms

being robust to such correlations can offer better performance.

The effect of sample size is investigated in the next scenario.

We next consider the following scenario to illustrate the

performance of the proposed blind source separation algorithm

for the convolutive mixtures of digital communication sources.

We consider 5 complex 4-QAM sources where we take 15

mixtures, an i.i.d. Gaussian convolutive mixing system with

order 5 and a separator of order 6. The sources are stationary

in this case. We use the objective function ¯

Jcs1as the BCA

algorithm for this simulation. The resulting Signal to Inter-

ference Ratio is plotted with respect to the sample lengths in

Figure 7. We have also compared our algorithm with a gradient

maximization of the criterion (kurtosis) of [32] (KurtosisMax.)

and Alg.2 of [33].

As it can be observed from Figure 7, the proposed BCA

approach achieves better performance than ICA based ap-

proaches. We again note that, the proposed method does not

assume/exploit statistical independence. The only impact of

short data length is on accurate representation of source box

0 1000 2000 3000 4000 5000 100007500

−20

−10

0

10

20

30

40

50

60

70

Signal to Interference Ratio (dB)

Sample Length

BCA (

¯

Jcs1)

Alg.2 of [33]

KurtosisMax. o f [32]

Fig. 7. Result of the proposed blind source separation algorithm performance

for the convolutive mixtures of digital communication sources for various

sample lengths.

boundaries. The simulation results suggest that the shorter

data records may not be sufﬁcient to reﬂect the stochastic

independence of the sources, and therefore, the compared

algorithms require more data samples to achieve the same SIR

level as the proposed approach.

In the last scenario, the mixing system is chosen as (2 x

2) paraunitary with order 4 and the sources are 16-QAM. We

use the objective function ¯

Jcs1as the BCA algorithm for this

simulation. The resulting Signal to Interference Ratio is plotted

with respect to the sample lengths in Figure 8. We have also

compared our algorithm with a gradient maximization of the

criterion (kurtosis) of [32] (KurtosisMax.), Alg.2 of [33], and

the algorithm introduced in [35]1.

0 2.5 5 7.5 10 20 30 40 50

5

10

15

20

25

30

35

Signal to Interference Ratio (dB )

Samp le Leng th (x10 0)

BCA (

¯

Jcs1)

KurtosisMax. o f [32]

Alg.2 of [33]

Algorithm of [35]

Fig. 8. Result of the proposed algorithm performance for the convolutive

mixtures of digital communication sources by a paraunitary channel for

various sample lengths.

We note that the proposed BCA approach achieves better

performance. We also point out that the algorithm introduced

in [35] assumes paraunitary channel and it requires numer-

ically stable prewhitening operation for general equalizable

channels [31] while the approach introduced in this paper

does not assume any structure for the mixing system beyond

equalizablitiy.

VI. CONCLUSION

In this article, we introduced deterministic and geometric

BCA frameworks for the convolutive blind source extraction

1The authors would like to thank Prof. Lathauwer for kindly sharing their

code for the algorithm in [35]

10

and separation problems. Contrary to the convolutive BCA

framework in [26], [27] for stationary sources, the proposed

deterministic framework is applicable to more general class

of stationary and non-stationary sources. We should note that

the proposed framework doesn’t take advantage of the any

special form of non-stationarity feature, rather it provides a

ﬂexible framework enabling separation of non-stationary sig-

nals in addition to stationary signals. Since the independence

assumption is replaced by a much weaker domain separability

assumption, the proposed framework is also ﬂexible in terms

of its ability to extract/separate dependent/correlated as well

as independent sources. The numerical examples illustrate the

dependent/independent source extraction/separation capability

and potential short data record performance advantage of the

proposed algorithms.

APPENDIX

A. Proof of Theorem 1

We ﬁrst note that, following similar steps as in [25], when

the assumption (A1) stated in Section II holds, we can write

the range of oas ˆ

R(o) = ||˜gT˜

Λ||1where ˜

Λ = I⊗Λis the

range matrix of ˜sP.

Since o(l) = ˜gT˜sP(l+M−1) for l= 1,2, . . . , N1, we

have

1

N1

N1

X

l=1

(o(l)−ˆµo)2

=1

N1

N

X

l=M

˜gT(˜sP(l)−ˆµ˜sP)(˜sP(l)−ˆµ˜sP)T˜g

=1

N1

N

X

l=M

˜gT˜

Λ(˜sP(l)−ˆµ˜sP)(˜sP(l)−ˆµ˜sP)T˜

ΛT˜g

=˜gT˜

Λˆ

R˜sP

˜

ΛT˜g,(17)

where ˆµ˜sP=1

N1PN

l=M˜sP(l),ˆµ˜sP=1

N1PN

l=M˜sP(l)and

ˆ

R˜sPis deﬁned as the sample covariance matrix of ˜sP.

We can further deﬁne qT=˜gT˜

Λand rewrite the equality

(1) in terms of qas

Je(q) = qqTˆ

R˜sP

q

||q||1

.(18)

Note that, maximizing Je(q)is equivalent to the corre-

sponding optimization setting

maximize qqTˆ

R˜sP

q

s.t. ||q||1≤γ

where γis a constant. Also note that, assuming ˆ

R˜sP0,

qqTˆ

R˜sP

qis a convex function and the region of ||q||1≤γ

corresponds to a convex polytope. From the deﬁnition of a con-

vex polytope (Vertex Representation [36]), this is the convex

hull of the vertices of polytope. Therefore, the maximum of

qqTˆ

R˜sP

qwill be attained at one of the vertices (whichever

has the maximum value) and therefore, the maximum will be

attained when qhas only one non-zero component. To see

that, we can take any vector qiinside the convex polytope

(i.e., satisfying ||qi||1≤γ). From the deﬁnition of vertex

representation [36], qi=α1qv1+α2qv2+. . .+αpP qvpP where

qv1,qv2,...,qvpP are vertices of polytope and PpP

l=1 αl= 1.

Deﬁning f(vq) = qqTˆ

R˜sP

qand using Jensen’s inequality,

we have

f(q)≤α1f(qv1) + α2f(qv2) + . . . +αpP f(qvpP )

≤max{f(qv1), f (qv2), . . . , f(qvpP )}.

Therefore, the maximum is attained at the vertex which has the

maximum value and this yields that qhas only one non-zero

component.

To observe that from a geometric point of view, assuming

ˆ

R˜sP0, for any constant γ, the vectors qsatisfying

qqTˆ

R˜sP

q=γconstitutes an hyper-ellipsoid. Note that, for

any constant value of ||q||1, the maxima of qqTˆ

R˜sP

qwill

be attained at one of the corner points ( i.e., where qhas

only one non-zero component ). A two dimensional example

is illustrated in Figure 9.

Fig. 9. Two dimensional example for the global maxima of (1).

Since ˜gT=qT˜

Λ−1,˜gwill also have only one non-zero

component, therefore, the global maxima of (1) correspond to

perfect extractors.

B. Proof of Theorem 2

We begin with noting that

R{o(k)}=

P−1

X

l=0

R{gT(l)}R{s(k+M−1−l)}

−I{gT(l)}I{s(k+M−1−l)}.

If we deﬁne

`g=R{gT(0)} − I{gT(0)}. . . −I{gT(P−1)}T,

`sP(k) = R{sT(k)}I{sT(k)}. . . I{sT(k−P+ 1)}T

we then have

R{o(k)}=`gT`sP(k+M−1),

for k= 1,2, . . . , N1. Following similar steps, we can write

the range of R{o}as ˆ

R(R{o}) = ||`gT`

Λ||1where `

Λ = I⊗Λ

11

is the range matrix of `sP. Similar to (17), we have

1

N1

N1

X

l=1

(R{o(l)} − R{ˆµo})2=`gT`

Λˆ

R`sP

`

ΛT`g,

where ˆ

R`sPis deﬁned as the sample covariance matrix of `sP.

Deﬁning `qT=`gT`

Λand rewriting the equality (4) in terms of

`qyields

Jce(`q) = q`qTˆ

R`sP

`q

||`q||1

.(19)

Following similar analogy, as a result, the maximum of (4) is

attained when `ghas only one non-zero component which also

implies that ˜ghas only one non-zero component. Note that

the non-zero component of ˜gwill be real or purely imaginary.

Therefore, the global maxima of (4) correspond to a subset of

perfect extractors for complex signals.

C. Proof of Theorem 3

We deﬁne the operator ΓKsuch that ΓK(˜

G)is a block

Toeplitz matrix of dimension Kp ×(K+P−1)pwhose ﬁrst

block row is [G(0) G(1) ... G(P−1) 0... 0]and

ﬁrst block column is hGT(0) 0... 0iT

where the zero

matrices (0) have the size p×psame as the matrices G(l)

for l= 0, . . . , P −1. This yields,

˜oK(l) = ΓK(˜

G)˜sK+P−1(l+M−1),

for l=K, K + 1, . . . , N1. Deﬁning A=K+P−1, we have

ˆ

R˜oK= ΓK(˜

G)˘

Λˆ

R˜sA

˘

ΛTΓK(˜

G)T,

where ˘

Λ = I⊗Λis the range matrix of ˜sAand ˆ

R˜sAis

the sample covariance matrix of ˜sA. Deﬁning Q= ΓK(˜

G)˘

Λ

yields ˆ

R˜oK=Qˆ

R˜sA

QT.

Following similar steps as in [25], under the assumption

(A1) stated in Section II, we can write the range of mth com-

ponent of oas ˆ

R(om) = || ˜

Gm,:˜

Λ||1. Note that, || ˜

Gm,:˜

Λ||1=

||Qm,:||1for m= 1,2, . . . , p. Therefore, the range vector for

the separator outputs can be rewritten as

ˆ

R(o) = ||Q1,:||1||Q2,:||1... ||Qp,:||1.

Rewriting the equality (7) in terms of Q, we obtain

Js1(˜

W) = qdet(Qˆ

R˜sA

QT)1/K

Qp

m=1 ||Qm,:||1

.(20)

We note that for any ˜

Gwhose rows are not linearly indepen-

dent we have det Qˆ

R˜sA

QT= 0, therefore, corresponding

˜

Gcan not be global maxima of (7). Hence for any ˜

G

whose rows are linearly independent, assuming ˆ

R˜sK+P−1=

ˆ

R˜sA0, to complete Qinto a full rank square matrix we

introduce a (P−1)p×Ap matrix M=DP where D=

diag(a1, a2, . . . , a(P−1)p)is a full rank diagonal matrix and P

is a permutation matrix such that det M BM T= 1 where

we deﬁne B=ˆ

R˜sA−ˆ

R˜sA

QTQˆ

R˜sA

QT−1

Qˆ

R˜sA. This

yields,

det Q

Mˆ

R˜sAQTMT= det Qˆ

R˜sA

QT

det Mˆ

R˜sA−ˆ

R˜sA

QTQˆ

R˜sA

QT−1

Qˆ

R˜sAMT

= det Qˆ

R˜sA

QTdet MBM T= det Qˆ

R˜sA

QT.

We note that Q

Mˆ

R˜sAQTMT0and

MBM Tis the Schur complement of Qˆ

R˜sA

QT, there-

fore, MBM T0. We also note that det MBM T=

a2

1a2

2. . . a2

(P−1)pdet ([B]per)where [B]per has the chosen

rows and columns of Bdepending on the positions of

a1, a2, . . . , a(P−1)p. Hence by choosing appropriate values for

a1, a2, . . . , a(P−1)pwe can obviously introduce a matrix M

such that

det Q

Mˆ

R˜sAQTMT= det Qˆ

R˜sA

QT.

Using Hadamard’s Inequality [37] yields

det Q

Mˆ

R˜sAQTMT

≤

Kp

Y

m=1 ||Qm,:||2

2

(P−1)p

Y

n=1 ||Mn,:||2

2det( ˆ

R˜sA).(21)

Note that QKp

m=1 ||Qm,:||2

2=Qp

m=1 ||Qm,:||2

2Ksince Qis

block Toeplitz matrix. Hence,

qdet(Qˆ

R˜sA

QT)1/K

≤

p

Y

m=1 ||Qm,:||2!

(P−1)p

Y

n=1 ||Mn,:||2

1/K

det( ˆ

R˜sA)1/2K.

Therefore, we have

Js1(˜

W) = qdet(Qˆ

R˜sA

QT)1/K

Qp

m=1 ||Qm,:||1

≤Qp

m=1 ||Qm,:||2

Qp

m=1 ||Qm,:||1

(P−1)p

Y

n=1 ||Mn,:||2

1/K

det( ˆ

R˜sA)1/2K

≤

(P−1)p

Y

n=1 ||Mn,:||2

1/K

det( ˆ

R˜sA)1/2K,(22)

due to the ordering ||q||1≥ ||q||2for any q.

To achieve the equality in (22), the equalities ||Qm,:||1=

||Qm,:||2for m= 1,2,...p and the equality in (21) should

be achieved. The equalities ||Qm,:||1=||Qm,:||2for m=

1,2,...p are achieved if and only if the ﬁrst prows of Q

has only one non-zero element. Since ΓK(˜

G) = Q˘

Λ−1, this

implies that each row of ˜

Ghas only one non-zero element.

The inequality in (21) is achieved if and only if the rows of Q

are perpendicular to each other and to the rows of Mwhich

12

yields that the rows of ΓK(˜

G)are perpendicular to each other

and to the rows of M. Note that since K≥P, the structure of

ΓK(˜

G)guarantees that there is a block column which contains

G(0),G(1),...,G(P−1), therefore, the non-zero entries of

˜

Gwould not be in the same position with respect to mod p,

since otherwise Js1(˜

W)would be simply 0.

As a result, the maximum is achieved if and only if ˜

G

corresponds to perfect separator transfer matrix in the form

G(z) = diag(α1z−d1, α2z−d2, . . . , αpz−dp)Pwhere G(z)is

the Z-transform of the overall system function {G(l); l∈

{0, . . . , P −1}},αk’s are non-zero real scalings, and dk’s

are non-negative integer delays.

We note that due to the structure of the set of global maxima

of the objective function Js1(˜

W), the sources are recovered up

to scaling and order. These indeterminacy cannot be resolved

without any extra information in BSS problems.

Here, we point out that the blind source extraction problem

is a special case of the blind source separation problem.

Therefore, this proof can simply be also applied to the blind

source extraction method. However, we treat the blind source

extraction problem as a separate case to provide alternative

geometric intuition.

D. Analysis of the Family of Objective Functions (Js2,r)

Before analysing this family of objective functions for some

special rvalues, similar to the proof of Theorem 3, we can

rewrite (8) in terms of Qand obtain

Js2,r(˜

W) = qdet(Qˆ

R˜sA

QT)1/K

||Q1,:||1||Q2,:||1... ||Qp,:||1T

p

r

.

Following similar steps, by modifying (22), we can obtain the

corresponding inequality

Js2,r(˜

W)≤Qp

m=1 ||Qm,:||2

||Q1,:||1||Q2,:||1... ||Qp,:||1T

p

r

(P−1)p

Y

n=1 ||Mn,:||2

1/K

det( ˆ

R˜sA)1/2K.

The results of analysing this family of objective functions, for

some special rvalues:

•r= 1 Case: In this case, we have

kQ1,:k1kQ2,:k1. . . kQp,:k1T

p

1

= p

X

m=1 kQm,:k1!p

≥pp

p

Y

m=1 kQm,:k1,

where the inequality comes from Arithmetic-Geometric-

Mean-Inequality, and the equality is achieved if and only

if all the rows Qhave the same 1-norm. Hence, we have

Js2,1(˜

W)≤Qp

m=1 ||Qm,:||2

ppQp

m=1 kQm,:k1

(P−1)p

Y

n=1 ||Mn,:||2

1

K

det( ˆ

R˜sA)1

2K≤1

pp

(P−1)p

Y

n=1 ||Mn,:||2

1

K

det( ˆ

R˜sA)1

2K.

As a result, Qis a global maximum of Js2,1(˜

W)if and

only if it is a perfect separator matrix of the form

Q=kPdiag(ρ),

where kis a non-zero value, ρ∈ {−1,1}pand Pis a

permutation matrix. This implies ˜

Gis a global maximum

of Js2,1(˜

W)if and only if the corresponding form is

satisﬁed

ΓK(˜

G) = kP˘

Λ−1diag(ρ).

Therefore, the global maxima of the objective function

Js2,1corresponds to a subset of perfect separators.

•r= 2 Case: In this case, using the basic norm inequality

and Arithmetic-Geometric-Mean-Inequality, for any x∈

Rp, we have

(||x||2)p≥1

√p||x||1p

≥pp/2

p

Y

m=1 |xm|

where the equality is achieved if and only if all the

components of xare equal in magnitude. As a result,

this yields

Js2,2(˜

W)≤1

pp/2

(P−1)p

Y

n=1 ||Mn,:||2

1

K

det( ˆ

R˜sA)1

2K.

Similarly, Js2,2has the same set of global maxima as

Js2,1.

•r=∞Case: Following similar steps, using the ba-

sic norm inequality and Arithmetic-Geometric-Mean-

Inequality, for any x∈Rp, we have

(||x||∞)p≥1

p||x||1p

≥

p

Y

m=1 |xm|,

where the equality is achieved if and only if all the

components of xare equal in magnitude. Based on this

inequality, we obtain

Js2,∞(˜

W)≤

(P−1)p

Y

n=1 ||Mn,:||2

1

2K

det( ˆ

R˜sA)1

2K.

Therefore, Js2,∞also has same set of global optima as

Js2,1and Js2,2.

E. Proof of Theorem 4

We begin with observing that

R{o(k)}=

P−1

X

l=0

R{GT(l)}R{s(k+M−1−l)}

−I{GT(l)}I{s(k+M−1−l)},

I{o(k)}=

P−1

X

l=0

I{GT(l)}R{s(k+M−1−l)}

+R{GT(l)}I{s(k+M−1−l)}.

Deﬁning `

G=

R{G0} −I{G0}. . . R{GP−1} −I{GP−1}

I{G0}R{G0}. . . I{GP−1}R{GP−1}and

13

`sK+P−1(k)=[ R{sT(k)}I{sT(k)}. . .

R{sT(k−K−P+ 2)}I{sT(k−K−P+ 2)}]Tyields

`oK(k)=Γ2K(`

G)`sK+P−1(k). Thus,

ˆ

R`oK= Γ2K(`

G)`

Λˆ

R`sK+P−1

`

ΛTΓ2K(`

G)T,

where `

Λ = I⊗Λis the range matrix of `sK+P−1and ˆ

R`sK+P−1

is deﬁned as the sample covariance matrix of `sK+P−1.

Deﬁning `

Q= Γ2K(`

G)`

Λand following similar steps, we

can write Q2p

m=1 ˆ

R(`om) = Q2p

m=1 || `

Qm,:||1. Rewriting (12) in

terms of `

Qyields

Jcs1(˜

W) = rdet( `

Qˆ

R`sA

`

QT)!1/K

Q2p

m=1 || `

Qm,:||1

.

Note that we have the similar expression as (20). Hence, the

proof of Theorem 3 also applies here. Note that the structure

of Γ2K(`

G)implies that the non-zero entries of ˜

Gcan only be

real or purely imaginary. Therefore, the set of global maxima

for the objective function (12) corresponds to a subset of

complex perfect separators.

REFERENCES

[1] A. Hyv¨

arinen, J. Karhunen, and E. Oja, Independent Component Anal-

ysis. John Wiley and Sons Inc., 2001.

[2] P. Comon and C. Jutten, Handbook of Blind Source Separation: Inde-

pendent Component Analysis and Applications. Academic Press, 2010.

[3] P. Comon, “Independent component analysis, A new concept?” Signal

Processing, vol. 36, pp. 287–314, April 1994.

[4] A. Belouchrani, K. Abed-Meraim, J.-F. Cardoso, and E. Moulines, “A

blind source separation technique using second-order statistics,” Signal

Processing, IEEE Transactions on, vol. 45, no. 2, pp. 434–444, Feb

1997.

[5] M. Matsuoka, M. Ohaya, and M. Kawamoto, “A neural net for blind

separation of nonstationary signals,” Neural Networks, vol. 8, no. 3, pp.

411–419, March 1995.

[6] P. Georgiev, F. Theis, and A. Cichocki, “Sparse component analysis and

blind source separation of underdetermined mixtures,” IEEE Transac-

tions on Neural Networks, vol. 16, no. 4, pp. 992–996, July 2005.

[7] A. J. van der Veen and A. Paulraj, “An analytical constant modulus

algorithm,” IEEE Transactions on Signal Processing, vol. 44, no. 5, pp.

1136–1155, May 1996.

[8] P. Jallon, A. Chevreuil, and P. Loubaton, “Separation of digital com-

munication mixtures with the CMA: Case of unknown symbol rates,”

Signal Processing, vol. 90, no. 9, pp. 2633–2647, September 2010.

[9] S. Talwar, M. Viberg, and A. Paulraj, “Blind separation of synchronous

co-channel digital signals using an antenna array. I. algorithms,” IEEE

Transactions on Signal Processing, vol. 44, no. 5, pp. 1184–1197, May

1996.

[10] C. G. Puntonet and A. Prieto, “Neural net approach for blind separation

of sources based on geometric properties,” Neurocomputing, no. 18, pp.

141164, 1998.

[11] D.-T. Pham, “Blind separation of instantaneous mixtures of sources

based on order statistics,” IEEE Trans. on Signal Processing, vol. 48,

pp. 363–375, February 2000.

[12] S. Cruces and I. Duran, “The minimum support criterion for blind source

extraction: A limiting case of the strengthened young’s inequality,”

Lecture Notes in Computer Science, pp. 57–64, September 2004.

[13] F. Vrins, M. Verleysen, and C. Jutten, “SWM: A class of convex

contrasts for source separation,” Proceedings of IEEE ICASSP 2005,

vol. V, pp. 161–164, March 2005.

[14] F. Vrins, J. A. Lee, and M. Verleysen, “A minimum-range approach to

blind extraction of bounded sources,” IEEE Trans. on Neural Networks,

vol. 18, no. 3, pp. 809–822, May 2007.

[15] F. Vrins and D. Pham, “Minimum range approach to blind partial

simultaneous separation of bounded sources: Contrast and discriminacy

properties,” Neurocomputing, vol. 70, pp. 1207–1214, March 2007.

[16] S. Vembu, S. Verdu, R. Kennedy, and W. Sethares, “Convex cost

functions in blind equalization,” IEEE Trans. on Signal Processing,

vol. 42, pp. 1952–1960, August 1994.

[17] A. T. Erdogan and C. Kizilkale, “Fast and low complexity blind equal-

ization via subgradient projections,” IEEE Trans. on Signal Processing,

vol. 53, pp. 2513–2524, July 2005.

[18] A. T. Erdogan, “A blind separation approach for magnitude bounded

sources,” Proceedings of IEEE ICASSP 2005, Philadelphia, vol. V, pp.

177–180, March 2005.

[19] A. T. Erdogan, “A simple geometric blind source separation method for

bounded magnitude sources,” IEEE Trans. on Signal Processing, vol. 54,

pp. 438–449, February 2006.

[20] A. T. Erdogan, “Globally convergent deﬂationary instantaneous blind

source separation algorithm for digital communication signals,” IEEE

Trans. on Signal Processing, vol. 55, no. 5, pp. 2182–2192, May 2007.

[21] A. T. Erdogan, “Adaptive algorithm for the blind separation of sources

with ﬁnite support,” Proceedings of EUSIPCO 2008, Lausanne, August

2008.

[22] S. Cruces, “Bounded component analysis of linear mixtures: A criterion

for minimum convex perimeter,” IEEE Trans. on Signal Process., vol. 58,

no. 4, pp. 2141–2154, April 2010.

[23] P. Aguilera, S. Cruces, I. Duran, A. Sarmiento, and D. Mandic, “Blind

separation of dependent sources with a bounded component analysis

deﬂationary algorithm,” IEEE Signal Processing Letters, vol. 20, no. 7,

pp. 709–712, July 2013.

[24] A. T. Erdogan, “On the convergence of symmetrically orthogonal-

ized bounded component analysis algorithms for uncorrelated source

separation,” IEEE Transactions on Signal Processing, pp. 6058–6063,

November 2012.

[25] A. T. Erdogan, “A class of bounded component analysis algorithms

for the separation of both independent and dependent sources,” IEEE

Transactions on Signal Processing, vol. 61, no. 22, pp. 5730–5743,

November 2013.

[26] H. A. Inan and A. T. Erdogan, “A bounded component analysis approach

for the separation of convolutive mixtures of dependent and independent

sources,” IEEE International Conference on Acoustics, Speech and

Signal Processing, May 2013.

[27] H. A. Inan and A. T. Erdogan, “Convolutive Bounded Component Anal-

ysis Algorithms for Independent and Dependent Source Separation”,

IEEE Transactions on Neural Networks and Learning Systems, to appear.

[28] M. D. Craig, “Minimum-volume transforms for remotely sensed data”,

Geoscience and Remote Sensing, IEEE Transactions on, vol. 32, no. 3,

pp. 542–552, May 1994.

[29] L. Miao and H. Qi, “Endmember Extraction From Highly Mixed Data

Using Minimum Volume Constrained Nonnegative Matrix Factoriza-

tion,” Geoscience and Remote Sensing, IEEE Transactions on, vol. 45,

no. 3, pp. 765–777, March 2007.

[30] W. S. B. Ouedraogo, A. Souloumiac, M. Jaidane, and C. Jutten, “Sim-

plicial Cone Shrinking Algorithm for Unmixing Nonnegative Sources,”

Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE Inter-

national Conference on, pp. 2405–2408, March 2012.

[31] Y. Inouye and R.-W. Liu, “A system-theoretic foundation for blind

equalization of an FIR MIMO channel system,” IEEE Trans. on CAS-I:

Fund. Th. and Apps., vol. 49, no. 4, pp. 425–436, April 2002.

[32] C. Simon, P. Loubaton, and C. Jutten, “Separation of a class of

convolutive mixtures: a contrast function approach,” Signal Processing,

vol. 81, no. 4, pp. 883–887, 2001.

[33] M. Castella and E. Moreau, “New kurtosis optimization schemes for

MISO equalization,” IEEE Transactions on Signal Processing, vol. 60,

no. 3, pp. 1319–1330, 2012.

[34] P. Jallon, M. Castella, and R. Dubroca. (2011) Matlab toolbox

for separation of convolutive mixtures. Available: http://www-public.

int-evry.fr/∼castella/toolbox/

[35] L. De Lathauwer, “Algebraic Techniques for the Blind Deconvolution

of Constant Modulus Signals,” Proc. 12th European Signal Processing

Conference (EUSIPCO 2004), pp. 225–228, September 2004.

[36] B. Grunbaum, V. Klee, A. Perles, and G. Shephard, Convex Polytopes.

John Wiley & Sons, Incorporated, 1967.

[37] D. Garling, Inequalities: a journey into linear analysis. Cambridge

University Press, 2007.