Transient and Steady-State Analysis of the Affine Combination of Two Adaptive Filters

Abstract

In this paper, we propose an approach to the transient and steady-state analysis of the affine combination of one fast and one slow adaptive filters. The theoretical models are based on expressions for the excess mean-square error (EMSE) and cross-EMSE of the component filters, which allows their application to different combinations of algorithms, such as least mean-squares (LMS), normalized LMS (NLMS), and constant modulus algorithm (CMA), considering white or colored inputs and stationary or nonstationary environments. Since the desired universal behavior of the combination depends on the correct estimation of the mixing parameter at every instant, its adaptation is also taken into account in the transient analysis. Furthermore, we propose normalized algorithms for the adaptation of the mixing parameter that exhibit good performance. Good agreement between analysis and simulation results is always observed.

Full text

Page 1

SUBMITTED TO THE IEEE TRANSACTIONS ON SIGNAL PROCESSING 20091
Transient and steady-state analysis of the affine
combination of two adaptive filters
Renato Candido, Magno T. M. Silva, Member, IEEE, and V´ ıtor H. Nascimento, Member, IEEE
Abstract—In this paper, we propose an approach to the
transient and steady-state analysis of the affine combination of
one fast and one slow adaptive filters. The theoretical models
are based on expressions for the excess mean-square error
(EMSE) and cross-EMSE of the component filters, which allows
their application to different combinations of algorithms, such
as least mean-squares (LMS), normalized LMS (NLMS), and
constant modulus algorithm (CMA), considering white or colored
inputs and stationary or nonstationary environments. Since the
desired universal behavior of the combination depends on the
correct estimation of the mixing parameter at every instant, its
adaptation is also taken into account in the transient analysis.
Furthermore, we propose normalized algorithms for the adapta-
tion of the mixing parameter that exhibit good performance.
Good agreement between analysis and simulation results is
always observed.
Index Terms—Adaptive filters, affine combination, tracking,
transient analysis, least mean square methods, unsupervised
learning.
I. INTRODUCTION
C
these schemes, the convex combination of two fixed step-
size adaptive filters has received attention due to its relative
simplicity, and the proof that it is universal in steady-state,
i.e., the combined estimate is at least as good as the best of
the component filters [8].
Convex combination schemes were proposed to improve the
fundamental tradeoff between convergence rate and steady-
state excess mean-square error (EMSE) in adaptive filters
[16]–[19]. Furthermore, such schemes have been exploited in
nonstationary environments to improve tracking performance,
considering, e.g, the algorithm proposed in [8] or the combina-
tion of algorithms with different tracking capabilities of [10].
The correct adjustment of the step-size for the updating of
the mixing parameter depends on some characteristics of the
filtering scenario, such as the input signal and additive noise
powers, or the step-sizes of the adaptive filters considered in
the combination. This problem was addressed in [9], where
a novel normalized scheme was proposed. It was shown that
OMBINATION schemes constitute an interesting way
to improve adaptive filter performance [3]–[15]. Among
This work was partly supported by FAPESP under Grants 2008/00773-1 and
2008/04828-5, and by CNPq under Grants 302633/2008-1 and 303361/2004-
2. Some preliminary parts of this work appeared as conference papers [1],
[2].
The authors are with the Electronic Systems Engineering Department,
Escola Polit´ ecnica, University of S˜ ao Paulo, S˜ ao Paulo, SP, Brazil, e-mails:
{renatocan, magno, vitor}@lps.usp.br, ph. +55-11-3091-5606, fax: +55-11-
3091-5718.
SP EDICS: 1.ASP-ANAL, 2.ASP-APPL, 3.SPC-BLND
Corresponding author: Renato Candido.
the new update rule preserves the good features of the existing
scheme and is more robust to changes in the filtering scenario.
Using a similar approach to that of [8], the authors of
[12] proposed an affine combination of two least mean-square
(LMS) algorithms, where the condition on the mixing parame-
ter is relaxed, allowing it to be negative. Thus, this scheme can
be interpreted as a generalization of the convex combination
since the mixing parameter is not restricted to the interval
[0, 1]. This approach allows for smaller EMSE in theory,
but suffers from larger gradient noise in some situations.
Under certain conditions, the optimum mixing parameter was
proved to be negative in steady-state. Although the optimal
linear combiner is unrealizable, two realizable algorithms were
introduced. One is based on a stochastic gradient search and
is referred to here as η-LMS algorithm. The other is based on
the ratio of the average error powers from each individual
adaptive filter. Under some circumstances, both algorithms
present performance close to the optimum. In the analysis
of [12], white Gaussian inputs and stationary environments
are assumed. Furthermore, the behavior of the mean-square
deviation is studied only after the fast filter has converged but
the slow filter has not yet converged.
Similarly to the convex combination, the correct adjustment
of the step-size for the updating of the mixing parameter
in the affine combination, denoted by µη, depends on some
characteristics of the filtering scenario. Hence, the desired
universal behavior of the affine combination cannot always be
ensured. To illustrate, Fig. 1 shows the EMSE as a function of
time for two LMS filters with step-sizes µ1=0.01 (µ1-LMS)
and µ2= 0.001 (µ2-LMS), and their affine combination. In
this scenario, it is necessary to use a high value for the step-
size of the η-LMS algorithm (e.g., µη= 3) in order to enable
the switching from the slow filter to the fast one. A large
value of µηmay, however, cause instability during the initial
convergence of the algorithm, thus [12] constrains η(n) to be
less than or equal to 1. Unfortunately, even with this constraint,
the higher the step-size µη, the higher the variance of the
mixing parameter during the initial iterations. Therefore, the
combination performance deviates from universal, as shown
in Fig. 1-(a). On the other hand, if the step-size is small
(e.g., µη= 0.1), the combination performs better in the initial
iterations but does not switch as fast as needed from the slow
filter to the fast one, as shown in Fig. 1-(b). In fact, we will
show that µηdepends on α(n) ? E{[y1(n)−y2(n)]2}, where
E{·} represents the expectation operation and yi(n), i = 1,2
are the outputs of the filters. As we can observe in Fig. 1-(c),
α(n) suffers a large variation during the adaptation of the
filters. Thus, a transient analysis and alternative algorithms to

Page 2

2SUBMITTED TO THE IEEE TRANSACTIONS ON SIGNAL PROCESSING 2009
adapt the mixing parameter are two important key issues for
the practical application of the affine combination of adaptive
filters.
EMSE (dB)
(a)
µη= 3
µ1-LMS
µ2-LMS
Combination
0
1234
5
678
-60
-40
-20
0
EMSE (dB)
(b)
µη= 0.1
0123
4
56
7
8
-60
-40
-20
0
iterations
×104
α(n) (dB)
(c)
0
1234567
8
-60
-40
-20
0
Fig. 1.
µη = 3.0; b) µη = 0.1; c) ensemble average of [y1(n) − y2(n)]2;
µ1 = 0.01, µ2 = 0.001, M = 7, identification of the system wo =
[0.9003 − 0.5377 0.2137 − 0.0280 0.7826 0.5242 − 0.0871]T, σ2
with variance σ2
EMSE for µ1-LMS, µ2-LMS and their affine combination a)
v= 0.01, white input
u= 1/7; 500 independent runs.
A. Contributions and organization of the paper
The present paper extends previous results in four ways:
1) providing a steady-state analysis for the optimum affine
combination of adaptive filters, which is valid for white
or colored inputs, stationary or nonstationary environ-
ments, and combinations based on different algorithms,
such as LMS, normalized LMS (NLMS), and the con-
stant modulus algorithm (CMA);
2) proposing a simple geometrical interpretation to explain
the behavior of the affine combination;
3) providing a transient analysis of the combination, taking
into account the adaptation of the component filters and
also the adaptation of the mixing parameter with the
η-LMS algorithm;
4) using the results of the transient analysis to facilitate the
adjustment of the free parameters of the scheme and to
propose two normalized algorithms to update the mixing
parameter.
To the best of our knowledge, all these are novel contributions.
The paper is organized as follows. In the next section, we
describe the affine combination of two adaptive filters for both
supervised (LMS and NLMS) and blind (CMA) algorithms.
In Section III, analytical expressions for the optimum mixing
parameter and the optimum EMSE of the combination are
obtained. In the steady-state analysis of Section IV, the results
of Section III are particularized for optimum combinations of
two LMS filters, two NLMS filters, and two CMA equaliz-
ers, considering stationary and nonstationary environments. In
Section V, transient analyses taking into account realizable
schemes are presented. Initially, we obtain an analytical ex-
pression for the EMSE of the combination, which depends
on the transient models of the combined algorithms and also
on the algorithm used to adapt the mixing parameter. We
summarize results for the transient analysis of LMS, NLMS,
and CMA. Then, in Section V-A, we present the transient anal-
ysis of the η-LMS algorithm. The resulting analysis suggests
the normalization procedure presented in Section V-B and an
algorithm with partial instantaneous normalization, proposed
and analyzed in Section V-C. Comparisons between analytical
and experimental results are shown through simulations in
Section VI. Section VII provides a summary of the main
conclusions of the paper.
II. PROBLEM FORMULATION
This section is divided into three parts. We first describe the
affine combination of one fast and one slow supervised algo-
rithms. In the sequel, the combination of two CMA equalizers
is presented. Then, we propose a common formulation for the
affine combination of supervised (LMS and NLMS) or blind
(CMA) algorithms.
A. Combination of supervised algorithms
The linear combination of two supervised adaptive filters
is depicted in Fig. 2, where the filter weights are adjusted
to minimize the mean-square error cost function, obtaining at
the output an estimate of the given “desired signal” d(n). The
output of the overall filter is given by
y(n) = η(n)y1(n) + [1 − η(n)]y2(n),
(1)
where η(n) is the mixing parameter and yi(n), i = 1,2 are the
outputs of two transversal filters, i.e., yi(n) = uT(n)wi(n−1).
The superscript T denotes transposition, wi(n − 1), i = 1,2
represent the length-M coefficient column-vectors character-
izing the component filters, and u(n) is their common input
regressor column-vector.
Fig. 2.Linear combination of two supervised adaptive filters.
We focus on the affine combination of two algorithms of
the following general class
wi(n) = wi(n − 1) + ρi(n)u(n)ei(n),
(2)
where ρi(n) is a step-size and ei(n) is the estimation error.
Many algorithms can be written as in (2), by proper choices

Page 3

CANDIDO, SILVA, AND NASCIMENTO: TRANSIENT AND STEADY-STATE ANALYSIS OF THE AFFINE COMBINATION OF TWO ADAPTIVE FILTERS3
of ρi(n) and ei(n). For some algorithms ρi(n) can even be a
matrix, as is the case of the recursive-least squares (RLS) algo-
rithm, where ρi(n) is an estimate of the inverse autocorrelation
matrix of the input signal. In supervised adaptive filtering, a
“desired signal” d(n) is available such that
ei(n) = d(n) − yi(n)
(3)
and a linear regression model holds, i.e.,
d(n) = uT(n)wo(n − 1) + v(n)
(4)
with wo(n − 1) being the time-variant optimal solution and
v(n) a zero-mean random process uncorrelated with u(n),
whose variance is denoted by σ2
to make performance analyses more tractable, the sequences
{u(n)} and {v(n)} are assumed stationary and we will use
the common assumption that v(n) is independent of u(n)
(not only uncorrelated) [17, Sec. 6.2.1]. Defining the weight-
error vectors ? wi(n) = wo(n) − wi(n), the a priori errors
we find that
ei(n) = ea,i(n) + v(n).
v = E{v2(n)}. In order
ea,i(n) = uT(n)? wi(n − 1), and using the linear model (4),
An important consequence of this model is that v(k) will be
independent of all wi(j), ? wi(j), and ea,i(k), i = 1,2, j < k,
Considering the combination of two LMS filters and
the minimization of the overall instantaneous square error
e2(n) = [d(n) − y(n)]2, [12] proposed the following gradient-
based algorithm
(5)
for any particular time instant k [17, Lemma 6.2.1].
η(n + 1) = η(n) + µηe(n)[y1(n) − y2(n)].
(6)
To obtain a tradeoff between stability of this recursion and
the algorithm’s tracking capability in the initial phase of
adaptation, η(n) in (6) must be constrained to be less than
or equal to 1 for all n. We should remark that this kind of
constraint is not needed in the normalized algorithms proposed
in Sections V-B and V-C.
B. Combination of blind algorithms
Fig. 3 shows a simplified communications system with a
combination of two blind equalizers. In this case, the signal
a(n), assumed i.i.d. (independent and identically distributed)
and non Gaussian, is transmitted through an unknown channel,
whose model is constituted by an FIR (finite impulse response)
filter and additive white Gaussian noise. From the received
signal u(n) and the known statistical properties of the trans-
mitted signal, the blind equalizer must mitigate the channel
effects and recover the signal a(n) for some delay τd. We
also assume that the equalization algorithms are implemented
in T/2-fractionally spaced form, due to its inherent advantages
(see, e.g., [20]–[23] and the references therein). This type of
implementation is widely used in the literature since it ensures
perfect equalization in a noise-free environment, under certain
well-known conditions. The output of the overall equalizer of
Fig. 3 is also given by (1).
Algorithms based on the constant modulus cost function
[24], [25] define the “estimation error” as
ei(n) = [r − y2
i(n)]yi(n),
(7)
Fig. 3.
blind equalizers.
Simplified communications system with a linear combination of two
where r = E{a4(n)}/E{a2(n)}. Using (7), CMA can also
be written as in (2). However, given the nonlinear nature of
CMA, additional assumptions are necessary to obtain a model
as simple as (5): essentially, large signal-to-noise ratio, circu-
lar symmetry of the transmitted constellation, and an initial
condition close to the zero-forcing solution (see Appendix A).
These assumptions were used in [10] and [26] to obtain simple
linear models that capture the behavior of CMA close to an
optimum solution. Thus, (7) was approximated by
ei(n) ≈ γ(n)ea,i(n) + β(n),
(8)
where
γ(n) ? 3a2(n − τd) − r
(9)
and
β(n) ? ra(n − τd) − a3(n − τd).
(10)
The variable β(n) is identically zero for constant-modulus
constellations, so the variability in the modulus of a(n) (as
measured by β(n)) plays the role of measurement noise for
constant-modulus based algorithms. Model (8) was proposed
in [10] to study convex combinations of constant-modulus-
based algorithms and extended in [26] to obtain explicit
stability conditions for CMA. The main assumptions and the
derivation of this model are summarized in Appendix A.
To update the mixing parameter in order to combine two
CMA equalizers, we could use a gradient rule to minimize the
instantaneous constant-modulus costˆJcm(n) = [r − y2(n)]2,
as considered in the convex combination of [27]. However,
we observed through simulations that the resulting algorithm
does not always ensure the desired universal behavior of
the combination, specially for nonconstant modulus signals.
Thus, we propose a stochastic gradient algorithm to minimize
the instantaneous square decision errorˆJd(n) = e2
ed(n) ? ˆ a(n−τd)−y(n) and ˆ a(n−τd) is the estimate of the
transmitted signal at the output of the decision device. This
results in the following update equation
d(n), where
η(n + 1) = η(n) + µηed(n)[y1(n) − y2(n)].
(11)
We observed through simulations that the decision-error-based
adaptation ensures a more adequate behavior than that of the

Page 4

4SUBMITTED TO THE IEEE TRANSACTIONS ON SIGNAL PROCESSING 2009
constant-modulus-based adaptation, even in presence of noise
and/or when both component filters are far from convergence.
Assuming that ˆ a(n−τd) = a(n−τd) and that the optimal
solution achieves perfect equalization (see Assumption B1 in
Appendix A), the minimization of Jd(n) is equivalent to the
minimization of the square a priori error, since under these
assumptions ed(n) ≈ ea(n).
C. A common formulation
Comparing (8) to (5), we can write the following general
expression
ei(n) = κ(n)ea,i(n) + ϕ(n), i = 1,2,
(12)
where κ = 1 and ϕ(n) = v(n) for a supervised algorithm or
κ(n) = γ(n) and ϕ(n) = β(n) for a blind one. In both cases
E{ϕ(n)} = 0. This model also holds for the overall scheme,
i.e.,
e(n) = κ(n)ea(n) + ϕ(n),
(13)
where
filter:
or
algorithms, and ea(n) is the a priori error of the overall
scheme. It should be noticed that (12) and (13) are
approximations in the blind case. For the sake of simplicity,
we use the equality sign here and in the expressions derived
from (12) and (13).
The supervised LMS and NLMS algorithms and the blind
CMA employ the step-sizes ρi(n) and the estimation errors
ei(n) as in Table I, where ǫN is a regularization factor and
? · ? represents the Euclidean norm. The models for the errors
ei(n) of these algorithms are also shown in this table for
convenient reference. The step-size interval which ensures the
convergence and stability is different for each algorithm. For
the LMS and NLMS algorithms, the step-size intervals are
well-known in the literature [16], [17], whereas for CMA, the
derivation of this interval was shown recently in [26].
e(n)
e(n) = d(n) − y(n)
e(n) = [r − y2(n)]y(n)
representsthe errorof the combined
algorithmsfor
for
supervised
constant-modulus-based
TABLE I
PARAMETERS OF THE CONSIDERED ALGORITHMS AND ERROR MODELS.
Alg.
ρ(n)ei(n)
Model for ei(n)
LMS
µi
µi
NLMS
ǫN+ ?u(n)?2
d(n) − yi(n)ea,i(n) + v(n)
CMA
µi
[r − y2
i(n)]yi(n)γ(n)ea,i(n) + β(n)
Using model (5) in the supervised case, and the fact that
ed(n) ≈ ea(n) in the blind case, we can write a general
expression for updating the mixing parameter, i.e.,
η(n + 1) = η(n) + µηeg(n)[y1(n) − y2(n)],
(14)
where
eg(n) = ea(n) + b(n)
(15)
and b(n) = v(n) for the combination of supervised algorithms
or b(n) = 0 for the combination of constant-modulus-based
algorithms. In both cases, η(n) is constrained to be less than
or equal to 1 for all n [12]. Algorithm (14) is denoted here
by η-LMS.
To close this section, it is important to observe that:
1) In order to simplify the arguments, we assume that all
the quantities are real. In the case of blind equalization
of complex constellations, complex extensions may be
developed, provided signal circularity conditions are
satisfied [28];
2) The analyses provided here can be extended straightfor-
wardly to the affine combination of two RLS filters [1],
of two Shalvi-Weinstein equalizers [10], [29], and also
to the combination of algorithms of different families,
as is the case of the combination of one LMS with one
RLS or of the combination of one CMA with one Shalvi-
Weinstein algorithm [10];
3) Besides the η-LMS algorithm, [12] proposed a scheme
based on error powers to update the mixing parameter.
Although this scheme also presents performance close to
the optimum under certain circumstances, its structure is
significantly different from that of η-LMS, so we leave
its analysis for a future work.
III. THE OPTIMUM MIXING PARAMETER AND EMSE
An analytical expression for the optimum mixing parameter
ηo(n) can be obtained equating to zero the expected value of
the gradient used to update η(n) in (14), i.e.,
E{eg(n)[y1(n) − y2(n)]} = 0.
(16)
The error eg(n) in (16) can be rewritten as a function of the
a priori errors ea,i(n), i = 1,2, as follows.
Using (1), (12), and (13), the a priori error ea(n) of the
overall scheme can be written as
ea(n) = η(n)ea,1(n) + [1 − η(n)]ea,2(n)
= ea,2(n) + η(n)[ea,1(n) − ea,2(n)].
(17)
Replacing (17) in (15), and remarking that y1(n) − y2(n) =
ea,2(n) − ea,1(n), (16) can be rewritten as
E?e2
+ E{b(n)[ea,2(n) − ea,1(n)]} = 0.
a,2(n) − ea,1(n)ea,2(n)?
− E?ηo(n)[ea,2(n) − ea,1(n)]2?
In the blind case, b(n) = 0 and in the supervised case,
b(n) = v(n), which is assumed independent of ea,i(n),
i = 1,2. Hence, in both cases the third term on the l.h.s.
of (18) is equal to zero.
To proceed, we remark that the EMSE of the component fil-
ters and the cross-EMSE can be calculated [8], respectively as
(18)
ζii(n) ? E{e2
ζ12(n) ? E{ea,1(n)ea,2(n)}.
a,i(n)}, i = 1,2, and(19)
(20)
Introducing the differences
∆ζii(n) ? ζii(n) − ζ12(n), i = 1,2,
(21)

Page 5

CANDIDO, SILVA, AND NASCIMENTO: TRANSIENT AND STEADY-STATE ANALYSIS OF THE AFFINE COMBINATION OF TWO ADAPTIVE FILTERS5
and using (19)-(21) in (18), we arrive at
ηo(n) =
∆ζ22(n)
∆ζ11(n) + ∆ζ22(n).
(22)
A similar expression was also obtained in [8, Eq.(29)] for the
convex combination of two LMS filters at the steady-state. We
should notice that (22) is more general: it holds for all n ≥ 0
(not only at the steady-state) and the mixing parameter is not
restricted to the interval [0, 1].
Defining the EMSE of the overall combined scheme as
ζ(n) = E{e2
a(n)},
(23)
we now obtain an analytical expression for its optimum value.
By squaring both sides of (17) with η(n) = ηo(n) and taking
expectations, we arrive at
E{e2
a(n)}=η2
o(n)E{e2
+ 2ηo(n)[1−ηo(n)]E{ea,1(n)ea,2(n)}.
a,1(n)}+[1−ηo(n)]2E{e2
a,2(n)}
(24)
Using (19)-(22) in (24), we obtain
ζo(n) = ζ22(n) − ηo(n)∆ζ22(n).
(25)
After some algebraic manipulations, (25) can be rewritten as
ζo(n) = ζ12(n) +
∆ζ11(n)∆ζ22(n)
∆ζ11(n) + ∆ζ22(n).
(26)
This expression was obtained in [8, Eq. (33)] for the convex
combination of two LMS filters at the steady-state, but again
it also holds for all n ≥ 0.
As already mentioned in [8], (22) and (26) hold for the
combination of any two algorithms that satisfy (12). The
values of ∆ζii(n), i = 1,2 however do depend on the
actual algorithms that are being combined. Thus, provided
approximations for ζij(n), i,j = 1,2 are available, (22) and
(26) can be applied to the affine combination of different
algorithms, including combinations of algorithms of different
families.
IV. STEADY-STATE ANALYSIS OF THE OPTIMUM COMBINER
In this section, the optimum mixing parameter and the opti-
mum EMSE of the combination, given respectively by expres-
sions (22) and (26), are particularized for the combination of
two LMS filters, two NLMS filters, and two CMA equalizers
in steady-state for stationary and nonstationary environments.
We do not rederive the steady-state expressions for ζij(∞),
i,j = 1,2 here, only use the best approximations from the
literature. As in [12], we assume that the algorithm which
updates the mixing parameter is able to achieve the optimum
value in steady-state. Realizable schemes for adaptation of
η(n) are taken into account in Section V.
We assume that in a nonstationary environment, the varia-
tion in the optimal solution wofollows a random-walk model
[17, p. 359], that is,
wo(n) = wo(n − 1) + q(n).
(27)
In this model, q(n) is an i.i.d. vector with positive-definite
autocorrelation matrix Q = E{q(n)qT(n)}, independent of
the initial conditions {wo(−1),w(−1),η(−1)} and of {u(l)}
for all l [17, Sec. 7.4]. In supervised filtering, q(n) is also
assumed independent of the desired response {d(l)} for all
l < n. In blind equalization, wo(n) represents the zero-forcing
solution and q(n) models the channel variation.
Table II lists analytical expressions of ζ12(∞) for some
pairs of filters. Expressions for ζii(∞) can be obtained from
Table II making µ1= µ2. Details about the derivation of these
expressions can be found in [8], [10], [13] for the cross-terms
and in [16], [17], [23], [28], [30]–[34] for the case µ1= µ2. In
this table, R ? E{u(n)uT(n)} is the autocorrelation matrix of
the input signal, Tr(A) stands for the trace of matrix A, and
νu? E??u(n)?−2?. For Gaussian inputs and large number
with σ2
ξ, which appear in the expression for the EMSE of CMA,
depend on statistics of the transmitted signal and are defined
in Appendix A.
of coefficients, νu can be approximated by 1/[σ2
u= E{u2(n)} [13], [35]. The constants σ2
u(M −2)]
β, ¯ γ, and
TABLE II
ANALYTICAL EXPRESSIONS FOR THE STEADY-STATE CROSS-EMSE OF
THE CONSIDERED COMBINATIONS.
Combination
ζ12(∞)
µ1-LMS and µ2-LMS
µ1µ2σ2
µ1+ µ2− µ1µ2Tr(R)
Tr(R)?µ1µ2σ2
µ1+µ2−µ1µ2
µ1µ2σ2
βTr(R) + Tr(Q)
¯ γ(µ1+ µ2) − µ1µ2Tr(R)ξ
vTr(R) + Tr(Q)
µ1-NLMS and µ2-NLMS
vνu+Tr(Q)?
µ1-CMA and µ2-CMA
A. Stationary environments
Replacing the expressions of Table II with Q = 0 in (22)
and (26), we obtain analytical expressions for the steady-
state optimum mixing parameter ηo(∞) and for the steady-
state optimum EMSE ζo(∞) in stationary environments. The
resulting expressions are shown in Table III, where we defined
δ ? µ2/µ1 with 0 < δ < 1. It is worth to notice that the
second filter of the combination is always assumed to be the
slower filter (µ2< µ1) which consequently presents the lower
steady-state EMSE in a stationary environment.
TABLE III
ANALYTICAL EXPRESSIONS FOR ηo(∞) AND ζo(∞).
Combination
ηo(∞)ζo(∞)
µ2σ2
δ+1−µ2Tr(R)
?Tr(R)µ2σ2
µ2σ2
βTr(R)
(δ+1)¯ γ−µ2Tr(R)ξ
(µ1and µ2)-LMS
δ[2 − µ1Tr(R)]
2(δ − 1)
1
2
?
vTr(R)
?
(µ1and µ2)-NLMS
δ[2 − µ1]
2(δ − 1)
1
2
vνu
δ + 1 − µ2
?
(µ1and µ2)-CMA
δ[2−µ1Tr(R)ξ(¯ γ)−1]
2(δ − 1)
1
2
??
The expressions of Table III show two interesting properties:
i) ηo(∞) is negative for all considered combinations,
which can be verified through the stability conditions of
the algorithms. To ensure the stability of µ1-LMS and
µ1-NLMS, the step-sizes should be chosen respectively
in the following ranges 0 < µ1 < 2/Tr(R) and

Page 6

6SUBMITTED TO THE IEEE TRANSACTIONS ON SIGNAL PROCESSING 2009
0 < µ1 < 2 [19]. In the case of µ1-CMA, assuming
model (8), it was shown in [26, Eq. (14)] that the range
of step-sizes 0 < µ1< 2¯ γ/[3Tr(R)ξ] guarantees good
performance. Choosing the step-sizes in these ranges,
we can verify from the expressions of Table III that
ηo(∞) < 0.
ii) δ ≈ 1 yields ζo(∞) ≈ ζ2(∞)/2. Since ζ2(∞) < ζ1(∞)
for all combinations, the affine combination provides a
3dB gain in relation to the best component filter. In this
case, ηo(∞)→−∞.
Property i) was observed in [12] for the combination of
two LMS filters, assuming Gaussian and white inputs, and
additional assumptions equivalent to choosing the LMS
step-size for maximum convergence speed. If we consider
µ1= 1/Tr(R) in the expression of Table III, we recover the
result of [12, Eq.(26)]. The same property was observed in [13]
for the affine combination of two NLMS filters, also assuming
white and Gaussian inputs.
An intuitive explanation for Property ii) can be found as
follows. Using (12), the overall steady-state error is written as
e(n)=e2(n)
? ?? ?
dη(n)
+η(n)κ(n)[w2(n)−w1(n)]Tu(n)
????
−uη(n)
.
(28)
From the point of view of the computation of η(n), dη(n)
represents the signal which has to be estimated, and uη(n)
plays the role of input signal. Assuming that wi(n), i = 1,2
vary slowly compared to η(n), (28) has a simple geometric
interpretation as shown in Fig. 4. The affine combination seeks
the best weight vector in the line w2+ η(w1− w2). In
Fig. 4-(a), the best linear combination of w1 and w2 is w.
In the case of close step-sizes, we also have close coefficient
vectors in steady-state1, i.e., w1≈ w2(Fig. 4-(b)), and η has
to assume a large value to take the combined vector close
to w, since the input signal uη(n) depends on the difference
between w1and w2. Thus, if (w1− w2) → 0, |η| → ∞.
Fig. 4. Geometric interpretation of the affine combination.
B. Nonstationary environments
In a nonstationary environment, the largest EMSE reduction
of the affine combination in relation to its components occurs
when ζ11(∞) ≈ ζ22(∞). This can happen in two situations
(see Table IV):
1It is possible to prove using the results of Tables II and V that if two
stable adaptive filters are initialized with the same vector and adapted with
close step-sizes the following limit holds
?
In other words, close step-sizes imply close coefficient vectors in steady-state.
lim
δ→1
lim
n→∞
E{?? w1(n) − ? w2(n)?2}
E{?? w1(n)?2}
?
= 0.
i) when the step-sizes are not close to one another and
Tr(Q) = q12, where q12is the value of Tr(Q) for which
ζ11(∞) ≈ ζ22(∞);
ii) when the component filters are adapted with close step-
sizes (δ ≈ 1). However, when δ ≈ 1 and Tr(Q) ≈ q12,
the gain is small.
Replacing the expressions of Table II under the small step-
size approximation2in (22) and (26), we obtain analytical
expressions for q12and ζo(∞) shown in Table IV. From these
expressions, we can observe that the EMSE reduction in all
cases is limited by 3 dB. A reduction close to 3 dB will occur
when δ → 0 in case (i) or when the environment tends to
be stationary (Tr(Q) ≈ 0) in case (ii). It is relevant to notice
that case (i) also occurs in the convex combination of adaptive
filters since in this case 0 < ηo(∞) < 1. On the other hand,
case (ii) occurs only in the affine combination since ηo(∞)
does not lie in the interval [0, 1].
The 3 dB gain is an interesting property inherent to the
affine combination. However, we should emphasize that using
the affine combination with the filters adapted with different
step-sizes is more worthwhile than using it with close step-
sizes. In the stationary case for µ2< µ1, the closer ζ22(∞) to
ζ11(∞) the closer the EMSE gain to 3 dB. Although a gain in-
crease can be obtained with close step-sizes, the EMSE of the
combination is higher in absolute terms as ζ22(∞) becomes
closer to ζ11(∞). On the other hand, for a single adaptive filter
in nonstationary environments there is an optimal value of the
step-size for which the steady-state EMSE is minimum [16],
[17]. The EMSE of the combination achieves its smallest value
when one of its component filters is adapted with this optimum
step-size. In this case, the combined estimate is as good as
that of the optimum component, and there is no EMSE gain,
as is illustrated in a simulation of Section VI-D (see Fig. 17).
Moreover, it was shown analytically in [36] that a combination
of two filters from the same family (i.e., two LMS or two RLS
filters) cannot improve the performance over that of a single
filter of the same type with optimal selection of the step-size
(or forgetting factor).
TABLE IV
ANALYTICAL EXPRESSIONS FOR q12AND ζo(∞) FOR CASES (i) AND (ii)
IN A NONSTATIONARY ENVIRONMENT.
Combination(i) (ii)
q12
ζo(∞)ζo(∞)
µ1-LMS
µ1µ2σ2
v
ζ22(∞)/2ζ22(∞)/2
and µ2-LMS
×Tr(R)+2δζ22(∞)
(1 + δ)2
+σ2
vTr(R)Tr(Q)
2ζ22(∞)
µ1-NLMS
µ1µ2σ2
v
ζ22(∞)/2ζ22(∞)/2
and µ2-NLMS
×νu
+2δζ22(∞)
(1 + δ)2
+σ2
v[Tr(R)]2Tr(Q)νu
2ζ22(∞)
µ1-CMA
µ1µ2σ2
β
ζ22(∞)/2ζ22(∞)/2
and µ2-CMA
×Tr(R)+2δζ22(∞)
(1 + δ)2
+
σ2
βTr(R)Tr(Q)
2¯ γζ22(∞)
2The small step-size approximation was assumed in order to obtain simpler
expressions.

Page 7

CANDIDO, SILVA, AND NASCIMENTO: TRANSIENT AND STEADY-STATE ANALYSIS OF THE AFFINE COMBINATION OF TWO ADAPTIVE FILTERS7
V. TRANSIENT ANALYSIS OF REALIZABLE SCHEMES
In this section, we take into account the adaptation of η(n)
in the analysis. Our focus will be on how much a realizable
estimate for ηo(n) deviates from the optimum, and how this
affects the combination’s overall performance.
By squaring both sides of (17) and taking expectations, we
obtain
E?e2
To proceed, we assume that:
A1. The adaptation of η(n) is slow so that the correlation
between it and ea,i(n)ea,j(n),
disregarded.
This assumption follows from observations: simulations show
that η(n) converges slowly compared to variations in the input
u(n) and thus to variations on the a-priori errors.
Using A1, (19)-(21) and (23), we can rewrite (29) as
ζ(n)≈ζ22(n)+E?η2(n)?α(n)−2E{η(n)}∆ζ22(n), (30)
where we defined
a(n)?=E?e2
a,2(n)?+E?η2(n)[ea,1(n)−ea,2(n)]2?
+ 2E?η(n)?ea,2(n)ea,1(n)−e2
a,2(n)??.
(29)
i,j = 1,2 can be
α(n) ? E{[y1(n) − y2(n)]2} = ∆ζ11(n) + ∆ζ22(n).
(31)
To estimate the EMSE of the combination for all n ≥ 0 using
(30), analytical expressions for ζij(n), i = 1,2, E{η(n)}, and
E{η2(n)} should be obtained.
It is common in the literature to evaluate the EMSE as
ζij(n) ? E{ea,i(n)ea,j(n)} ≈ Tr(RSij(n − 1)),
(32)
where
Sij(n) ? E{? wi(n)? wT
j(n)}, i = 1,2
(33)
is the covariance (i = j) or the cross-variance (i ?= j) matrix
of the weight-error vector. This approach is based on the
independence assumption between the regressor vector u(n)
and weight-error vectors ? wi(n−1), i = 1,2 and is justified for
in u(n) and ? wi(n−1). This condition is a part of the widely
[19], [37]. Recursions for Sii(n), i = 1,2 are generally
obtained in the transient analysis of adaptive filters (see, e.g,
[16], [18], [26], [35] and their references). In the transient
analysis of linear combinations of two adaptive filters, an
estimate of Sij(n), i ?= j should also be obtained, which
is a straightforward extension from the case i = j [10]. In
Table V, we show the recursions for the cross-variance matrix
S12(n). Expressions for the covariance matrix Sii(n), i = 1,2
can be obtained from this table, making µ1 = µ2. Using
the expressions of Table V in conjunction with (32), ζij(n),
i,j = 1,2 can be estimated for all n ≥ 0. The expression
for S12(n) considering the combination of two NLMS was
derived using the approach from [35], under the assumptions
of Gaussian inputs and large number of coefficients. We should
notice that steady-state approximations for the EMSE and
cross-EMSE of the component filters can be obtained from
the expressions of Table V, i.e., ζij(∞) ≈ Tr(RSij(∞)),
small step-sizes due to the different time-scales for variations
used independence assumptions in adaptive filter theory [16]–
TABLE V
RECURRENT EXPRESSIONS FOR CROSS-VARIANCE MATRIX S12(n).
Combination
S12(n)
µ1-LMS
S12(n) ≈ S12(n−1)−µ1RS12(n−1)
and
−µ2S12(n−1)R+µ1µ2[2RS12(n − 1)R
µ2-LMS
+RTr(RS12(n − 1))+σ2
vR] + Q
µ1-NLMS
S12(n) ≈ S12(n−1) −
µ1
σ2
u(M−2)RS12(n−1)
µ1µ2
σ4
u(M−2)(M−4)
and
−
µ2
σ2
u(M−2)S12(n−1)R+
×?2RS12(n−1)R+RTr(RS12(n−1))+σ2
S12(n) ≈ S12(n−1)−µ1¯ γRS12(n−1)
µ2-NLMS
vR?+Q
µ1-CMA
and
−µ2¯ γS12(n−1)R+µ1µ2[2ξRS12(n−1)R
µ2-CMA
+ξRTr(RS12(n−1))+σ2
βR] + Q
i,j = 1,2. However, this procedure leads to more complex
expressions than those of Table III.
Expressions for E{η(n)} and E{η2(n)} depend on the
mixing parameter adaptation. In the next section, we assume
that η(n) is updated with the η-LMS algorithm.
A. Adaptation of the mixing parameter using η-LMS
Replacing (17) in (15), we get
eg(n) = ea,2(n) − η(n)[ea,2(n) − ea,1(n)] + b(n).
(34)
Using (34) and remarking that y1(n) − y2(n) = ea,2(n) −
ea,1(n), the update equation of η-LMS, given by (14), can be
rewritten as
η(n + 1) =
A
??
??
C
??
?
+
?
η(n)?1 − µη[ea,2(n) − ea,1(n)]2?
?
+
µηb(n)[ea,2(n) − ea,1(n)].
B
?
µη[e2
a,2(n) − ea,1(n)ea,2(n)]
?
?
(35)
Using (35), we can obtain recursions for the first and the
second moments of η(n).
1) First-order analysis: Using the same arguments of Sec-
tion III, we remark that E{C} = 0. Assuming A1 and taking
expectations in (35), we get
E{η(n+1)}=E{η(n)}[1−µηα(n)] + µη∆ζ22(n).
(36)
Since the constraint η(n) ≤ 1 is imposed in the η-LMS
algorithm, we truncate at each iteration the theoretical value
of E{η(n+1)} estimated by (36), so that E{η(n+1)} ≤ 1.
Taking the limit for n → ∞ on both sides of (36), we obtain
lim
n→∞E{η(n)} = ηo(∞).
(37)
Thus, as observed in [12], the η-LMS algorithm converges in
the average to the optimum mixing parameter at the steady-
state.

Page 8

8SUBMITTED TO THE IEEE TRANSACTIONS ON SIGNAL PROCESSING 2009
A sufficient condition for the exponential stability of (36)
is given by [38, p. 73]
|1 − µηα(n)| < 1 − ε, ∀n,
(38)
where ε is a small positive constant. In particular, for a
constant step-size, a sufficient condition is
0 < µη<
2
max{α(n)}.
(39)
2) Second-order analysis: Squaring (35) and taking expec-
tations, we obtain
E?η2(n+1)?= E?A2?+E?B2?+E?C2?+E{2AB}
+ E{2AC} + E{2BC}.
To evaluate the terms of (40), we assume that
A2. The a priori errors ea,1(n) and ea,2(n) are jointly
Gaussian with zero-mean, which implies [39]
E?e3
E?e2
Although this condition is violated in general, it is frequently
used to make the transient analysis of adaptive filters more
tractable [16]–[19]. This assumption tends to be reasonable
for small step-sizes and long filters [17].
Now, using A1 and A2 we can evaluate the terms of (40):
E{A2}: Using A1 and (42), we obtain
?
(40)
a,i(n)ea,j(n)?= 3ζii(n)ζij(n), i,j = 1,2, (41)
a,1(n)e2
E?[ea,2(n)−ea,1(n)]4?= 3α2(n),
(42)
(43)
a,2(n)?= ζ11(n)ζ22(n) + 2ζ2
12(n).
E{A2} = Eη2(n)?1−µη[ea,2(n)−ea,1(n)]2?2?
≈ E{η2(n)}?1−2µηα(n) + 3µ2
E{B2}: Using (41) and (43), we have
ηE?[e2
ηα2(n)?. (44)
E{B2} = µ2
a,2(n) − ea,1(n)ea,2(n)]2?
≈ µ2
ηζ22(n)α(n)+2µ2
η∆ζ2
22(n).
(45)
E{C2}: Since b(n) is assumed independent of ea,i(n), i =
1,2, E{b(n)} = 0, and E{b2(n)} = σ2
E?C2?= E?µ2
b, we get
ηb2(n)[ea,2(n) − ea,1(n)]2?
≈ µ2
ησ2
bα(n).
(46)
For the combination of two CMA equalizers this term is null,
since b(n) ≡ 0.
E{2AB}: Using A1, (41) and (43), we obtain
??1−µη[ea,2(n)−ea,1(n)]2?
× [e2
E{2AB} = 2µηE{η(n)}E
a,2(n)−ea,1(n)ea,2(n)]
?
≈ 2µηE{η(n)}[ζ22(n)−3µη∆ζ22(n)α(n)]. (47)
E{2AC} and E{2BC}: Since b(n) is assumed independent
of ea,i(n), i = 1,2 and E{b(n)} = 0, these terms
are null. Again, for the combination of two CMA
equalizers these terms are null by definition, since
b(n) ≡ 0.
Replacing the approximations (44)-(47) in (40), we finally
arrive at
E{η2(n + 1)} ≈ E{η2(n)}?1−2µηα(n) + 3µ2
+ 2µηE{η(n)}[ζ22(n)−3µη∆ζ22(n)α(n)]
+ µ2
η
ηα2(n)?
η∆ζ2
?ζ22(n) + σ2
b
?α(n) + 2µ2
22(n). (48)
Using (48) and (36) in conjunction with the expressions
of Table V, the EMSE of the combination for all n ≥ 0
considering the η-LMS algorithm can be estimated via (30).
From (48), the range of step-sizes to ensure the mean-square
stability of η-LMS is given by [38]
0 < µη<
2
3max{α(n)},
(49)
which is more restrictive than (39).
The stability of η-LMS depends on α(n). From Fig. 1, we
can see that α(n) = E{[y1(n)−y2(n)]2} is large at first when
the fast filter has almost converged but the slow filter is still
far from the optimum solution. At this point, µη should be
small, as required by (49). However, when the EMSE of the
slow and fast filters are similar, α(n) is small. At this point,
a large µη is required so the combination will switch to the
slow filter. This is the reason why [12] needs to constrain
η(n) ≤ 1. To guarantee that η-LMS switches quickly to the
slow filter at the proper time, µη must be chosen so large
that η-LMS will be unstable at the beginning, when α(n) is
large. Therefore, some sort of normalization is necessary for
the estimation of η. Thus, we propose in the following sections
two normalized algorithms to update the mixing parameter.
B. Adaptation of the mixing parameter using η-PN-LMS
Using an instantaneous normalization, i.e., replacing the
step-size by µη(n)=˜ µη/[y1(n)−y2(n)]2, can also lead to
divergence (see, e.g, [40]). One possible solution is to nor-
malize the algorithm using an estimate of α(n), as in [9].
The resulting normalized algorithm is called power normalized
least mean-square (η-PN-LMS) algorithm and updates the
mixing parameter via the recursion
η(n + 1) = η(n) + µη(n)eg(n)[y1(n) − y2(n)]
(50)
where
µη(n) ?
? µη
ǫ + p(n),
(51)
p(n) = λp(n − 1) + (1 − λ)[y1(n) − y2(n)]2
(52)
is a low-pass filtered estimate for the power of y1(n)−y2(n),
ǫ is a small positive constant used to avoid large step-sizes
when p(n) becomes small, and 0 ≪ λ < 1 is a forgetting
factor. The stability of (50) is ensured for 0 < ? µη< 2 [38]
In the analysis of the η-PN-LMS algorithm, we assume that
A3. The forgetting factor λ is sufficiently close to one, so
that the variance of p(n) is small and the step-size µη(n)
is weakly correlated with the a priori errors ea,i(n), i =
1,2 and the mixing parameter η(n).
and no constraint on η(n) is necessary.

Page 9

CANDIDO, SILVA, AND NASCIMENTO: TRANSIENT AND STEADY-STATE ANALYSIS OF THE AFFINE COMBINATION OF TWO ADAPTIVE FILTERS9
Using A3, the analysis of η-LMS can be directly extended
to η-PN-LMS, replacing µk
expressions of Section V-A. Hence, we only need to estimate
E{µη(n)}, as shown in the sequel.
Expanding µη(n) as a Taylor series, around the expected
value E{p(n)} ? ¯ p(n), we obtain
ηby [E{µη(n)}]k, k = 1,2 in the
µη(n)≈
? µη
ǫ+¯ p(n)−? µη[p(n)−¯ p(n)]
Taking expectations on both sides of (53), we arrive at
[ǫ+¯ p(n)]2
+? µη[p(n)−¯ p(n)]2
[ǫ+¯ p(n)]3
. (53)
E{µη(n)}≈
? µη
ǫ+¯ p(n)+
? µησ2
p(n)
[ǫ+¯ p(n)]3,
(54)
where we denoted σ2
the second term on the r.h.s. of (54) can be disregarded, which
leads to
p(n) = E{[p(n) − ¯ p(n)]2}. Assuming A3,
E{µη(n)} ≈
? µη
ǫ + ¯ p(n).
(55)
Using the same arguments, the second moment of the step-size
µη(n) can be approximated by E{µ2
Now, we obtain a recursion for ¯ p(n). Taking expectations
on both sides of (52), we get
η(n)} ≈ [E{µη(n)}]2.
¯ p(n)=λ¯ p(n−1)+(1−λ)E?[y1(n)−y2(n)]2?.
Remarking that y1(n)−y2(n)=ea,2(n)−ea,1(n), the following
recursion holds
(56)
¯ p(n)=λ¯ p(n−1)+(1−λ)α(n).
(57)
At steady-state, we have ¯ p(∞) = α(∞).
C. Adaptation of the mixing parameter using η-SR-LMS
Although η-PN-LMS circumvents the problem encountered
in the convergence of η-LMS, three parameters must be
adjusted: ? µη, λ, and ǫ. The forgetting factor λ is relatively
the step-size ? µηand of the regularization factor ǫ needs some
order to avoid these extra adjustments and since a normal-
ization is necessary, we can employ a partial instantaneous
normalization using µη(n) = µηs/|y1(n) − y2(n)| as step-
size. With this choice, the update rule (50) reduces to
easy to be adjusted (e.g., λ = 0.99). However, the choice of
care, as we show through the simulations of Section VI. In
η(n + 1) = η(n) + µηseg(n)sign[y1(n) − y2(n)],
(58)
where sign[·] is the sign function defined as
sign[x] ?



+1, x > 0
x = 0
x < 0
0,
−1,
.
(59)
We call this algorithm sign regressor least mean-square algo-
rithm (η-SR-LMS).
Using (34) and remarking that xsign[x] = |x| and that
y1(n)−y2(n)=ea,2(n)−ea,1(n), (58) can be rewritten as
η(n + 1) =
D
??
?
+
?
η(n)(1 − µηs|ea,2(n)−ea,1(n)|)
E
??
?
?
?
µηsea,2(n)sign[ea,2(n)−ea,1(n)]
+
F
???
µηsb(n)sign[ea,2(n)−ea,1(n)].
(60)
Using (60), we can obtain recursions for the first and second
moments of η(n).
1) First-order analysis: Assuming A1, taking expectations
in (60), and remarking that E{F} = 0, we obtain
E{η(n+1)}=E{η(n)}?1−µηsE{|ea,2(n)−ea,1(n)|}?
+µηsE{ea,2(n)sign[ea,2(n)−ea,1(n)]}.
(61)
Assuming A2 and using a special case of Price’s theorem (see,
e.g, [39], [17, p. 306]), the following approximations hold
?
and
E{|ea,2(n)−ea,1(n)|} ≈
2α(n)
π
,
(62)
E{ea,2(n)sign[ea,2(n)−ea,1(n)]} ≈
∆ζ22(n)
?πα(n)/2.
?
(63)
Replacing (62) and (63) in (61), we arrive at
E{η(n+1)}≈E{η(n)}
?
1−µηs
?
2α(n)
π
+µηs
∆ζ22(n)
?πα(n)/2.
(64)
Taking the limit for n → ∞ on both sides of (64), we obtain
limn→∞E{η(n)} = ηo(∞). Hence, the η-SR-LMS algorithm
also converges in the average to the optimum mixing parameter
at the steady-state.
The range of step-sizes that guarantees stability of (64) is
given by [38]
?
0 < µηs<
2π
max{α(n)}.
(65)
2) Second-order analysis: Squaring (60) and taking expec-
tations, we obtain
E?η2(n+1)?= E?D2?+E?E2?+E?F2?+E{2DE}
+ E{2DF} + E{2EF}.
(66)
Using A1 and A2, we can evaluate the terms of (66):
E{D2}: Using A1 and (62), we obtain
?
≈ E{η2(n)}1−µηs
E{D2} = Eη2(n)?1−µηs|ea,2(n)−ea,1(n)|?2?
8α(n)/π + µ2
?
?
ηsα(n)
?
.
(67)
E{E2} and E{F2}: Using the fact that sign2[x] = 1 almost
everywhere on the real line, we get

Page 10

10SUBMITTED TO THE IEEE TRANSACTIONS ON SIGNAL PROCESSING 2009
E{E2} = E?µ2
and
E?F2?= E?µ2
ηse2
a,2(n)sign2[ea,2(n) − ea,1(n)]?
≈ µ2
ηsζ22(n),
(68)
ηsb2(n)sign2[ea,2(n) − ea,1(n)]?
≈ µ2
ηsσ2
b.
(69)
E{2DE}: Using A1 and (63), we obtain
?
× ea,2(n)sign[ea,2(n)−ea,1(n)]
?
E{2DE}=2µηsEη(n)?1−µηs|ea,2(n)−ea,1(n)|?
?
. (70)
≈ 2µηsE{η(n)}
∆ζ22(n)
?πα(n)/2− µηs∆ζ22(n)
?
Since b(n) is assumed independent of ea,i(n), i = 1,2 and
E{b(n)} = 0, we have E{2DF} ≈ 0 and E{2EF} ≈ 0.
Replacing the approximations (67)-(70) in (66), we finally
arrive at
?
?
+ µ2
ηs
E{η2(n+1)}≈E{η2(n)}1−µηs
?
8α(n)
π
+µ2
ηsα(n)
?
?
(71)
+ 2µηsE{η(n)}
∆ζ22(n)
?πα(n)/2− µηs∆ζ22(n)
?σ2
b+ ζ22(n)?.
The range of step-sizes that guarantees stability of (71) is
given by [38]
?
0 < µηs<
8
π max{α(n)}.
(72)
It should be noticed that this range is more restrictive than
(65). Although the step size µηsstill depends on an estimate
of α(n), this dependence is weaker than that of the η-LMS
algorithm due to the square-root in (72). Furthermore, µηscan
be adjusted based on the analytical EMSE of the combination
(see Fig. 11). Thus, the η-SR-LMS algorithm can perform
better than η-LMS, following quickly the variations on ηo(n)
with a small EMSE and with only one free parameter to adjust,
as shown in the simulations of Section VI-A.
VI. SIMULATION RESULTS
The simulations are divided into four parts. First, we verify
the accuracy of the transient analysis for the introductory
simulations shown in Fig. 1. We also verify the behavior of the
proposed algorithms η-PN-LMS and η-SR-LMS in the same
simulation scenario. In the second part, we show some results
concerning the analysis of combinations of NLMS filters and
CMA equalizers. In the third part, we verify the validity of the
analysis of combinations of LMS filters with close step-sizes.
Finally, we focus on the tracking analysis and compare the
performances of the affine and convex combinations.
A. Recalling the introductory simulation
To verify the validity of the transient analysis in the
supervised case, we consider the identification of a time-
invariant system. The optimum solution is formed with M = 7
independent random values between -1 an 1, and is given by
wo= [+0.90 −0.54 −0.03 +0.78 +0.52 −0.09].
(73)
We assume white Gaussian input with variance 1/M so that
Tr(R) = 1, and an average of 500 runs. Moreover, i.i.d. noise
v(n) with variance σ2
signal.
Figures 5 and 6 show the results of the EMSE and the
mixing parameter for the affine combination of two LMS filters
in the same situations considered in Figures 1-(a) and (b),
in which the mixing parameter is updated with the η-LMS
algorithm. In Fig. 5, where µη= 3, the analysis can predict
that the performance of the combination is far from universal
in the initial iterations. Similarly, with µη= 0.1, the analysis
can predict that the combination is not able to switch to the
slow filter, as shown in Fig. 6. We should notice that, due to
the constraint imposed in the η-LMS algorithm (η(n) ≤ 1),
these situations become difficult to model and there is a small
gap between the experimental and theoretical EMSE during
the initial iterations. Moreover, the mixing parameter does not
achieve the optimum value obtained in the analysis, which is
higher than one in the initial iterations.
v= 0.01 is added to form the desired
EMSE (dB)
(a)
µ1-LMS
µ2-LMS
Combination
ζ(n) (Theoretical)
ζo(n) from (26)
0
1234
5
67
8
-60
-40
-20
0
iterations
×104
E{η(n)}
(b)
Experimental
Theoretical
ηo(n) from (22)
01
2
34
5
678
0
0.5
1
Fig. 5.
ensemble average of η(n) adapted with the η-LMS algorithm and theoretical
ηo(n); µ1 = 0.01, µ2 = 0.001, µη = 3, M = 7; identification of the
system given by (73), σ2
independent runs.
a) EMSE for µ1-LMS, µ2-LMS and their affine combination; b)
v= 0.01, white input with variance σ2
u= 1/7; 500
In the same scenario, the algorithms η-PN-LMS or η-SR-
LMS can circumvent the problem, as shown in Figures 7 and 8
respectively. These two algorithms have a similar performance,
which is predicted by the analysis with a good accuracy in
both cases. In addition, the experimental mixing parameter
is higher than one in the initial iterations, being far from
its theoretical optimum value during the very first iterations,
as shown in Figures 7-(c) and 8-(c). However, this does not
represent an issue since the combination presents a close to
universal performance.
To illustrate the influence of the parameters ǫ and ? µηin the
performance of the η-PN-LMS algorithm, Figures 9 and 10

Page 11

CANDIDO, SILVA, AND NASCIMENTO: TRANSIENT AND STEADY-STATE ANALYSIS OF THE AFFINE COMBINATION OF TWO ADAPTIVE FILTERS11
EMSE (dB)
(a)
µ1-LMS
µ2-LMS
Combination
ζ(n) (Theoretical)
ζo(n) from (26)
0
12
3
4567
8
-60
-50
-40
-30
-20
-10
0
iterations
×104
E{η(n)}
(b)
Experimental
Theoretical
ηo(n) from (22)
01
2
3456
7
8
0
0.5
1
Fig. 6.
ensemble average of η(n) adapted with the η-LMS algorithm and theoretical
ηo(n); µ1 = 0.01, µ2 = 0.001, µη = 0.1, M = 7; identification of the
system given by (73), σ2
independent runs.
a) EMSE for µ1-LMS, µ2-LMS and their affine combination; b)
v= 0.01, white input with variance σ2
u= 1/7; 500
EMSE (dB)
(a)
µ1-LMS
µ2-LMS
Combination
ζ(n) (Theoretical)
ζo(n) from (26)
0
12
3
4567
8
-60
-50
-40
-30
-20
-10
0
iterations
×104
E{η(n)}
(b)
Experimental
Theoretical
ηo(n) from (22)
01
2
3456
7
8
0
0.5
1
iterations
×104
E{η(n)}
(c)
Experimental
Theoretical
ηo(n) from (22)
00.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
0
5
10
Fig. 7.
b) ensemble average of η(n) adapted with the η-PN-LMS algorithm and
theoretical ηo(n); µ1= 0.01, µ2= 0.001, ? µη = 3×10−3, ǫ = 5×10−4,
from n = 0 until n = 0.5×104(note the different x-scaling).
a) EMSE for µ1-LMS, µ2-LMS and their affine combination;
λ = 0.99, M = 7; identification of the system given by (73), σ2
white input with variance σ2
v= 0.01,
u= 1/7; 500 independent runs; c) detail of b)
show the theoretical, experimental, and optimal EMSE of the
combination at three time instants as a function of ǫ (Fig. 9)
and of ? µη(Fig. 10). The time instants were chosen in order to
at n = 15×103the slower filter has not converged yet, the
time instant n = 40×103is close to the switching between the
faster to the slower filter, and at n = 65×103both filters have
converged. The same simulation setting of Fig. 7 is considered.
Similarly, Fig. 11 shows the results on the influence of µηs
for the η-SR-LMS algorithm, considering the same simulation
setting of Fig. 8. The analysis provides an accurate estimation
of the EMSE in all cases, which enables the adjustment of
the parameters through the analytical results. We can also
check the accuracy of the analysis in three different situations:
observe that the optimum value of ǫ, ? µη, or µηsis different for
possible to choose an intermediate value to obtain a tradeoff
in these three situations.
each time instant considered in the simulations. However, it is
EMSE (dB)
(a)
µ1-LMS
µ2-LMS
Combination
ζ(n) (Theoretical)
ζo(n) from (26)
0
12
3
4567
8
-60
-50
-40
-30
-20
-10
0
iterations
×104
E{η(n)}
(b)
Experimental
Theoretical
ηo(n) from (22)
01
2
3456
7
8
0
0.5
1
iterations
×104
E{η(n)}
(c)
0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
Experimental
Theoretical
ηo(n) from (22)
0
0
5
10
Fig. 8.
b) ensemble average of η(n) adapted with the η-SR-LMS algorithm and
theoretical ηo(n); µ1 = 0.01, µ2 = 0.001, µηs = 2.5 × 10−2, M = 7;
identification of the system given by (73), σ2
variance σ2
n = 0.5×104(note the different x-scaling).
a) EMSE for µ1-LMS, µ2-LMS and their affine combination;
v= 0.01, white input with
u= 1/7; 500 independent runs; c) detail of b) from n = 0 until
EMSE (dB)
n = 15000
ζ(n) (Theoretical)
ζ(n) (Experimental)
ζo(n) from (26)
0.5
11.5
2
2.533.54
4.5
55.5
-43
-42
-41
EMSE (dB)
n = 40000
0.51
1.5
22.533.5
4
4.55
5.5
-52
-50
-48
-46
-44
ǫ
×10−3
EMSE (dB)
n = 65000
0.5
11.52 2.5
3
3.54 4.55
5.5
-54
-52
-50
-48
Fig. 9.
instants for the affine combination of µ1-LMS and µ2-LMS using the η-PN-
LMS for different values of ǫ; µ1= 0.01, µ2= 0.001, ? µη = 3, λ = 0.99,
was calculated by the mean EMSE of 50 samples around the considered time
instant.
Theoretical, experimental and optimal EMSE at three different time
M = 7; identification of the system given by (73), σ2
with variance σ2
v= 0.01, white input
u= 1/7; 500 independent runs; each experimental value
B. Combinations of two NLMS filters and two CMA equalizers
To verify that the transient analysis is also accurate for
the affine combination of the other algorithms, Fig. 12 and

Page 12

12SUBMITTED TO THE IEEE TRANSACTIONS ON SIGNAL PROCESSING 2009
EMSE (dB)
n = 15000
ζ(n) (Theoretical)
ζ(n) (Experimental)
ζo(n) from (26)
12
3
4567
8
910
-44
-42
-40
EMSE (dB)
n = 40000
1
2345
6
78
9
10
-52
-50
-48
-46
? µη
×10−3
EMSE (dB)
n = 65000
1
2
34
5
6789
10
-54
-52
-50
Fig. 10. Theoretical, experimental and optimal EMSE at three different time
instants for the affine combination of µ1-LMS and µ2-LMS using the η-PN-
LMS for different values of ? µη; µ1= 0.01, µ2= 0.001, ǫ = 5×10−4, λ =
was calculated by the mean EMSE of 50 samples around the considered time
instant.
0.99, M = 7; identification of the system given by (73), σ2
input with variance σ2
v= 0.01, white
u= 1/7; 500 independent runs; each experimental value
EMSE (dB)
n = 15000
ζ(n) (Theoretical)
ζ(n) (Experimental)
ζo(n) from (26)
1
2
34
5
6789
10
-44
-42
-40
EMSE (dB)
n = 40000
12
3
4567
8
9 10
-52
-50
-48
-46
µηs
×10−2
EMSE (dB)
n = 65000
12
3
45
6
78910
-54
-52
-50
Fig. 11.
time instants for the affine combination of µ1-LMS and µ2-LMS using the
η-SR-LMS for different values of µηs; µ1 = 0.01, µ2 = 0.001, M = 7;
identification of the system given by (73), σ2
variance σ2
calculated by the mean EMSE of 50 samples around the considered time
instant.
Theoretical, experimental and optimal EMSE at three different
v= 0.01, white input with
u= 1/7; 500 independent runs; each experimental value was
Fig. 13 show the results for combination of two NLMS filters
with the η-PN-LMS algorithm and two CMA equalizers with
the η-SR-LMS algorithm, respectively. For the NLMS case,
to obtain a better estimate for the EMSE of the compo-
nent filters using the expression of Table V, we consider
M = 32 coefficients and the optimum solution (wo) from [12,
Fig. 2]. Again, we can observe a good agreement between
analysis and simulation. In the CMA case, we assume the
channels h1 = [+0.1 +0.3 +1.0 −0.1 +0.5 +0.2]
h2= [+0.25 +0.64 +0.80 −0.55]
of noise and the transmission of a 4-PAM (pulse amplitude
modulation) signal, i.e., a(n) = ±1 or a(n) = ±3, with
statistics r = 8.2, σ2
each component filter has M =4 coefficients implemented as
a T/2-fractionally spaced equalizer (FSE) and is initialized
with only one non-null and unitary element in the second
position. Fig. 13 shows the results for the EMSE and the
mixing parameter considering the channel h1until n = 4×104
and the channel h2after that. To smooth the EMSE curves,
they were filtered by a moving-average filter of 32 coefficients.
Although there is no exact agreement between analysis and
simulation, the predicted values model the overall behavior of
the combination, considering that a difference of a few dB
is common in models of blind algorithms due to the strong
assumptions necessary for the analysis.
Tand
T[27], [33] in the absence
β= 28.8, and ¯ γ = 6.8. In the combination,
EMSE (dB)
(a)
µ1-NLMS
µ2-NLMS
Combination
ζ(n) (Theoretical)
ζo(n) from (26)
00.51 1.5
2
2.53
-50
-40
-30
-20
-10
iterations
×104
E{η(n)}
(b)
Experimental
Theoretical
ηo(n) from (22)
0
0.511.52
2.5
3
0
1
2
3
Fig. 12.
b) ensemble average of η(n) adapted with the η-PN-LMS algorithm and
theoretical ηo(n); µ1= 0.1, µ2= 0.01, ? µη = 3 × 10−3, ǫ = 5 × 10−4,
a) EMSE for µ1-NLMS, µ2-NLMS and their affine combination;
λ = 0.99, M = 32; identification of the system considered in [12], σ2
0.01, white input with variance σ2
v=
u= 1/32, Q = 0; 500 independent runs.
EMSE (dB)
(a)
µ1-CMA
µ2-CMA
Combination
ζ(n) (Theoretical)
ζo(n) from (26)
01
2
3456
-30
-25
-20
-15
-10
-5
0
iterations
×104
E{η(n)}
(b)
Experimental
Theoretical
ηo(n) from (22)
0
1
2345
6
0
0.5
1
1.5
Fig. 13. a) EMSE for µ1-CMA, µ2-CMA and their affine combination; b) en-
semble average of η(n) adapted with the η-SR-LMS algorithm and theoretical
ηo(n); µ1= 1×10−3, µ2= 1×10−4, µηs= 0.5; Equalizers with M = 4
as T/2-FSE, initialized with [0 1 0 0]T; channel h1=[0.1 0.3 1.0 − 0.1 0.5 0.2]T
until n = 4×104and h2=[0.25 0.64 0.80 − 0.55]Tafter n =4×104; Q = 0;
4-PAM transmitted signal; 500 independent runs; EMSE curves filtered by a
moving-average filter of 32 coefficients.

Page 13

CANDIDO, SILVA, AND NASCIMENTO: TRANSIENT AND STEADY-STATE ANALYSIS OF THE AFFINE COMBINATION OF TWO ADAPTIVE FILTERS13
C. Affine combination of filters with close step-sizes
We now consider an affine combination of two LMS filters
with close step-sizes in a stationary environment. We assume
that the optimum solution is given by (73) and the input u(n)
is generated using a first-order autoregressive model, whose
transfer function is
model is fed with and i.i.d. Gaussian random process, whose
variance is 1/M, such that Tr(R) = 1. Again, to form the
desired signal, white noise v(n) with variance σ2
is added. Fig. 14 shows the EMSE and the mixing param-
eter along the iterations for two LMS filters with step-sizes
µ1 = 0.01 and µ2 = 0.009, using the η-LMS algorithm
with µη = 600. This high value of µη is needed in order
to ensure a high convergence rate for the combination since
[y1(n)−y2(n)] is small. In this situation, the performances
of the component filters are very close and the combination
provides a 3 dB EMSE gain in steady-state, as shown in
Fig. 14-(a) and predicted by the analysis. To smooth the EMSE
curves, they were filtered by a moving-average filter with
256 coefficients. We can observe that, due to the constraint
(η(n) ≤ 1) imposed in the η-LMS algorithm, the mixing
parameter does not achieve its optimum value, which may
be close to 25 in some time instants, as shown in Fig. 14-(b).
Consequently, the EMSE of the combination is far from the
optimum EMSE in some time instants.
?1−̺2/(1−̺z−1), with ̺ = 0.8. This
v= 0.01
EMSE (dB)
(a)
µ1-LMS
µ2-LMS
Combination
ζ(n) (Theoretical)
ζo(n) from (26)
01
2
3456
7
8
-50
-40
-30
-20
iterations
×104
E{η(n)}
(b)
Experimental
Theoretical
ηo(n) from (22)
0
1
2345
6
78
-10
0
10
20
30
Fig. 14.
ensemble average of η(n) adapted with the η-LMS algorithm and theoretical
ηo(n); µ1 = 0.01, µ2 = 0.009, µη = 600, M = 7; identification of the
system given by (73), σ2
at 0.8) with variance σ2
by a moving-average filter with 256 coefficients.
a) EMSE for µ1-LMS, µ2-LMS and their affine combination; b)
v= 0.01, colored input (AR model, 1storder, pole
u= 1/7; 500 independent runs; EMSE curves filtered
In the same scenario, the η-PN-LMS and η-SR-LMS algo-
rithms circumvent the problem since no constraint in η(n) is
necessarily used, as show respectively in Figures 15 and 16. A
3 dB EMSE gain can be observed in steady-state and there is
also an EMSE gain in the transient, being both well predicted
by the analysis.
D. Accuracy of tracking analysis
To verify the validity of the tracking analysis, the affine
combination is compared to the convex combination assuming
two LMS filters with different step-sizes. The same simulation
setting of Fig. 14 is considered, but with fixed µ1= 0.1 and
EMSE (dB)
(a)
µ1-LMS
µ2-LMS
Combination
ζ(n) (Theoretical)
ζo(n) from (26)
01
2
3456
7
8
-50
-40
-30
-20
iterations
×104
E{η(n)}
(b)
Experimental
Theoretical
ηo(n) from (22)
0
1234
5
678
-10
0
10
20
30
Fig. 15.
b) ensemble average of η(n) adapted with the η-PN-LMS algorithm and
theoretical ηo(n); µ1 = 0.01, µ2 = 0.009, ? µη = 0.4, ǫ = 9 × 10−4,
500 independent runs; EMSE curves filtered by a moving-average filter with
256 coefficients.
a) EMSE for µ1-LMS, µ2-LMS and their affine combination;
λ = 0.99, M = 7; identification of the system given by (73), σ2
colored input (AR model, 1storder, pole at 0.8) with variance σ2
v= 0.01,
u= 1/7;
EMSE (dB)
(a)
µ1-LMS
µ2-LMS
Combination
ζ(n) (Theoretical)
ζo(n) from (26)
01
2
3456
7
8
-50
-40
-30
-20
iterations
×104
E{η(n)}
(b)
Experimental
Theoretical
ηo(n) from (22)
0
1234
5
678
-10
0
10
20
30
Fig. 16.
b) ensemble average of η(n) adapted with the η-SR-LMS algorithm and
theoretical ηo(n); µ1 = 0.01, µ2 = 0.009, µηs = 0.5, M = 7;
identification of the system given by (73), σ2
model, 1storder, pole at 0.8) with variance σ2
runs; EMSE curves filtered by a moving-average filter with 256 coefficients.
a) EMSE for µ1-LMS, µ2-LMS and their affine combination;
v= 0.01, colored input (AR
u= 1/7; 500 independent
µ2= δµ1. Fig. 17-(a) shows the theoretical and experimental
values of ζii(∞), i = 1,2 for the component filters and
the values of ζ(∞) for the affine and convex combinations
as functions of δ, considering a nonstationary environment
with Q = 4×10−7I. The ratio ζ(∞)/min{ζii(∞)} is also
shown in Fig. 17-(b). It can be noticed that there is an
EMSE reduction for both the affine and convex combinations
when Tr(Q) = q12 = µ1µ2σ2
δ = 0.025 in this case. An EMSE reduction for the affine
combination also occurs when µ1 ≈ µ2, i.e., δ ≈ 1. In
this case, the convex combination can only perform as its
best component filter, since the mixing parameter needs to be
negative to cause the EMSE reduction, as shown in Fig. 17-
(c). In both cases, the reduction is limited to 3 dB, which
agrees with the results of Table IV. The theoretical results for
the convex combination were obtained truncating the value
of the optimal mixing parameter to the interval [0,1]. It is
important to remark, though, that both points at which the
largest EMSE reduction happens do not represent optimal
vTr(R), which corresponds to

Page 14

14SUBMITTED TO THE IEEE TRANSACTIONS ON SIGNAL PROCESSING 2009
situations, as can be seen in Fig. 17-(a). For a single LMS filter
in a nonstationary environment, there is an optimum value of
the step-size that minimizes the EMSE. The minimum EMSE
value for the affine and convex combinations (≈ −38 dB)
occurs exactly when µ2 assumes this optimum value, which
happens for δ = 0.17 in this example. In this case, both
combinations perform as their best component filter µ2-LMS.
Therefore, using the affine combination of filters of the same
family updated with different step-sizes is more worthwhile
than using it with close step-sizes.
ζii(∞) and ζ(∞) (dB)
(a)
µ1-LMS
µ2-LMS
Affine comb.
Convex comb.
0
0.1
0.20.30.40.5
0.6
0.70.80.91
-38
-36
-34
-32
-30
-28
ζ(∞)
min{ζii(∞)}
(b)
00.1 0.2
0.3
0.4 0.5 0.60.7
0.8
0.91
0.5
0.6
0.7
0.8
0.9
1
1.1
δ
E{η(∞)}
(c)
0
0.10.20.30.4
0.5
0.60.7 0.80.9
1
-8
-6
-4
-2
0
Fig. 17.
ζ(∞); b) ζ(∞)/min{ζii(∞)}, i = 1,2; and c) E{η(∞)} for the affine
(µη = 1) and convex (µa = 100, a+= 4 and ǫ = 0.1 [8]) combinations
of two LMS filters with µ1= 0.1, µ2= δµ1 and M = 7; identification of
the system given by (73), σ2
pole at 0.8) with variance σ2
The theoretical values are indicated by lines and the experimental values by
△, ?, ?, and ∗.
Theoretical and experimental values of a) ζii(∞), i = 1,2 and
v= 0.01, colored input (AR model, 1storder,
u= 1/7; Q = 4×10−7I; 50 independent runs.
VII. CONCLUSION
As an extension of [12] and [13], we proposed transient
and steady-state analyses for the EMSE and the mixing
parameter of the affine combination, based on the theoretical
EMSE and cross-EMSE of the component filters and on the
adaptation of the mixing parameter. This allows the application
to different combinations of algorithms, such as LMS, NLMS
and CMA, considering white or colored inputs and stationary
or nonstationary environments. Good agreement between the
analysis and the simulations was always observed. Moreover,
we proposed and analyzed two normalized algorithms for
updating the mixing parameter. The theoretical models can
predict situations in which these algorithms can achieve a
better performance, being useful for the designer.
APPENDIX A
ASSUMPTIONS FOR THE CMA ANALYSIS
Model (8) is based on the following assumption:
B1. The channel noise power is small enough for the zero-
forcing solution woto be one of the global minimizers
of the constant-modulus cost function. In other words,
the optimal solution achieves perfect equalization, i.e.,
a(n − τd) ≈ uT(n)wo(n − 1) [10], [23], [26], [33].
Using B1, the filter output can be approximated by
yi(n) ≈ a(n − τd) − ea,i(n), i = 1,2.
(74)
Equation (8) is obtained replacing (74) in (7) and assuming
that terms depending on ek
to be disregarded for all n ≥ 0. In other words, we assume
that the deviation between the component equalizers and the
zero-forcing solution is always small.
To calculate the first and second moments of the random
i.i.d. variables γ(n) and β(n), we assume that
B2. The constellation used to generate the a(n) has circular
symmetry, so that E{ak(n)} = 0 for all odd integers
k > 0. This assumption is not restrictive, since this
condition is true for practical constellations.
Using B2, we find that E{β(n)} = 0,
a,i(n), k ≥ 2 are sufficiently small
σ2
¯ γ ? E{γ(n)} = 3E{a2(n)} − r,
β? E{β2(n)} = E{a6(n) − r2a2(n)},
(75)
(76)
and
ξ ? E{γ2(n)} = 3rE{a2(n)} + r2.
(77)
REFERENCES
[1] R. Candido, M. T. M. Silva, and V. H. Nascimento, “Affine combinations
of adaptive filters,” in Proc. 42th Asilomar Conf. Signals, Systems, and
Computers, Pacific Grove, CA, 2008, pp. 236–240.
[2] ——, “On combinations of CMA equalizers,” in Proc. IEEE
Int. Conf. Acoustics, Speech, and Signal Process., Taipei, Taiwan, 2009,
pp. 2865–2868.
[3] P. Andersson, “Adaptive forgetting in recursive identification through
multiple models,” International Journal of Control, vol. 42, pp. 1175–
1193, Nov. 1985.
[4] M. Nied´ zwiecki, “Identification of nonstationary stochastic systems
using parallel estimation schemes,” IEEE Trans. Autom. Control, vol. 35,
pp. 329–334, Mar. 1990.
[5] ——, “Multiple-model approach to finite memory adaptive filtering,”
IEEE Trans. Signal Process., vol. 40, pp. 470–473, Feb. 1992.
[6] A. C. Singer and M. Feder, “Universal linear prediction by model order
weighting,” IEEE Trans. Signal Process., vol. 47, pp. 2685–2700, Oct.
1999.
[7] M. Mart´ ınez-Ram´ on et al., “An adaptive combination of adaptive filters
for plant identification,” in Proc. 14th Int. Conf. Digital Signal Process.,
vol. 2, Santorini, Greece, 2002, pp. 1195–1198.
[8] J. Arenas-Garc´ ıa, A. R. Figueiras-Vidal, and A. H. Sayed, “Mean-square
performance of a convex combination of two adaptive filters,” IEEE
Trans. Signal Process., vol. 54, pp. 1078–1090, Mar. 2006.
[9] L. A. Azpicueta-Ruiz, A. R. Figueiras-Vidal, and J. Arenas-Garc´ ıa,
“A normalized adaptation scheme for the convex combination of two
adaptive filters,” in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal
Process., Las Vegas, NV, 2008, pp. 3301–3304.
[10] M. T. M. Silva and V. H. Nascimento, “Improving the tracking capability
of adaptive filters via convex combination,” IEEE Trans. Signal Process.,
vol. 56, pp. 3137–3149, Jul. 2008.
[11] J. Arenas-Garcia and A. R. Figueiras-Vidal, “Adaptive combination of
proportionate filters for sparse echo cancellation,” IEEE Trans. Au-
dio, Speech, and Language Process., vol. 17, pp. 1087–1098, Aug. 2009.

Page 15

CANDIDO, SILVA, AND NASCIMENTO: TRANSIENT AND STEADY-STATE ANALYSIS OF THE AFFINE COMBINATION OF TWO ADAPTIVE FILTERS15
[12] N. J. Bershad, J. C. M. Bermudez, and J.-Y. Tourneret, “An affine com-
bination of two LMS adaptive filters - transient mean-square analysis,”
IEEE Trans. Signal Process., vol. 56, pp. 1853–1864, May 2008.
[13] J. C. M. Bermudez, N. J. Bershad, and J.-Y. Tourneret, “An affine
combination of two NLMS adaptive filters - transient mean-square
analysis,” in Proc. 42th Asilomar Conf. Signals, Systems, and Computers,
Pacific Grove, CA, 2008, pp. 230–236.
[14] A. T. Erdogan, S. S. Kozat, and A. C. Singer, “Comparison of convex
combination and affine combination of adaptive filters,” in Proc. IEEE
Int. Conf. Acoustics, Speech, and Signal Process., Taipei, Taiwan, 2009,
pp. 3089–3092.
[15] S. S. Kozat and A. C. Singer, “A performance-weighted mixture of LMS
filters,” in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Process.,
Taipei, Taiwan, 2009, pp. 3101–3104.
[16] S. Haykin, Adaptive Filter Theory, 4th ed. Prentice Hall, Upper Saddle
River, 2002.
[17] A. H. Sayed, Fundamentals of Adaptive Filtering. John Wiley & Sons,
NJ, 2003.
[18] B. Farhang-Boroujeny, Adaptive Filters - Theory and Applications. John
Wiley & Sons, West Sussex, 1998.
[19] P. S. R. Diniz, Adaptive Filtering: Algorithms and Practical Implemen-
tation, 3rd ed.Springer, 2008.
[20] J. R. Treichler, I. Fijalkow, and C. R. Johnson Jr., “Fractionally spaced
equalizers,” IEEE Signal Process. Mag., vol. 13, pp. 65–81, May 1996.
[21] Y. Li and Z. Ding, “Global convergence of fractionally spaced Godard
(CMA) adaptive equalizers,” IEEE Trans. Signal Process., vol. 44, pp.
818–826, Apr. 1996.
[22] M. Mboup and P. A. Regalia, “A gradient search interpretation of the
super-exponential algorithm,” IEEE Transactions on Information Theory,
vol. 46, pp. 2731–2734, Nov. 2000.
[23] J. Mai and A. H. Sayed, “A feedback approach to the steady-state
performance of fractionally spaced blind adaptive equalizers,” IEEE
Trans. Signal Process., vol. 48, pp. 80–91, Jan. 2000.
[24] D. N. Godard, “Self-recovering equalization and carrier tracking in
two dimensional data communication system,” IEEE Trans. Commun.,
vol. 28, pp. 1867–1875, Nov. 1980.
[25] J. R. Treichler and B. Agee, “A new approach to multipath correction of
constant modulus signals,” IEEE Trans. Acoust., Speech, Signal Process.,
vol. ASSP-28, pp. 334–358, Apr. 1983.
[26] V. H. Nascimento and M. T. M. Silva, “Stochastic stability for the
Constant Modulus Algorithm,” IEEE Trans. Signal Process., vol. 56,
pp. 4984–4989, Oct. 2008.
[27] J. Arenas-Garc´ ıa and A. R. Figueiras-Vidal, “Improved blind equal-
ization via adaptive combination of constant modulus algorithms,” in
Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Process., vol. III,
Toulouse, France, 2006, pp. 756–759.
[28] M. T. M. Silva and V. H. Nascimento, “Tracking analysis of the Constant
Modulus Algorithmn,” in Proc. IEEE Int. Conf. Acoustics, Speech, and
Signal Process., Las Vegas, NV, 2008, pp. 3561–3564.
[29] O. Shalvi and E. Weinstein, “Super-exponential methods for blind
deconvolution,” IEEE Transactions on Information Theory, vol. 39, pp.
504–519, Mar. 1993.
[30] D. T. M. Slock, “On the convergence behavior of the LMS and the
normalized LMS algorithms,” IEEE Trans. Signal Process., vol. 41, pp.
2811–2825, Sep. 1993.
[31] E. Eweda, “Comparison of RLS, LMS and sign algorithms for tracking
randomly time-varying channels,” IEEE Trans. Signal Process., vol. 42,
pp. 2937–2944, Nov. 1994.
[32] N. R. Yousef and A. H. Sayed, “A unified approach to the steady-state
and tracking analyses of adaptive filters,” IEEE Trans. Signal Process.,
vol. 49, pp. 314–324, Feb. 2001.
[33] ——, “A feedback analysis of the tracking performance of blind adaptive
equalization algorithms,” in Proc. IEEE Conf. on Dec. and Control,
vol. 1, Dec. 1999, pp. 174–179.
[34] M. T. M. Silva and M. D. Miranda, “Tracking issues of some blind
equalization algorithms,” IEEE Signal Process. Lett., vol. 11, pp. 760–
763, Sept. 2004.
[35] M. C. Costa and J. C. M. Bermudez, “An improved model for the
normalized LMS algorithm with Gaussian inputs and large number of
coefficients,” in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal
Process., vol. II, Orlando, FL, 2002, pp. 1385–1388.
[36] V. H. Nascimento, M. T. M. Silva, L. A. Azpicueta-Ruiz, and J. Arenas-
Garc´ ıa, “On the tracking performance of combinations of least mean
squares and recursive least squares adaptive filters,” in Proc. IEEE
Int. Conf. Acoustics, Speech, and Signal Process., Dallas, TX, 2010,
pp. 3710–3713.
[37] J. E. Mazo, “On the independent theory of equalizer convergence,” Bell
Syst. Tech. J., vol. 58, pp. 963–993, May/Jun. 1979.
[38] B. D. O. Anderson, R. R. Bitmead, C. R. Johnson Jr., P. V. Kokotovic,
R. L. Kosut, I. M. Y. Mareels, L. Praly, and B. D. Riedle, Stability
of adaptive systems: passivity and averaging analysis.
Cambridge, MA, 1986.
[39] A. Papoulis, Probability, Random Variables and Stochastic Processes.
McGraw-Hill Companies, Inc., 1991.
[40] N. J. Bershad, “Analysis of the normalized LMS algorithm with Gaus-
sian inputs,” IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-
34, pp. 793–806, Aug 1986.
MIT Press,