Adaptive Volterra Filters With Evolutionary Quadratic Kernels Using a Combination Scheme for Memory Control

Abstract

This paper proposes a new paradigm for adaptive Volterra filtering using a time-variant size of the quadratic kernel memory in order to optimally identify any unknown transversal second-order nonlinear system. To this end, competing Volterra structures of different sizes are employed in a hierarchical combination scheme so as to find the best configuration of the second-order kernel memory, using the already known diagonal-coordinate representation. The length and number of required quadratic kernel diagonals can be concurrently estimated by monitoring the combination performance. Subsequently, the memory size of the involved models is dynamically increased or decreased, following a set of intuitive rules. Since this automatic memory adaptation is performed along with the coefficient updates, an efficient Volterra filter is realized, offering great flexibility and minimizing the risk of under- or overmodeling any given quadratic nonlinearity. Besides the straightforward scheme, a simplified version is presented, greatly reducing the algorithmic demands. This efficient version is based on a virtualization of the competing Volterra filters by jointly using common coefficients and hence exhibits a computational complexity suitable for practical implementations. The robust estimation performance of the approach is demonstrated by various examples for a nonlinear acoustic echo cancellation scenario, involving stationary noise, real speech signals and realistic Volterra kernels.

Full text

Page 1

1
Adaptive Volterra Filters with Evolutionary
Quadratic Kernels using a Combination
Scheme for Memory Control
Marcus Zeller∗, Luis A. Azpicueta-Ruiz, Jer´ onimo Arenas-Garc´ ıa, Member, IEEE,
and Walter Kellermann, Fellow, IEEE
Abstract
This paper proposes a new paradigm for adaptive Volterra filtering using a time-variant size of the
quadratic kernel memory in order to optimally identify any unknown transversal second-order nonlinear
system. To this end, competing Volterra structures of different sizes are employed in a hierarchical
combination scheme so as to find the best configuration of the second-order kernel memory, using the
already known diagonal-coordinate representation. The length and number of required quadratic kernel
diagonals can be concurrently estimated by monitoring the combination performance. Subsequently, the
memory size of the involved models is dynamically increased or decreased, following a set of intuitive
rules. Since this automatic memory adaptation is performed along with the coefficient updates, an efficient
Volterra filter is realized, offering great flexibility and minimizing the risk of under- or overmodeling any
given quadratic nonlinearity. Besides the straightforward scheme, a simplified version is presented, greatly
reducing the algorithmic demands. This efficient version is based on a virtualization of the competing
Volterra filters by jointly using common coefficients and hence exhibits a computational complexity
suitable for practical implementations. The robust estimation performanceof the approach is demonstrated
Manuscript received May 25, 2010; accepted December 02, 2010.
M. Zeller and W. Kellermann are with the Chair of Multimedia Communications and Signal Processing at the University of
Erlangen-Nuremberg, Cauerstr. 7, 91058 Erlangen, Germany (e-mail: {zeller,wk}@LNT.de). L. Azpicueta-Ruiz and J. Arenas-
Garc´ ıa are with the Department of Signal Theory and Communications at the Universidad Carlos III de Madrid, Avda. de la
Universidad 30, 28911 Legan´ es-Madrid, Spain (e-mail: {lazpicueta,jarenas}@tsc.uc3m.es). This work has partly been performed
while M. Zeller was a visiting researcher at the Dept. of Signal Theory and Communications in Madrid and has been supported by
the Deutsche Forschungsgemeinschaft (DFG) under contract numbers KE 890/5-1 and KE 890/6-1. The work of L. Azpicueta-
Ruiz and J. Arenas-Garc´ ıa has partly been funded by MEC Project TEC2008-02473.
Copyright (c) 2010 IEEE. Personal use of this material is permitted. However, permission to use this material for any other
purposes must be obtained from the IEEE by sending a request to pubs-permissions@ieee.org.
December 14, 2010DRAFT

Page 2

2
by various examples for a nonlinear acoustic echo cancellation scenario, involving stationary noise, real
speech signals and realistic Volterra kernels.
Index Terms
Nonlinear system identification, Volterra filters, structure selection, combination of filters, echo
cancellation
I. INTRODUCTION
Adaptive Volterra filters (VFs) are a popular model for the identification and compensation of nonlin-
earities with memory [1], [2]. The main reason is that these nonlinear filter structures can intuitively be
interpreted as either extensions of the linear convolution towards higher-order transversal filtering or as
Taylor series with memory [1]. As such, they exhibit a great modeling power for a broad class of nonlinear
distortions and are used in a variety of applications where the system’s output can be sufficiently well
described by first- and higher-order input regression terms. Among these, the probably most prominent
ones are given by loudspeaker modeling and inversion [3], [4], nonlinear acoustic echo and noise control
[5]–[8] and equalization in communications [9]–[12].
However, a major drawback of these filters is the potentially large number of model coefficients that
are required to fully characterize the nonlinear system. Since this number grows exponentially with the
order of nonlinearity, this situation is especially challenging in the acoustic context, where relatively long
transversal filters are typically employed. In order to alleviate this problem, several authors have proposed
simplified structures, exploiting knowledge that can be inferred from certain model assumptions, as e.g.
in [5], [13]–[15]. A more appropriate addressing of the Volterra kernel coefficients is given by a multi-
channel representation along higher-order diagonals [2], [10]. Nevertheless, as nearly all simplifications
are based on rigid assumptions about the memory size of the involved blocks, they still imply the risk of
over- or undermodeling the unknown nonlinear system. Thus, it is highly desirable to develop adaptive
filtering techniques with an evolutionary learning of the underlying structure of the system to be identified.
Regarding linear models, various variable tap-length strategies have already been presented in [16]–[18],
seeking to find the optimum length of adaptive FIR filters. This is often done by relying on the assumption
of exponentially decaying impulse responses [19], [20]. Since well-matched adaptive models are even
more desirable in case of nonlinear system identification, Billings et al. [21] have proposed methods for
determining the optimum NARMAX (nonlinear auto-regressive moving average with exogeneous input)
model structure online, where the error reduction ratios of different nonlinear terms are compared with a
December 14, 2010DRAFT

Page 3

3
defined tolerance using sliding-data windows. A similar approach has been investigated for radial basis
function neural networks in [22], whereas other algorithms perform hypothesis testing with generated
bootstrap data for structure detection [23]. Although the NARMAX model can be understood as the most
general class of nonlinear systems, it is also well-known that adaptive VFs exhibit sufficient modeling
power for many applications. This is especially true in the field of acoustics [3]–[5] or communications
[10], [11] where cascaded linear and nonlinear physical transmission systems have to be modeled. On
the other hand, these scenarios generally imply relatively large time lags of the regression variables (and
hence large Volterra kernel sizes) as well as nonstationary input data for which the methods in [2], [21]–
[23] have not been tested. This contribution therefore concentrates on the class of nonlinear transversal
filters with nonlinear distortions occuring only on the input data.
Several authors have applied genetic algorithms for identifying the most appropriate structure for these
nonlinear models, based on neural networks [24], Hammerstein systems [25] and also VFs [26], [27].
In [2] Glentis et al. have proposed various search algorithms based on the Akaike or the Bayesian
information criteria and/or user specifications, performing either a global exhaustive search up to a given
maximum size of the nonlinear system or operating locally, i.e., only on a desired subset of possible
nonlinear terms. Besides, Schoukens et al. have addressed the problem of finding the best (inverse)
model for approximate linearization in [28]. These approaches effectively realize nonlinear models with
adaptive structures, however, most algorithms are performed in an “offline” manner or may involve a
high computational complexity due to the effort of testing many inefficient configurations. For example,
the genetic algorithm [26] implies random mutations and some “survival of the fittest”-rule which do not
necessarily ascertain a well-defined and timely convergence when applied to potentially large kernels.
Extending the work presented in [29], [30], the algorithm proposed in this paper represents a novel
approach to Volterra filtering and nonlinear model adaptation, resulting in a more target-oriented evolution
towards the optimum kernel memory than realized by genetic algorithms. This is accomplished by
monitoring the mixing parameters from combinations of differently-sized VFs in a two-stage hierarchical
scheme, thus estimating both the optimum length and the number of necessary diagonals of a quadratic
Volterra kernel at the same time. Following some intuitive rules, the required kernel memory is then
adjusted over time so as to realize a completely adaptive model that can track the dynamic changes
of nonlinear systems with time-variant quadratic kernel memory “online”. Furthermore, a simplified
and much more efficient version based on the ideas in [30], [31] is outlined, allowing for practical
implementations due to considerable savings in computations. Although this paper assumes only a second-
order nonlinear system with fixed linear kernel memory for notational simplicity, it should be mentioned
December 14, 2010DRAFT

Page 4

4
that this principle can be extended to first- and/or higher-order kernels as well. However, such an extension
would also call for estimating the order of nonlinearity (i.e. quadratic, cubic, etc.), which is beyond the
scope of this work and will be investigated in future work.
The rest of this article is organized as follows: First, Section II briefly reviews the second-order Volterra
filtering and its notation. Section III then presents the underlying combination scheme whereas the actual
control and estimation algorithm is described in detail in Section IV. The simplification by means of
virtual VF components is outlined in Section V before the performance of both variants is evaluated for
various experiments considering a nonlinear acoustic echo cancellation (NLAEC) scenario as an example
for challenging large Volterra kernels. Finally, Section VII addresses the expected algorithmic complexity
for realistic kernel sizes and concluding remarks are given in Section VIII.
II. DIAGONAL-COORDINATE VOLTERRA FILTERS AND PROBLEM FORMULATION
The output of a second-order VF [1] is defined by the superposition
y(k) = y1(k) + y2(k)
(1)
for each discrete-time instant k. In (1), the output y1(k) is computed by the convolution
y1(k) =
N1−1
?
n=0
h1,nx(k − n),
(2)
relating the linear Volterra kernel h1,n of size N1 and the input signal x(k). Employing a diagonal-
coordinate representation [5], [10], the superimposed output y2(k) of the quadratic kernel reads
y2(k) =
W−1
?
w=0
N2,w−1
?
n=0
h2,n,w+nx2,w(k − n).
(3)
Here, the regressors x2,w(k) := x(k)x(k − w) can be interpreted as input signals to different diagonals
0 ≤ w ≤ W − 1 with decreasing lengths
N2,w:= N2− w
(4)
in a two-dimensional coefficient plane. The associated coordinate interpretation n1:= n,n2:= w + n
of the second-order kernel h2,n1,n2is illustrated in Fig. 1. Note that the computational restriction to a
certain width W of contributing diagonals represents a reasonable modification for cascaded systems
[13], [14], since the VF can easily be scaled. For instance, W = 0 implies that y(k) is given by a
purely linear system h1due to y2(k) := 0, whereas selecting W = N2can still account for all unique
coefficients in h2,n1,n2. Hence, the overall filtering is mainly governed by the computation of (3) as
obviously O(N1) < O(WN2) ≈ O(N2
2) even for moderate sizes N1,N2,W. It is therefore highly
December 14, 2010DRAFT

Page 5

5
n1
n2
n
w
0
size N2
width W
Fig. 1.Illustration of Cartesian and diagonal coordinates in
the second-order kernel h2,n1,n2of full size N2× N2.
x(k)
y(k)
? y(k)
EVOLVE
n(k)
d(k)
e(k)
unknown
adaptive VF
Fig. 2. Nonlinear system identification with unknown second-
order nonlinearity and adaptive Volterra filter (VF) based on the
evolutionary Volterra estimation (EVOLVE) technique.
desirable to suitably choose the memory of the quadratic kernel, so as to optimally model a given
unknown second-order nonlinear system with the least number of coefficients.
Seeking a concise vector notation for the rest of this paper, we define
h1:=?h1,0,...,h1,N1−1
?T,
x1(k) :=?x(k),...,x(k − N1+ 1)?T,
(5)
as the linear kernel coefficient and regression vectors and
h2,w:=?h2,0,w+0,...,h2,N2,w−1,w+N2,w−1
as the corresponding quantities of the quadratic kernel. Complete stacking yields (1) in the form of
?T,
x2,w(k) :=?x2,w(k),...,x2,w(k − N2,w+ 1)?T,
(6)
y(k) = hT
1x1(k) +
W−1
?
2,0,...,hT
w=0
hT
2,wx2,w(k) := hTx(k),
(7)
where
h =
?hT
1,hT
2,W−1
?T,
x(k) =?xT
1(k),xT
2,0(k),...,xT
2,W−1(k)?T.
(8)
The signal model for the fundamental nonlinear system identification task is depicted in Fig. 2. As
can be seen, the reference signal d(k) = y(k) + n(k) contains the unknown system’s output y(k)
and may be corrupted by local interference or additive noise n(k). Despite the assumption that the
nonlinearity is of second order, there is generally no a priori knowledge about the optimum kernel
memory sizes N1,opt,N2,opt, Woptthat should ideally be used. It is worthwhile mentioning that these
parameters will also depend on the specific scenario as determined by, e.g., the signal-to-noise ratio (SNR)
or the adaptation step size. Nevertheless, assuming a known linear kernel size N1,optfor simplicity, the
following different situations can occur for the second-order kernel memory:
• N2< N2,opt and/or W < Wopt: length and/or width undermodeling, yielding reduced adaptation
performance since the true size of the unknown quadratic kernel exceeds the memory of the VF.
December 14, 2010 DRAFT