Page 1

IAA SPECTRAL ESTIMATION: FAST IMPLEMENTATION USING THE

GOHBERG-SEMENCUL FACTORIZATION

Ming Xue, Luzhou Xu, and Jian Li∗

University of Florida

Dept. of Electrical and Computer Engineering

Gainesville, FL 32611-6130, USA

Petre Stoica

Uppsala University

Dept. of Information Technology

Uppsala, Sweden

ABSTRACT

We consider a fast implementation of the weighted least-squares

based iterative adaptive approach (IAA) for spectral estimation of

uniformly sampled sequences. IAA is a robust, user parameter-free

and nonparametric adaptive algorithm that can work with a single

data sequence or snapshot. Compared with the conventional peri-

odogram, IAA can be used to significantly increase the resolution

and suppress the sidelobe levels. However, due to its high computa-

tional complexity, IAA can only be used in applications with short

data sequences. We present herein a novel fast implementation of

IAA using a Gohberg-Semencul (G-S)-type factorization of the IAA

covariance matrix. By exploiting the Toeplitz structure of the said

matrix, we are able to reduce the computational cost by two orders

of magnitudes even for sequences with moderate lengths.

Index Terms— Toeplitz Matrices, Gohberg-Semencul Factor-

ization, Iterative Adaptive Approach (IAA), Spectral Estimation

1. INTRODUCTION

Spectral estimation has a variety of applications ranging from

direction-of-arrival (DOA) estimation using an array of sensors, to

synthetic aperture radar (SAR) and synthetic aperture sonar (SAS)

imaging. The conventional data-independent periodogram, though

simple and efficient to implement, suffers from poor resolution and

high sidelobe level problems. Several data-adaptive algorithms,

such as the Capon [1] and APES [2] methods, were proposed for

enhanced resolution and low sidelobe levels. However, both Capon

and APES require multiple realizations of a random process to esti-

mate the sequence covariance matrices. To use Capon or APES with

a single sequence for spectral estimation, spatial smoothing type of

approaches are needed to “create” multiple snapshots, resulting in

reduced data lengths [2]. Recently, a weighted least squares (WLS)

∗Please address all correspondence to: Dr. Jian Li, Department of Elec-

trical and Computer Engineering, P. O. Box 116130, University of Florida,

Gainesville, FL 32611, USA. Phone: (352) 392-2642. Fax: (352) 392-0044.

E-mail: li@dsp.ufl.edu.

This work was supported in part by the U.S. Army Research Laboratory

and the U.S. Army Research Office under Grant No. W911NF-07-1-0450,

the National Science Foundation (NSF) under Grant No. ECCS-0729727,

the Office of Naval Research (ONR) under Grant No. N00014-09-1-0211,

the Swedish Research Council (VR), and the European Research Council

(ERC). The views and conclusions contained herein are those of the authors

and should not be interpreted as necessarily representing the official policies

or endorsements, either expressed or implied, of the U.S. Government. The

U.S. Government is authorized to reproduce and distribute reprints for Gov-

ernmental purposes notwithstanding any copyright notation thereon.

based iterative adaptive approach (IAA) [3] has been proposed for

spectral estimation. IAA obviates the need for spatial smoothing, re-

sulting in improved resolution and performance compared to Capon

and APES. IAA is a robust, user parameter-free, and nonparametric

adaptive spectral analysis algorithm that can work with a single data

sequence. It has been shown that IAA can provide higher resolution

and lower sidelobe levels than periodogram. Nonetheless, the high

computational complexity of the existing IAA implementations [3]

prohibits its application to long sequences.

The fast implementation of IAA we present herein is inspired by

those of Capon and APES [4–9], which first compute the Cholesky

factor or the Gohberg-Semencul (G-S)-type factor of the inverse of

the sample covariance matrixˆR

compute the final spectral estimate. In [4–6,8,9], efficient decom-

positions ofˆR−1are explored by using the Toeplitz structure of the

approximatedˆR or the data matrices. Efficient Cholesky factoriza-

tion ofˆR−1is used in [5,6], while [7] only uses a direct Cholesky

factorization ofˆR−1. A G-S-type factorization ofˆR−1is exploited

in [4,8,9]. By utilizing the trigonometric structure of the pertinent

steering vectors, [6] and [7] take advantage of the fast Fourier trans-

form (FFT) to speed up the spectral estimation after the Cholesky

factorization ofˆR−1. Given the G-S decomposition ofˆR−1, and in

light of the fact that the data sampling grid and the frequency scan-

ning grid are both uniformly distributed, [8] and [9] first compute the

coefficients of a polynomial and then perform FFT of the coefficients

to determine the spectral estimate. To compute the polynomial coef-

ficientsmoreefficientlythan[8][9], both[4]and[5]leveragethefact

that the coefficients of the product of two polynomials can be com-

puted though convolution and thus FFT. Nevertheless, the scheme

in [4] is much more efficient than that in [5] due to the former ex-

ploiting the Toeplitz structure produced by the G-S factorization of

ˆR−1, whichisabsentintheCholeskyfactorsin[5]. Byandlarge,[4]

provides the most efficient implementations of Capon and APES for

spectral estimation. Although IAA differs from Capon and APES

in many details, the efficient implementation ideas in the aforemen-

tioned works form the foundation of our fast IAA implementation

approach.

In this paper, we present an efficient implementation of IAA for

spectral estimation of uniformly sampled data sequences. Similarly

to [4], we also exploit the G-S-type factorization of R−1, where R

denotes the IAA covariance matrix. However, there is more matrix

structure in IAA for us to exploit, which enables us to perform al-

most all operations in IAA by FFT or inverse FFT (IFFT), except

for the G-S factorization step. As a result, we can achieve a dramatic

reduction of the computational cost by two orders of magnitude even

−1, and then exploit fast schemes to

4304 978-1-4577-0539-7/11/$26.00 ©2011 IEEEICASSP 2011

Page 2

for sequences with moderate lengths.

Notation: We denote vectors and matrices by boldface lower-

case and boldface uppercase letters, respectively. We use ? for a

definition, (·)T, (·)∗, and (·)Hfor the transpose, complex conjugate,

and conjugate transpose, respectively. ˇ x denotes the vector x flipped

up-side-down.

The rest of this paper is organized as follows. In Section 2, we

briefly introduce the IAA algorithm. In Section 3, we introduce the

G-S type factorization of a general matrix, and present the proposed

fast IAA implementation. Then, we show the speed gain of the fast

implementation of IAA in the numerical examples in Section 4. Fi-

nally, we conclude this paper in Section 5.

2. IAA

IAA is a WLS based data-dependent, nonparametric algorithm. Al-

though IAA can be used for spectral estimation of either a single

data sequence or multiple snapshots, we only consider the former

case. The extension of the algorithms in this paper to multiple snap-

shots is straightforward, and thus omitted.

Consider a uniformly sampled sequence of M samples, and

let yM denote the corresponding data vector.

[1,ejω,...,ej(M−1)ω]Tdenote the steering vector, where ω ∈

[0,2π) denotes the frequency. Consider a uniform frequency grid

with K grid points: ωk = 2πk/K, k = 0,1,...,K − 1. Define

AM ? [aM(ω0),aM(ω1),...,aM(ωK−1)]. The data model can

be written as:

yM = AMxK+ eM,

Let aM(ω) ?

(1)

where xK = [x0,x1,...,xK−1]T, with xk denoting the complex

amplitude associated with ωk, and eMdenotes the noise term. Given

yM and AM, IAA solves (1) by minimizing the following WLS cost

function:

?yM − aM(ωk)xk?2

Q−1

M(ωk),k = 0,1,...,K − 1,

(2)

where ?x?2

Q−1

M(ωk)? xHQ−1

M(ωk)x, and

QM(ωk) = RM − pkaM(ωk)aH(ωk),

is the IAA interference (signals at frequency grid points other than

ωk) and noise covariance matrix. In (3), pk = |xk|2denotes the

signal power at grid point ωk, and the IAA covariance matrix has

the expression:

RM = AMPKAH

(3)

M,

(4)

where PKis a diagonal matrix with diagonal entries from the vector

pK = [p0,p1,··· ,pK−1]T. Minimizing (2) with respect to xk, and

applying the matrix inversion lemma, we obtain the IAA estimate of

xk, xIAA

k:

xIAA

k

=

aH

M(ωk)R−1

M(ωk)R−1

MyM

MaM(ωk),

aH

k = 0,1,...,K − 1.

(5)

Using the following definitions

φN(ω) = aH

M(ω)R−1

MyM,

(6)

and

φD(ω) = aH

M(ω)R−1

MaM(ω),

(7)

we can rewrite (5) as

xIAA

k

=φN(ωk)

φD(ωk),k = 0,1,...,K − 1.

(8)

As information on the signal power, i.e., PK in (4), is required

in the IAA estimate in (5), IAA needs to be carried out in an iterative

way. In this paper, we use the periodogram, or the matched filtering

(MF), to initialize IAA.

We remark that although the IAA estimate in (5) appears to be

like what would be obtained by employing Capon [1] and APES [2],

these estimates are actually different in that Capon and APES form

ˆR orˆQ from the measured data, while IAA forms R or Q from the

spectral estimate obtained from the previous iteration’s power esti-

mate (e.g., as in (3), and (4)). This difference brings more structure

into the IAA covariance matrix R. In Capon or APES,ˆR orˆQ,

while having low displacement rank, does not have a Toeplitz struc-

ture. For IAA, R is Hermitian Toeplitz. Therefore, as shown in later

sections, the displacement rank of R for IAA is much lower than

that ofˆR for Capon and APES [4] [5]. In addition, the G-S factors

of R can also be computed more efficiently in IAA than in Capon or

APES.

3. FAST COMPUTATION OF IAA

We start by constructing the signal covariance matrix RM using the

knowledge of PK (obtained from the previous iteration) and the

FFT. Then, we proceed by introducing the displacement represen-

tation (G-S factors) of RM. Finally, we explain how to compute

the IAA estimate by FFT using the results generated in the previous

steps.

3.1. Formation of RM Using the FFT

Given the estimated signal power PK from a previous iteration, we

need to recompute the IAA covariance matrix RM. This step, (4),

of the original IAA in [3] requires O(M2K) flops, which is com-

putationally expensive, as K usually takes a much larger value (e.g.,

5-10 times) than M. However, we can save computations by exploit-

ing the fact that AM has a Vandermonde structure, which means that

RM is Toeplitz, in addition to being Hermitian. We rewrite RM as:

RM =

?

???

?

r0

r1

···

...

...

r−1

rM−1

...

r−1

...

r−M+1

r0

...

···

r1

r0

?

???

?

=

K

?

k=1

pkaM(ωk)aH

M(ωk).

(9)

Note that {rm}M−1

m=−M+1in (9) can be expressed as

rm =

K−1

?

k=0

pke−j2πmk/K,

m = −M + 1,...,−1,0,1,...,M − 1.

From (10), we see that RM can be computed from PK via the FFT

in just O(K log2(K)) flops.

(10)

4305

Page 3

3.2. G-S Decomposition of R−1

M

The complexity of matrix computations involving RM and R−1

where RM is a Toeplitz matrix, can be significantly reduced com-

pared to an arbitrary matrix. This computation reduction is not lim-

ited to the Toeplitz matrices only, but is applicable to a more general

category of matrices which have low displacement ranks [10].

The displacement representation of an M × M matrix T with

respect to matrices ZM and ZT

M,

Mis defined as:

?ZM,ZT

MT ? T − ZMTZT

M,

(11)

where ZM is an M × M strictly lower-triangular matrix in the gen-

eral case, and a lag-1 shifting matrix in the case of interest here:

ZM =

?

??

?

0

10

...

...

10

?

??

?.

(12)

It is well-known that ?ZM,ZT

rank-2 [8,11,12], given that RM is Toeplitz and non-singular. Using

this fact, one can arrive at the following G-S decomposition of R−1

(or, the G-S formula) [8,11,12]:

MR−1

M, like ?ZM,ZT

MRM, is of

M

R−1

M= LM(tM,ZM)LH

− LM(sM,ZM)LH

M(tM,ZM)

M(sM,ZM),

(13)

where

tM =

1

√αM−1

?

1

wM−1

?

and

sM =

1

√αM−1

?

0

ˇ w∗

M−1

?

.

In (13), LM(xM,Zk) denotes the so-called Krylov matrix with

M columns generated by a vector xM and ZM, which is a lower-

triangular Toeplitz matrix:

LM(xM,ZM) ? [xM,ZMxM,...,ZM−1

Furthermore, wM−1 and αM−1 are parameters that can be effi-

ciently computed in O(M2) flops by solving a Toeplitz system in-

volving RM using a Levinson-Durbin-type algorithm [12,13].

M

xM].

(14)

3.3. Fast Implementation of IAA

3.3.1. The Computation of φN(ω)

We decompose the computation of φN(ω) = aH

steps: zM = R−1

R−1

a series of Toeplitz matrix-vector products, and thus FFT/IFFT [12].

The second step φN(ω) = aH

ωk, k = 0,1,...,K − 1, as AH

padded IFFT of zM. Notice that the second step is also used for the

periodogram initialization of IAA.

M(ω)R−1

MyM in two

MyM and φN(ω) = aH

M(ω)zM. After expressing

MyM can be computed by

Mas in (13), the first step zM = R−1

M(ω)zM can be computed for all ω =

MzM, which is simply the zero

3.3.2. The Evaluation of φD(ω)

Forauniformlysampledsequenceandauniformscanninggridinthe

frequency domain, the evaluation of φD(ω) in (7) can be transformed

into computing the coefficients of a univariate polynomial on the

unit circle, which is followed by performing FFT on the polynomial

coefficients [4, 5, 8, 9]. We follow the basic idea in [4] but use a

more efficient coefficient computing scheme for R−1

we further exploit the Hermitian property of RM, in addition to its

Toeplitzstructuretoreducethecomputationalcost. Comparedto[4],

our discussion here is more direct.

Define a polynomial related to R−1

Min IAA, where

Min (7) and (13) as

φ(z) = αT

M(z−1)R−1

= αT

MαM(z)

M(z−1)[LM(tM,ZM)LH

− LM(sM,ZM)LH

M−1

?

m=−M+1

M(tM,ZM)

M(sM,ZM)]αM(z)

(15)

=

cmzm,

(16)

where

αM(z−1) = [1,z−1,...,z−M+1]T,

αM(z) = [1,z,...,zM−1]T,

andcmisthecoefficientforzm. Then, φD(ω) = aH

in (7) corresponds to the evaluation of φ(z) at the point z = ejω:

M(ω)R−1

MaM(ω)

φD(ω) = φ(z)|z=ejω =

M−1

?

m=−M+1

cmejωm,

(17)

When the scanning frequency grid {ωk}K−1

k = 0,1,...,K − 1, can be efficiently computed via applying the

zero padded FFT to the coefficients {cm}M−1

Let c ? [c−M+1,··· ,c−1,c0]T. It follows from [4] that:

c = LM(˜tM,ZM)t∗

where ˜ xM = [xM−1,2xM−2,...,Mt0]T.

is a summation of Toeplitz matrix-vector products, which can be

computed efficiently by FFT/IFFT. Since R−1

{φD(ωk)}K−1

c−m = c∗

and this concludes the fast computation of the {cm}M−1

(17).

The fast IAA algorithm can be summarized as follows:

k=0is uniform, φD(ωk),

m=−M+1.

M− LM(˜ sM,ZM)s∗

M,

(18)

We note that (18)

Mis Hermitian and

k=0are real-valued, from (17) we have

m,m = 0,1,...,M − 1,

(19)

m=−M+1in

1 Compute the periodogram initial estimate pK using IFFT, as

mentioned in Section 3.3.1, with O(K log2K) flops.

For each iteration of IAA, do the following:

2 Given pKfrom the previous iteration, compute r, which con-

tains the distinct elements of RM, via FFT, as in (10), with

O(K log2K) flops.

3 Given r, compute the generators, tM and sM, of R−1

Section 3.2, with M2flops.

4 Given the generators of R−1

using FFT/IFFT as discussed in Section 3.3. Since the G-

S factorization of R−1

O(K log2K) flops to compute φN(ω) and φD(ω). The new

IAA estimate {xIAA

computed in 2K flops.

M, as in

M, compute φN(ω) and φD(ω)

Mis used, it takes O(M log2M) +

k}K−1

k=0of (8), and thus pK, can then be

We note that the main computational burden of the fast IAA al-

gorithm is Step 3, which requires M2flops. In contrast, the existing

IAA algorithm [3] would require O(M2K) flops in each iteration.

4306

Page 4

4. NUMERICAL EXAMPLES

In this section, we compare the computational complexity of the fast

IAA with that of the existing IAA algorithm [3] via numerical exam-

ples. We generate the data sequences according to (1), and use unit

power noise and signal powers around 10 dB. The simulations are

conducted in MATLAB

with 8-core 2.27 GHz CPU and 24 GB RAM.

R ?using only one CPU core of a workstation

0123456

?40

?30

?20

?10

0

10

20

Frequency

Signal Power / dB

0123456

?40

?30

?20

?10

0

10

20

Frequency

Signal Power / dB

(a) IAA-GS(b) IAA

Fig. 1. Spectral estimate provided by (a) the fast IAA and (b) the

existing IAA for a case with M = 100 and K = 5M.

0 100200 300

M

400 500600

?20

?15

?10

?5

0

5

10

15

20

Computational Time

K=3M

K=5M

K=7M

K=9M

0 100200 300

M

400 500600

?20

?15

?10

?5

0

5

10

15

20

Computational Time

K=3M

K=5M

K=7M

K=9M

(a) IAA-GS(b) IAA

Fig. 2. Computational time for (a) the fast IAA and (b) the existing

IAA for the 1-D case. Note that the times are recorded in seconds

and plotted on a logarithmic scale (10log10(·)).

In Figure 1, we show a line spectral estimation example using

both IAA-GS and IAA, with M = 100 and K = 5M. The true sig-

nal powers are marked by light green circles. The computation times

needed by both IAA-GS and IAA are recorded in seconds and plot-

ted on a logarithmic scale (10log10(·)) in Figure 2 for various values

of M and K. We notice that for large values of M (e.g., M > 120),

IAA-GS can provide about two orders of magnitude improvement

compared with the existing IAA, while generating exactly the same

spectral estimate. As K/M increases, the computational complex-

ity of the fast IAA shows little increase, while that of the existing

IAA grows proportionately to the ratio of K/M. This fact makes the

fast IAA algorithm even more desirable for high resolution spectrum

analysis applications. From both the trend of the curves in Figure 2

and the computational cost analysis, we can conjecture that, as the

sequence length M increases, the computational savings provided

by IAA-GS will become even more significant compared with the

existing IAA algorithm.

5. CONCLUSIONS

We have presented a fast implementation for 1-D IAA spectral es-

timation of a uniformly sampled data sequence. As the G-S-type

factorization of the covariance matrix is used, we are able to lever-

age the Toeplitz structure and thus the use of FFT/IFFT to compute

the IAA spectral estimate. By associating the IAA estimate with

a univariate polynomial on the unit circle, the computational com-

plexity of IAA is reduced from O(M2K) to O(M2). Significant

computational reductions have been demonstrated using numerical

examples.

6. REFERENCES

[1] J. Capon, “High resolution frequency-wavenumber spectrum

analysis,” Proceedings of the IEEE, vol. 57, pp. 1408–1418,

August 1969.

[2] J. Li and P. Stoica, “An adaptive filtering approach to spectral

estimation and SAR imaging,” IEEE Transactions on Signal

Processing, vol. 44, no. 6, pp. 1469–1484, June 1996.

[3] T. Yardibi, J. Li, P. Stoica, M. Xue, and A. B. Baggeroer,

“Source localization and sensing: A nonparametric iterative

adaptive approach based on weighted least squares,”

TransactionsonAerospaceandElectronicSystems, vol.46, no.

1, pp. 425 – 443, Jan. 2010.

[4] G. Glentis, “A fast algorithm for APES and Capon spectral

estimation,” IEEE Transactions on Signal Processing, vol. 56,

no. 9, pp. 4207–4220, September 2008.

[5] E. Larsson and P. Stoica,

dimensional APES and Capon spectral estimators,” Multidi-

mensional Systems and Signal Processing, vol. 13, pp. 35–54,

January 2002.

[6] T. Ekman, A. Jakobsson, and P. Stoica, “On the efficient im-

plementation of the Capon spectral estimator,” Proceedings

of European Signal Processing Conference, Tampere, Finland,

September 4-8 2000.

[7] Z. Liu, H. Li, and J. Li, “Efficient implementation of Capon

and APES for spectral estimation,”

Aerospace and Electronic Systems, vol. 34, no. 4, pp. 1314–

1319, October 1998.

[8] B. R. Musicus, “Fast MLM power spectrum estimation from

uniformly spaced correlations,” IEEE Transactions on Accous-

tics, Speech, and Signal Processing, vol. 33, no. 4, pp. 1333–

1335, October 1985.

[9] A. Jakobsson, S. L. Marple, and P. Stoica, “Computationally

efficient two-dimensional Capon spectrum analysis,”

Transactions on Signal Processing, vol. 48, no. 9, pp. 2651–

2661, September 2000.

[10] T. Kailath and A. H. Sayed, “Displacement structure: Theory

and applications,” SIAM Review, vol. 37, no. 3, pp. 297–386,

September 1995.

[11] P. Stoica and R. L. Moses,

Prentice-Hall, Upper Saddle River, NJ, 2005.

[12] J. R. Jain, “An efficient algorithm for a large Toeplitz set of

linear equations,” IEEE Transactions on Acoustics, Speech,

and Signal Processing, vol. 27, no. 6, pp. 612–615, December

1979.

[13] S. Zohar, “The solution of a Toeplitz set of linear equations,”

Journal of the Association for Computing Machinery, vol. 21,

no. 2, pp. 272–276, April 1974.

IEEE

“Fast implementation of two-

IEEE Transactions on

IEEE

Spectral Analysis of Signals,

4307