Robust ML estimation for unknown numbers of signals: Performance study

Abstract

We study the performance of a recently proposed robust ML estimation procedure for unknown numbers of signals. This approach finds the ML estimate for the maximum number of signals and selects relevant components associated with the true parameters from the estimated parameter vector. Its computational cost is significantly lower than conventional methods based on information theoretic criteria or multiple hypothesis tests. We show that the covariance matrix of relevant estimates is upper and lower bounded by two covariance matrices. These bounds are easy to compute by existing results for standard ML estimation. Our analysis is further confirmed by numerical experiments over a wide range of SNRs.

Full text

Page 1

ROBUST ML ESTIMATION FOR UNKNOWN NUMBERS OF SIGNALS:
PERFORMANCE STUDY
Pei-Jung Chung
Institute for Digital Communications
School of Engineering and Electronics, The University of Edinburgh, UK
peijung.chung@gmail.com
ABSTRACT
We study the performance of a recently proposed robust
ML estimation procedure for unknown numbers of signals.
This approach finds the ML estimate for the maximum num-
berofsignalsandselects relevantcomponentsassociatedwith
the true parameters from the estimated parameter vector. Its
computational cost is significantly lower than conventional
methods based on information theoretic criteria or multiple
hypothesis tests. We show that the covariance matrix of rele-
vant estimates is upper and lower bounded by two covariance
matrices. These bounds are easy to compute by existing re-
sults for standard ML estimation. Our analysis is further con-
firmed by numerical experiments over a wide range of SNRs.
1. INTRODUCTION
The problem of estimating direction of arrival (DOA) plays a
key role in array processing, radar, communications and geo-
physics. The maximum likelihood (ML) approach is charac-
terizedbyexcellentstatistical propertiesandrobustnessagainst
small sample numbers, signal coherence and closely located
sources.
The standard ML method assumes the number of signals,
m, to be known and maximizes the concentrated likelihood
function over an m-dimensional parameter space. In the case
ofunknownnumbersofsignals,conventionalinformationthe-
oretic criterion based approaches [7] or multiple hypothesis
testing procedures [3] compute the ML estimates for a se-
quence of model orders and select the best estimate accord-
ing to the underlying criterion. The total computational cost
can be very high due to the multi-dimensionalsearch for each
candidate model order.
In contrary to the aforementioned methods, the recently
proposed ML estimation procedure [2] carries out optimiza-
P.-J. Chung acknowledges support of her position from the Scottish
Funding Council and their support of the Joint Research Institute with the
Heriot-Watt University as a component part of the Edinburgh Research Part-
nership.
tion for the maximum possible number of signals. The result-
ing parameter vector contains relevant components that are
associated with the true parameters. They can be recognized
on the relevance value that measures each component’s con-
tribution to the likelihood function. Further, the number of
relevant components provides an estimate for the number of
signals.
The purposeof this contributionis to get moreinsightinto
the performance of this algorithm. In particular, we shall in-
vestigate estimation errors of relevant estimates. The main
difficultyhereisthataoverparameterizedmodelisnotuniquely
identified. We overcome this problem by introducing pseudo
sourceswithnegligiblepowers. Basedonthismodifiedmodel,
an upper bound and a lower bound on the covariance matrix
of relevant estimates are derived.
This paper is outlined as follows. Section 2 gives a brief
description of the data model. The robust ML direction find-
ing algorithm for unknown numbers of signals [2] is summa-
rized in Section 3. We develop the main result, upper and
lower bounds on the covariance matrix, in Section 4. Numer-
ical results are presented in Section 5. Concluding remarks
are given in Section 6.
2. SIGNAL MODEL
Consider an array of n sensors receiving m narrow band sig-
nalsemittedbyfar-fieldsourceslocatedatθm=[θ1,...,θm]T.
The array output x(t) is described as
x(t) = Hm(θm)sm(t) + n(t),t = 1,...,T,
(1)
where the ith column d(θi) of
Hm(θm)=[d(θ1)···d(θi)···d(θm)]
represents the steering vector associated with the signal ar-
riving from θi. The signal vector sm(t) is considered as a
stationary, temporally uncorrelated complex normal process
withzeromeanandcovariancematrixCsm= E[sm(t)sm(t)?]
where (·)?denotes the Hermitian transpose. The noise vector
(2)
86
978-1-4244-2241-8/08/$25.00 ©2008 IEEE

Page 2

n(t) is a spatially and temporally uncorrelated complex nor-
malprocesswithzeromeanandcovariancematrixνInwhere
ν is the noise spectral parameter and Inis an n × n identity
matrix. Under these assumptions, the array output x(t) is
complex normally distributed with zero mean and covariance
matrix
Cx= Hm(θm)CsmHm(θm)?+ νIn.
(3)
Based on array observations {x(t)}T
number of signals, m, the ML estimateˆθmis obtained by
minimizing the negative concentrated likelihood function [1]
t=1and a pre-specified
lT(θm) = logdet?Pm(θm)ˆCxPm(θm)+ˆ νP⊥
ˆ ν =
n − mtr?P⊥
where Pm(θm) represents the projectionmatrix onto the col-
umn space of Hm(θm) and P⊥
ˆCx=1
T
?T
trix. The problem of central interest is to estimate the DOA
parameterswhen the true numberof signals, m0, is unknown.
m(θm)?, (4)
1
m(θm)ˆCx
?
(5)
m(θm) = In− Pm(θm).
t=1x(t)x(t)?denotes the sample covariance ma-
3. ROBUST ML ESTIMATION FOR UNKNOWN
NUMBERS OF SIGNALS
From the asymptotic property of ML estimation under mis-
specified numbers of signals, we know that for m > m0, the
ML estimateˆθmcontains (m−m0) componentscorrespond-
ing to the true parameters in θ0[4]. Moreover, the value of
thelikelihoodfunctionattainsits minimumat m0andremains
constant for m ≥ m0. Motivated by this observation,we pro-
posed a robust ML estimation procedure in [2] that requires
only an upper bound on the number of signals M. The likeli-
hood function is maximized over an M-dimensional parame-
ter space. From the resulting estimate, we select the relevant
components according to their contribution to the likelihood
function.
More precisely, letˆθMdenote the ML estimate obtained
by minimizing the negative log-likelihood function (4) with
m = M:
ˆθM= argmin
θM
lT(θM).
(6)
Since M ≥ m0, the M × 1 vectorˆθM= [ˆθ1,...,ˆθM]Tcon-
tains more elements than the true parameter vector θ0does.
For large T, a subset of the elements inˆθM coincide with
those of θ0. The elements ofˆθM that are associated with
θ0, are referred to as relevant estimates.
To select the relevant components, we compute the rele-
vant value for each componentˆθias follows
R(ˆθi) =
1
lT(ˆθM)[lT(˜θi) − lT(ˆθM)],
(7)
where˜θicontains all elements ofˆθM except the ith compo-
nentˆθi:
˜θi= [ˆθ1,...,ˆθi−1,ˆθi+1,...,ˆθM].
Note that the log-likelihood lT(˜θi) is computed by (4) us-
ing the (M − 1) dimensional vector˜θi. The value of R(ˆθi)
indicates the contribution of the ith elementˆθmto the log-
likelihood function. The normalizing factor 1/lT(ˆθM) is
used to improve numerical stability. As pointed out in [2],
a high relevance value R(ˆθi) implies that ˆθi is associated
with one of θ0’s components. A low relevance value (usually
close to zero) indicates thatˆθi does not correspond to any
true parameter. Based on this observation, we decideˆθito be
a relevant estimate if R(ˆθi) exceeds a given threshold α.
(8)
Let R(1)≥ R(2)≥ ... ≥ R(M)denote the ordered rele-
vancevalues with corresponding{ˆθ(1),ˆθ(2),...,ˆθ(M)}. The set
of relevant estimates is given by
S = {ˆθ(1),ˆθ(2),...,ˆθ(k)}
(9)
where
R(1)≥ R(2)≥ ... ≥ R(k)≥ α.
Wealsodefinethek×1relevantvectorasˆθ0= [ˆθ(1),...,ˆθ(k)]T.
Asymptotically the number of components in S orˆθ0coin-
cides with the true number of signals m0.
(10)
4. COVARIANCE MATRIX OF RELEVANT
ESTIMATES
The consistency property of ML estimation has been estab-
lished previously [4]. In the variance study, the main chal-
lengeis thatanoverparameterizedmodelis notuniquelyiden-
tified. The asymptotic covariance matrix represents a class of
matrices involving generalizedinverse, rather than a uniquely
defined matrix [5]. In the following, rather than applying the
elegant results of [5] directly, we shall take a more practical
approach to assess estimation errors. More specifically, we
show that the asymptotic covariance matrix of relevant esti-
mates is lower and upper bounded by two covariance matri-
ces associated with m0and (M + 1) signals in standard ML
estimation.
To tackle the identification problem, we consider a modi-
fied version of the signal model (1) :
x(t) = H0(θ0)s0(t) + H?(θ?)s?(t) + n(t),
(11)
where the subscript (·)0is used to emphasize that the steering
matrixH0(θ0) andthesignalvectors0(t) areassociatedwith
the real signal sources. The n×(M−m0) matrix H?(θ?) de-
notes the steering matrix associated with the pseudo sources.
The (M − m0) × 1 signal vector s?(t) corresponding to the
87

Page 3

pseudo sources is independent of s0(t) with the covariance
matrix
C?= E[s?(t)s?(t)?] = ?In
(12)
where ? > 0 is positive with small magnitude compared to
the real signal powers, i.e. ? << tr(Cs0). The covariance
matrix of array outputs (3) becomes
Cx= H0Cs0H?
0+ H?C?H?
?+ νIn
(13)
whereH0=H0(θ0), H?=H?(θ?)andCs0= Es0(t)s0(t)?.
Note that for ? → 0, (11) and (13) approach to the true signal
model and covariance matrix, respectively. Without loss of
generality, we assume H = [H0H?] is of full rank.
Based on the model (11) and results from standard ML
estimation [6], we shall derive the upper and lower bounds on
the asymptotic covariance matrix of relevant estimates
K0= E[(ˆθ0− θ0)(ˆθ0− θ0)T].
(14)
4.1. Existing Results for Standard ML Estimation
The following properties for standard ML estimation [6] are
essential to the proof of Theorem 1.
Result 1 (Explicit formula for asymptotic covariancematrix)
Suppose the signal model (1) and the number of signals, m,
to be known. The asymptotic covariance matrix of the ML
estimator K(m) = E[(ˆθm− θm)(ˆθm− θm)T] is given by
ν
2T
?
Re
?
(DH
mP⊥
mDm)?(CsmHH
mC−1
xmHmCsm)T?
,
?−1
(15)
where ? denotes the Hadamard elementwise product. The
n × m matrix Dm = [d?(θ1)···d?(θm)] contains the first
derivative of the steering vectors d?(θi)=∂d(θi)/∂θi.
Result 2 ( R7 of [6] ) If the (m + 1)st source is uncorrelated
with the other m, then K(m) increases monotonically with
increasing m. More specifically,
[ Im 0 ]K(m + 1)
?
Im
0
?
≥ K(m) ,
(16)
where Imis an m × m identity matrix and the (m + 1) ×
(m+1)matrixK(m+1)denotesthe covariancematrixwhen
(m + 1) signals are present.
4.2. Upper bound and lower bound on K0
We introduced (M − m0) pseudo signals in the approximate
signal model (11) so that the parameter θM = [ θT
is uniquely identified. The normalized estimator√T(ˆθM−
θM) has asymptotic normal distribution with zero mean and
covariance matrix K(M). This matrix varies with different
0
θT
?]T
values of θ?, ?. However, for any choice of the (M + 1)st
uncorrelated signal, K(M) is upper bounded by K(M + 1).
According to Result 2:
[ IM 0 ]K(M + 1)
?
IM
0
?
≥ K(M) ,
(17)
where K(M + 1) is the covariance matrix when m = M +
1. Multiplying both sides of (17) with [ Im00M−m0] and
[ Im00M−m0]Twhere 0M−m0denotes an (M − m0) ×
(M − m0) zero matrix, we obtain an upper bound for the
covariance matrix K0as follows,
[ Im00M+1−m0]K(M+1)
?
Im0
0M+1−m0
?
≥ K0. (18)
The lower bound is derived by applying Result 2 again.
More specifically,
K0= [ Im00M−m0]K(M)
?
Im0
0M−m0
?
≥ K(m0)
(19)
where K(m0) is the covariance matrix for the standard ML
estimatior with m = m0. Eq. (19) shows that the estimation
accuracy is affected by overparameterization. By eq. (18)
we show that the impact of model order mismatch on esti-
mation performance is equivalent to that caused by adding
(M + 1 − m0) additional signals to the original m0signals.
A great advantageof the aboveanalysis is that K(M +1)
and K(m0) can be easily computed by the existing formula
(15). TheparametervectorθM+1= [θT
at which K(M +1) is evaluated has the true parameter θ0as
its first m0elements. The remaining parameters can be cho-
sen arbitrarily but has to ensure the corresponding steering
matrix HM+1to be full rank. The matrix K(m0) is evalu-
ated at the true parameter θ0. The lower bound can only be
achieved when M = m0, i.e. when no model order mismatch
is present. Although these bounds are not necessarily tight,
theyprovidea convenientmethodtoevaluatetheperformance
of an overparameterized model. The following theorem sum-
marizes the above discussion.
0, θm0+1,··· ,θM+1]T
Theorem 1
The asymptotic covariance matrix K0= E[(ˆθ0− θ0)(ˆθ0−
θ0)T] whereˆθ0consists of relevant estimates is upper and
lowerboundedbyK(M+1)andK(m0), respectively. More
precisely,
Consider the algorithm described in Section 3.
[Im00M+1−m0]K(M+1)
?
Im0
0M+1−m0
?
≥ K0≥ K(m0).
(20)
The covariancematrices K(M +1) and K(m0) can be com-
puted by the closed form expression for covariance matrix of
standard ML estimaion (15).
88

Page 4

5. SIMULATION
In the simulation, a uniform linear of 10 sensors with inter-
element spacings of half a wavelength is employed. The nar-
row band signals are generated by m0= 2 uncorrelated sig-
nals located at θ0= [28◦36◦] of various strengths. The data
lengthis givenbyT = 100. Thedifferenceofsignalstrengths
is [ 1 0 ] dB where 0 dB corresponds to the reference signal.
The signal to noise ratio (SNR) varies from −10 to 10 dB in
a 2 dB step. We consider two upper bounds on the number
of signals: M = 3 and M = 4. The later represents a larger
mismatch in the model order. The threshold α is chosen to be
0.015. Each experiment performs 200 Monte Carlo trials.
Fig 1. shows the squareroot of empiricalvariances forˆθ1,
ˆθ2. The lower bound and upper bound computed from The-
orem 1 are denoted by LBiand UBi, i = 1,2, respectively.
We can observe that the variances decrease with increasing
SNR. As predicted by our analysis, var(ˆθ1) and var(ˆθ2) are
lower bounded by the variances for the optimal case, K(2),
and upper bounded by the worst case, K(4). Note that K(2)
is calculated at θ0 = [28◦36◦] and K(4) is calculated at
[28◦36◦20◦50◦] by eq (15). The reletive strength of the ad-
ditional two signals is 0 dB.
In Fig. 2 we observe that the empirical variances for
M = 4 increase due to a larger mismatch in numbers of sig-
nals. However, they are still well boundedby the two theoret-
icalboundsobtainedfromK(2) andK(5). ThematrixK(5)
is calculated at [28◦36◦20◦50◦80◦]. The reletive strength
of the additional three signals is 0 dB.
In summary, the variance of relevant estimates is larger
than that in the optimal case where no mismatch is present,
i.e. m = m0. Furthermore, it is also upper bounded by the
corresponding diagonal elements of K(M + 1) as predicted
by Theorem 1.
6. CONCLUSION
We studied performance of a recently proposed robust ML
direction finding algorithm for unknown numbers of signals.
TheunderlyingalgorithmcomputesMLestimatesforthemax-
imally hypothesized model and select DOA estimates by the
relevant value associated with each component. Due to over-
parameterization,the signal model and the asymptoticcovari-
ance matrix are not uniquely identifiable. We overcome this
difficulty by considering an approximate signal model with
pseudosignals ofnegligiblestrengthsandderivean upperand
lower bound on the varince of relevant estimates. The impact
of overparameterization on variance is similar to that of in-
troducingadditional signals. Simulation results under various
settings show good agreement with the theoretical analysis.
−10 −8−6 −4−20246810
10
−2
10
−1
10
0
10
1
?
var(ˆ θ1)
Variance, M=3


θ1
LB1
UB1
−10−8 −6−4−20246810
10
−2
10
−1
10
0
10
1
SNR
?
var(ˆ θ2)


θ2
LB2
UB2
Fig. 1. Square root of empirical variances of relevant estimates. m0= 2,
M = 3.
Variance, M=4
−10−8−6 −4−202468 10
10
−2
10
−1
10
0
10
1
?
var(ˆ θ1)


θ1
LB1
UB1
−10−8 −6−4 −202468 10
10
−2
10
−1
10
0
10
1
SNR
?
var(ˆ θ2)


θ2
LB2
UB2
Fig. 2. Square root of empirical variances of relevant estimates. m0= 2,
M = 4.
89

Page 5

7. REFERENCES
[1] J. F. B¨ ohme. Statistical array processing of measured
sonar and seismic data. In Proc. SPIE 2563 Advanced
Signal Processing Algorithms, pp. 2-20, San Diego, Jul
1995.
[2] P.-J. Chung. Robust ML estimation for unknown num-
bers of signals. In Proc. EUSIPCO, Pozna´ n, Poland,
Septermber 2007.
[3] P.-J.Chung and J. F. B¨ ohme and C. F. Mecklenbr¨ auker
and A. O. Hero. Detection of the Number of Signals
Using the Benjamini-Hochberg Procedure. IEEE Trans.
Signal Processing, 55(6), pp. 2497-2508,June 2007.
[4] P.-J. Chung. Stochastic maximum likelihood estimation
undermisspecifiednumbersofsignals, IEEETrans.Sig-
nal Processing, Vol 55(9), pp. 4726 - 4731, September
2007.
[5] A. Shapiro. Asymptotic theory of overparameterized
structural models. Journal of American Association,
81(393):142-149,March 1986.
[6] P. Stoica and A. Nehorai. Performance study of condi-
tional and unconditional direction-of-arrival estimation.
IEEE Trans. Acoustics, Speech, and Signal Processing,
38(10):1783-1795,October 1990.
[7] M. Wax and I. Ziskind. Detection of the number of
coherent signals by the MDL Principle. IEEE Trans.
Acoust., Speech, Signal Processing, 37(8), pp. 1190-
1196, August 1989.
90