No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
A modified version of the Pisarenko method to estimate the power spectral density of any asymptotically wide sense stationary vector process
Abstract
The Pisarenko method estimates the power spectral density (PSD) of Gaussian wide sense stationary (WSS) 1-dimensional (scalar) processes when the PSD is Lipschitz. In this paper we modify the Pisarenko method to estimate the generating function of a sequence of block Toeplitz matrices from another sequence of matrices when both sequences are asymptotically equivalent. This modified version of the Pisarenko method allows us to estimate the PSD of any asymptotically WSS (AWSS) multidimensional (vector) process even if the process is not Gaussian and even if the PSD is continuous, but not Lipschitz.
... In this paper, we use the Cholesky decomposition to give a parameter estimation method for any perturbed VAR or VMA process, whenever the sequence of correlation matrices of the perturbed process is asymptotically equivalent to the sequence of correlation matrices of the original process in the Gray sense [1]. Specifically, our parameter estimation method combines the Cholesky decomposition with the periodogram method for perturbed block Toeplitz matrices presented in [2]. In order to combine them, we first need to generalize a result given in [3] on the Cholesky decomposition of Toeplitz matrices to perturbed block Toeplitz matrices. ...
... The paper is organized as follows. In Section 2, we set up notation and we review the periodogram method for perturbed block Toeplitz matrices presented in [2]. In Section 3, we generalize a result given in [3] on the Cholesky decomposition of Toeplitz matrices to perturbed block Toeplitz matrices. ...
... In this section, we set up notation and we review the periodogram method for perturbed block Toeplitz matrices presented in [2]. ...
In this paper, we combine the periodogram method for perturbed block Toeplitz matrices with the Cholesky decomposition to give a parameter estimation method for any perturbed vector autoregressive (VAR) or vector moving average (VMA) process, when we only know a perturbed version of the sequence of correlation matrices of the process. In order to combine the periodogram method for perturbed block Toeplitz matrices with the Cholesky decomposition, we first need to generalize a known result on the Cholesky decomposition of Toeplitz matrices to perturbed block Toeplitz matrices.
... If F(ω) is Hermitian for all ω ∈ R (or equivalently, T n (F) is Hermitian for all n ∈ N (see, e.g., [6], Theorem 4.4), then inf F denotes inf ω∈ [0,2π] λ N (F(ω)). We recall that (see [7], Proposition 3): 2π] λ N (F(ω)). ...
... Proof. As {T n (X)} = {E x n:1 x * n:1 }, T n (X) is positive semidefinite for all n ∈ N. Consequently, from [7], Proposition 3, X(ω) is positive semidefinite for all ω ∈ R. Therefore, applying Equation (1), inf X > 0 if and only if det(X(ω)) = 0 for all ω ∈ R. Equation (17) is a direct consequence of Equation (1), Theorem 1, and Lemma 6. Theorem 4. Let {x n } be a VMA(q) process as in Definition 2. Suppose that det(Λ) = 0 and { (T n (G)) −1 2 } is bounded with G(ω) = I N + ∑ q k=1 e −kωi G −k for all ω ∈ R. If {x n } is real and Gaussian, and D ∈ 0, inf n∈N λ nN E x n:1 x n:1 , there exists K ∈ [0, ∞) such that: ...
In this paper, we study the asymptotic optimality of a low-complexity coding strategy for Gaussian vector sources. Specifically, we study the convergence speed of the rate of such a coding strategy when it is used to encode the most relevant vector sources, namely wide sense stationary (WSS), moving average (MA), and autoregressive (AR) vector sources. We also study how the coding strategy considered performs when it is used to encode perturbed versions of those relevant sources. More precisely, we give a sufficient condition for such perturbed versions so that the convergence speed of the rate remains unaltered.
... dω. From Equation (12), ([1] (Theorem 6.6)), and ( [14] (Proposition 2)) yields ...
... x n:1 x n:1 } = {T n (X)} ∼ {T n (X)}. Theorem 5 now follows from ( [14] (Proposition 3)) and Theorem 4. ...
In this paper, we present a low-complexity coding strategy to encode (compress) finite-length data blocks of Gaussian vector sources. We show that for large enough data blocks of a Gaussian asymptotically wide sense stationary (AWSS) vector source, the rate of the coding strategy tends to the lowest possible rate. Besides being a low-complexity strategy it does not require the knowledge of the correlation matrix of such data blocks. We also show that this coding strategy is appropriate to encode the most relevant Gaussian vector sources, namely, wide sense stationary (WSS), moving average (MA), autoregressive (AR), and ARMA vector sources.
... Proof. From [13,Proposition 3] we have that X(ω) is positive semidefinite for all ω ∈ R, and consequently, λ N (X(ω)) ≥ 0 for all ω ∈ R. Therefore, as det(X(ω)) = 0 for all ω ∈ R, λ N (X(ω)) > 0 for all ω ∈ R. Hence, using [13, Proposition 3] we obtain inf n∈N λ nN E x n:1 x n:1 = inf n∈N λ nN (T n (X)) = min ω∈ [0,2π] λ N (X(ω)) > 0. ...
In the present article, the differential entropy rate and the rate distortion function (RDF) are computed for certain nonstationary real Gaussian autoregressive moving average (ARMA) vector sources.
Для определенных нестационарных источников вещественных гауссовских векторных процессов авторегрессии со скользящим средним (ARMA-процессов) вычисляются скорость создания дифференциальной энтропии и скорость как функция искажения.
As the correlation matrices of stationary vector processes are block Toeplitz, autoregressive (AR) vector processes are non-stationary. However, in the literature, an AR vector process of finite order is said to be “stationary” if it satisfies the so-called stationarity condition (i.e., if the spectral radius of the associated companion matrix is less than one). Since the term “stationary” is used for such an AR vector process, its correlation matrices should “somehow approach” the correlation matrices of a stationary vector process, but the meaning of “somehow approach” has not been mathematically established in the literature. In the present paper we give necessary and sufficient conditions for AR vector processes to be “stationary”. These conditions show in which sense the correlation matrices of an AR “stationary” vector process asymptotically behave like block Toeplitz matrices. Applications in information theory and in statistical signal processing of these necessary and sufficient conditions are also given.
The theory of generalized locally Toeplitz (GLT) sequences is a powerful apparatus for computing the asymptotic spectral distribution of matrices A_n arising from virtually any kind of numerical discretization of differential equations (DEs). Indeed, when the mesh fineness parameter n tends to infinity, these matrices A_n give rise to a sequence {A_n }_n, which often turns out to be a GLT sequence or one of its "relatives", i.e., a block GLT sequence or a reduced GLT sequence. In particular, block GLT sequences are encountered in the discretization of systems of DEs as well as in the higher-order finite element or discontinuous Galerkin approximation of scalar DEs. Despite the applicative interest, a solid theory of block GLT sequences has been developed only recently, in 2018. The purpose of the present paper is to illustrate the potential of this theory by presenting a few noteworthy examples of applications in the context of DE discretizations.
Let k( ⋅ , ⋅ ) be a continuous kernel defined on Ω × Ω, Ω compact subset of , , and let us consider the integral operator from into ( set of continuous functions on Ω) defined as the map is a compact operator and therefore its spectrum forms a bounded sequence having zero as unique accumulation point. Here, we first consider in detail the approximation of by using rectangle formula in the case where Ω = [0,1], and the step is h = 1 ∕ n. The related linear application can be represented as a matrix An of size n. In accordance with the compact character of the continuous operator, we prove that {An} ∼ σ0 and {An} ∼ λ0, that is, the considered sequence has singular values and eigenvalues clustered at zero. Moreover, the cluster is strong in perfect analogy with the compactness of . Several generalizations are sketched, with special attention to the general case of pure sampling sequences, and few examples and numerical experiments are critically discussed, including the use of GMRES and preconditioned GMRES for large linear systems coming from the numerical approximation of integral equations of the form (1) with and datum g(x). Copyright © 2014 John Wiley & Sons, Ltd.
The present monograph studies the asymptotic behaviour of
eigenvalues, products and functions of block Toeplitz matrices
generated by the Fourier coefficients of a continuous matrix-valued
function. This study is based on the concept of asymptotically equivalent
sequences of non-square matrices. The asymptotic results on
block Toeplitz matrices obtained are applied to vector asymptotically
wide sense stationary processes. Therefore, this monograph is a generalization
to block Toeplitz matrices of the Gray monograph entitled
“Toeplitz and circulant matrices: A review”, which was published in
the second volume of Foundations and Trends in Communications
and Information Theory, and which is the simplest and most famous
introduction to the asymptotic theory on Toeplitz matrices.
For the engineering community, Gray's tutorial monograph on Toeplitz and circulant matrices has been, and remains, the best elementary introduction to the Szego theory on large Toeplitz matrices. In this paper, the most important results of the cited monograph are generalized to block Toeplitz (BT) matrices by maintaining the same mathematical tools used by Gray, that is, by using asymptotically equivalent sequences of matrices. As applications of these results, the geometric minimum mean square error (MMSE) for both an infinite-length multivariate linear predictor and an infinite-length decision feedback equalizer (DFE) for multiple-input-multiple-output (MIMO) channels, are obtained as a limit of the corresponding finite-length cases. Similarly, a short derivation of the well-known capacity of a time-invariant MIMO Gaussian channel with intersymbol interference (ISI) and fixed input covariance matrix is also presented.
An explicit expression is derived for the minimum discrimination
information (MDI) measure with respect to Gaussian priors for sources
characterized by their mean and by any principal leading block of their
covariance matrix. An explicit expression is provided for the MDI
extension of the given partial covariance of the source with respect to
a Gaussian prior. For zero-mean sources and zero-mean Gaussian priors
that are asymptotically weakly stationary (AWS) processes, it is shown
that the asymptotic MDI measure equals half the Itakura-Saito distortion
measure between the asymptotic power spectral densities of the source
and prior. Asymptotic MDI modelling of a given AWS source by
autoregressive and autoregressive moving average models, which are AWS
models, is considered, and conditions are given for convergence of the
sample covariance estimator of the source to the stationary covariance
used in the modelling
The fundamental theorems on the asymptotic behavior of eigenvalues, inverses, and products of "finite section" Toeplitz matrices and Toeplitz matrices with absolutely summable elements are derived in a tutorial manner. Mathematical elegance and generality are sacrificed for conceptual simplicity and insight in the hopes of making these results available to engineers lacking either the background or endurance to attack the mathematical literature on the subject. By limiting the generality of the matrices considered the essential ideas and results can be conveyed in a more intuitive manner without the mathematical machinery required for the most general cases. As an application the results are applied to the study of the covariance matrices and their factors of linear models of discrete time random processes. Acknowledgements The author gratefully acknowledges the assistance of Ronald M. Aarts of the Philips Research Labs in correcting many typos and errors in the 1993 revision, Liu Mingyu in pointing out errors corrected in the 1998 revision, Paolo Tilli of the Scuola Normale Superiore of Pisa for pointing out an incorrect corollary and providing the correction, and to David Neuho# of the University of Michigan for pointing out several typographical errors and some confusing notation. For corrections, comments, and improvements to the 2001 revision thanks are due to William Trench, John Dattorro, and Young Han-Kim. In particular, Trench brought the Wielandt-Ho#man theorem and its use to prove strengthened results to my attention. Section 2.4 largely follows his suggestions, although I take the blame for any introduced errors. Contents 1
.<F3.852e+05> It is well known that the<F4.039e+05> generating<F3.852e+05> function<F3.69e+05> f<F4.126e+05> #<F3.69e+05> L<F2.722e+05> 1<F3.852e+05><F4.126e+05><F3.69e+05> ([-#,<F3.852e+05><F3.69e+05> #],<F4.094e+05><F3.852e+05> R) of a class of Hermitian Toeplitz matrices<F3.69e+05><F3.664e+05> An<F3.852e+05><F3.69e+05><F3.852e+05> (f) describes very precisely the spectrum of each matrix of the class [U. Grenader and G. Szego,<F4.039e+05> Toeplitz Forms and Their<F3.852e+05> Applications, 2nd ed., Chelsea, New York, 1984; E. E. Tyrtyshnikov,<F4.039e+05> Linear Algebra<F3.852e+05> Appl., 232 (1996), pp. 1--43]. In this paper we consider<F3.69e+05><F4.126e+05><F3.69e+05> n×n<F3.852e+05> block Toeplitz matrices with<F3.69e+05><F4.126e+05><F3.69e+05> m×m<F3.852e+05> blocks generated by a Hermitian matrix-valued generating function<F3.69e+05> f<F4.126e+05> #<F3.69e+05> L<F2.722e+05> 1<F3.852e+05><F4.126e+05><F3.69e+05> ([-#,<F3.852e+05><F3.69e+05> #],<F4.094e+05> C<F3.664e+05><F2...
An elementary and direct proof of the Szegö formula is given, for both eigen and singular values. This proof, which is based on tools from linear algebra and does not rely on the theory of Fourier series, simultaneously embraces multilevel Toeplitz matrices, block Toeplitz matrices and combinations of them. The assumptions on the generating
function f are as weak as possible; indeedf is a matrix-valued function of p variables, and it is only supposed to be integrable. In the case of singular values f(x), and hence the block p-level Toeplitz matrices it generates, are not even supposed to be square matrices. Moreover, in the asymptotic formulas for eigen and singular values the test functions involved are not required to have compact support.
The present paper considers a special class of vector random processes that we call multivariate asymptotically wide sense stationary (WSS) processes. A multivariate random process is said to be asymptotically WSS if it has constant mean and the sequence of its autocorrelation matrices is asymptotically equivalent (a.e.) to the sequence of autocorrelation matrices of some multivariate WSS process. It is shown that this class of processes contains meaningful processes other than multivariate WSS processes. In particular, we give sufficient conditions for multivariate moving average (MA) processes, multivariate autoregressive (AR) processes and multivariate autoregressive moving average (ARMA) processes to be asymptotically WSS. Furthermore, in order to solve multiple-input-multiple-output (MIMO) problems in communications and signal processing involving this kind of processes, we extend the Gray definition of a.e. sequences of matrices and his main results on these sequences to non-square matrices. As an example, the derived results on a.e. sequences of non-square matrices are applied to compute the differential entropy rate and the minimum mean square error (MMSE) for a linear predictor of a multivariate asymptotically WSS process.
We study the asymptotic behavior of the eigenvalues of Hermitian n × n block Toeplitz matrices Tn, with k × k blocks, as n tends to infinity. No hypothesis is made concerning the structure of the blocks. Such matrices {Tn} are generated by the Fourier coefficients of a Hermitian matrix-valued function , and we study the distribution of their eigenvalues for large n, relating their behavior to some properties of f as a function; in particular, we show that the distribution of the eigenvalues converges to a limit , and we explicitly compute in terms of f, showing that \int F\, d\mu_f=1/k\int\tr F(f). Some consequences of this distribution and some localization results for the eigenvalues of Tn are discussed. We also study the eigenvalues of the preconditioned matrices {Pn-1Tn}, where the sequence {Pn} is generated by a positive definite matrix-valued function p. We show that the spectrum of any Pn-1Tn is contained in the interval [r,R], where r is the smallest and R the largest eigenvalue of p-1f. We also prove that the first m eigenvalues of Pn-1Tn tend to r and the last m tend to R, for any fixed m. Finally, the exact limit value of the condition number of the preconditioned matrices is computed.
The estimation of spectra of random stationary processes is an important part of the statistics of random processes. There are several books on spectral analysis, e.g. Blackman & Tukey, Hannan, and Jenkins & Watts. As a rule, spectral estimators are quadratic functions of the realizations. Recently Capon suggested a new method for estimation of spectra of random fields, in which a non-quadratic function of the realization is used: he considered a homogeneous random field ξ(t, x1, x2), i.e. one which is stationary in time and space and a random function of the time and space co-ordinates t, x1, x2. For the sake of expository convenience we shall consider ordinary stationary processes of time only, ξ(t); the generalization of our results to the case of random fields is easy.
Comparison of the conventional spectral estimator and the ‘high-resolution’ estimator for an artificial example showed that the latter has less smoothing effect on the true spectrum (Capon). This was later confirmed by examples using real data (Capon). However, it was not clear whether for a finite realization the high-resolution estimator distorted the true spectrum, i.e. whether it behaved for example like a conventional estimator raised to some power.
In the present paper we introduce and study a new class of spectral estimators which are generally non-linear and non-quadratic functionals of the realizations. These estimators include the conventional and high-resolution ones, for which we shall give the approximate distributions. We derive under rather general conditions the limiting distribution of the new class of estimators, and illustrate them with several examples. As a matter of fact, these new estimators are weighted means of the eigenvalues of the covariance matrix, e.g. the arithmetic mean, geometric mean, and so on.
Since covariance matrices of weakly stationary random processes are Toeplitz, much of the theory involving asymptotic results for such processes is simply the theory of the asymptotic behavior of Toeplitz forms. The fundamental theorem of this type is the Szegö theorem on the asymptotic eigenvalue distribution of Toeplitz matrices. This theorem is often quoted but relatively little understood in the engineering literature. In this tutorial paper we prove the Szegiö theorem for the special case of finite-order Toeplitz matrices. In this setting the mathematical sophistication of the classical proofs is not required and the proof is both simple and intuitive--yet it contains the important concepts involved in the most general case.
The output of an array of sansors is considered to be a homogeneous random field. In this case there is a spectral representation for this field, similar to that for stationary random processes, which consists of a superposition of traveling waves. The frequency-wavenumber power spectral density provides the mean-square value for the amplitudes of these waves and is of considerable importance in the analysis of propagating waves by means of an array of sensors. The conventional method of frequency-wavenumber power spectral density estimation uses a fixed-wavenumber window and its resolution is determined essentially by the beam pattern of the array of sensors. A high-resolution method of estimation is introduced which employs a wavenumber window whose shape changes and is a function of the wavenumber at which an estimate is obtained. It is shown that the wavenumber resolution of this method is considerably better than that of the conventional method. Application of these results is given to seismic data obtained from the large aperture seismic array located in eastern Montana. In addition, the application of the high-resolution method to other areas, such as radar, sonar, and radio astronomy, is indicated.
Singular values and eigenvalues of non-Hermitian block Toeplitz matrices
- Tilli