No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
A Minimum Free-Energy Hybrid Algorithm for 2-D Spectral Estimation
Abstract
A separable quasi 2-D spectral estimation algorithm is presented.
This hybrid algorithm combines the 1-D minimum free-energy method with
the 1-D periodogram. The algorithm reduces to the corresponding maximum
entropy hybrid spectral estimator in some cases. The minimum free-energy
hybrid algorithm is preferable to the maximum entropy version when the
estimated autocorrelation sequence is not extendable or when the
signal-to-noise ratio is low. A simulation helps illustrate this point
Kiernan, P., Two-Dimensional Spectral Estimation by Free Energy Minimization,Digital Signal Processing6 (1996), 160–168.We present a high-resolution 2-D minimum free energy (MFE) spectral estimation technique which is a 2-D extension of the MFE spectral estimation method. We demonstrate the performance of the technique for spectral estimation of closely spaced 2-D sinusoids in white Gaussian noise. Results from our tests on the effect of signal processing temperature illustrate that our method provides accurate low model order autoregressive spectral estimation. The method provides superior spectral estimation with similar computational burden to that achieved with the Levinson algorithm for a number of cases involving sinusoids in white noise at various signal-to-noise ratio (SNR) levels. An example is provided. The method is faster than the modified covariance method (MCV). An example is given from a number of cases involving two closely spaced sinusoids at low SNR, where MFE estimates are as well resolved as MCV estimates. MFE models may also be used for correlation extension and for field modeling and synthesis. We indicate possible extensions to the MFE method for computational efficiency improvement, a priori temperature determination, and the use of higher-order statistics.
We extend the minimum free energy (MFE) parameter estimation method to 2-D fields. This 2-D MFE method may be used to determine autoregressive (AR) model parameters for spectral estimation of 2-D fields. It may also be used to provide AR models for texture synthesis. The performance of the technique for closely spaced sinusoids in white noise is demonstrated by numerical example. Better results can be achieved than with the multidimensional Levinson algorithm.
The author proposes a 2-D extension of the minimum free energy
(MFE) parameter estimation method which may be used to determine
autoregressive (AR) model parameters for 2-D spectral estimation. The
performance of the technique for spectral estimation of 2-D sinusoids in
white noise is demonstrated by numerical example. It is seen that MFE
can provide superior spectral estimation over that which can be achieved
with the multidimensional Levinson algorithm with equivalent
computational burden. The performance of the technique in terms of
computational expense and accuracy of spectral estimation over a number
of simulation trials is compared with a modified covariance technique
We introduce a 3-D discrete transform that depends on the choice
of a certain pair of 3-D arrays called inverse pairs. Many choices of
inverse pairs are possible, and each choice gives a new 3-D transform. A
number of new 2-D and 1-D transforms are also derived
We present a 2-D extension of the minimum free energy (MFE)
autoregressive parameter and spectral estimation technique. MFE can
outperform the multidimensional Levinson algorithm producing superior
spectral estimation of 2-D sinusoids in white Gaussian noise. MFE
spectral estimates are found to be comparable to modified covariance
method estimates. MFE also outperforms the conventional Fourier method
for closely spaced sinusoids at low signal-to-noise ratio and low data
set size
The three main signal processing techniques used to analyze seismic records taken in the exploration for petroleum and natural gas are (1) deconvolution, (2) stacking, and (3) migration. The use of these three methods results in a solution of the geophysical inverse scattering problem. A geophysical survey collects data in the form of reflection seismograms (also called seismic traces) which are received at the surface of the ground due to sources (such as explosions, vibrations, thumping) activated at the surface of the ground. The reflection seismo-grams are recordings of the reflected and other back-scattered energy due to inhomogeneities (such as interfaces between sedimentary rock layers) deep within the earth. The geophysical inverse problem is thus the problem of converting surface data into subsurface information. This subsurface information about the geologic structures can then be used to determine favorable sites for the drilling of wildcat wells. The solution of the geophysical inverse scattering problem depends upon our ability to transform seismic energy recorded at the surface to its pro-per places at depth. The three operations deconvolution, stacking, and migration represent ways of concentrating and moving energy. Deconvolution shifts energy to its proper place with respect to the time axis, whereas stacking and migration shift energy with respect to the spatial axes. Together these methods form an efficient, robust, and stable solution to the geophysical inverse scattering problem.
We present an overview of the regularization of inverse problems from a Bayesian viewpoint. We derive a Bayesian principle for the optimal choice of regularizing parameter. We apply the principle to zeroth order quadratic regularization and extend it to also determine the optimal derivative order for higher order regularization of convolution problems. We also briefly discuss the Generalized Cross-Validation method for choosing the optimal regularization parameters. We present numerical results for a severely ill-posed problem and for the well-posed Fourier problem.
The two-dimensional spectral estimation problem is considered. The rationale for choosing the minimum free energy method to estimate the power spectral density function of a signal is described. The minimization problem required in the minimum free energy method is difficult to analyze directly. Fortunately the free energy functional is convex, thus we can form a dual problem. It turns out to be easier to work with the dual of the minimum free energy problem. Using the extra information provided by the dual problem, we show the minimum free energy problem has a unique minimum for all positive values of the temperature parameter. We also use the dual to produce error bounds on the solution and to show the minimum free energy problem is stable. We discuss various issues which arise when designing a numerical algorithm to solve the dual problem.
The Bayesian derivation of “Classic” MaxEnt image processing (Skilling 1989a) shows that exp(αS(f,m)), where S(f,m) is the entropy of image f relative to model m, is the only consistent prior probability distribution for positive, additive images. In this paper the derivation of “Classic” MaxEnt is completed, showing that it leads to a natural choice for the regularising parameter α, that supersedes the traditional practice of setting x2=N. The new condition is that the dimensionless measure of structure -2αS should be equal to the number of good singular values contained in the data. The performance of this new condition is discussed with reference to image deconvolution, but leads to a reconstruction that is visually disappointing. A deeper hypothesis space is proposed that overcomes these difficulties, by allowing for spatial correlations across the image.
A new method of parametric spectral estimation, which is called minimum-free-energy (MFE) estimation, is introduced. The MFE method produces a generic theoretic estimation model that is particularly relevant to signal-analysis problems that suffer from incomplete and/or noisy data. In the general MFE formulation, the objective function is defined as a linear combination of a mean-square-error-energy expression and a signal entropy expression. This objective function form is analogous to a free-energy function in statistical thermodynamics. The negative coefficient of the entropy term is represented by an effective signal-processing temperature that drives noise-induced fluctuations in the statistical model. The model parameters that characterize the spectrum are determined commensurate with a minimum of the objective function. The mathematical details and solution methods are developed for a specific embodiment of the MFE method, called the MFE-ACS method, in which the error energy is defined as the window-weighted sum of the absolute square of the difference between the initial and final estimated values of the autocorrelation sequence. The order of the autocorrelation sequence used corresponds to the parametric model order for the spectral estimation procedure. Simulations for a variety of narrow-band and broadband test signals and combinations thereof are presented. These simulations are performed for a variety of signal-to-noise-ratio (SNR) scenarios. The MFE algorithms have a broad application domain because they are not restricted to narrow-band sources as are the signal-noise-subspace algorithms. The MFE-ACS algorithm is shown to compare quite favorably with the signal-subspace Tufts-Kumaresan noise-reduced modified-covariance algorithm for closely separated narrow-band sources in the low-SNR regime (~10 dB).
NMR spectroscopy is intrinsically insensitive, a frequently serious limitation especially in biochemical applications where sample size is limited and compounds may be too insoluble or unstable for data to be accumulated over long periods. Fourier transform (FT) NMR was developed by Ernst1 to speed up the accumulation of useful data, dramatically improving the quality of spectra obtained in a given observing time by recording the free induction decay (FID) data directly in time, at the cost of requiring numerical processing. Ernst also proposed that more information could be obtained from the spectrum if the FID was multiplied by a suitable apodizing function before being Fourier transformed. For example (see ref. 2), an increase in sensitivity can result from the use of a matched filter1, whereas an increase in resolution can be achieved by the use of gaussian multiplication1,3, application of sine bells4–8 or convolution difference9. These methods are now used routinely in NMR data processing. The maximum entropy method (MEM)10 is theoretically capable of achieving simultaneous enhancement in both respects11, and this has been borne out in practice in other fields where it has been applied. However, this technique requires relatively heavy computation. We describe here the first practical application of MEM to NMR, and we analyse 13C and 1H NMR spectra of 2-vinyl pyridine. Compared with conventional spectra, MEM gives considerable suppression of noise, accompanied by significant resolution enhancement. Multiplets in the 1H spectra are resolved better leading to improved visual clarity.
In this paper, a direct-data, 2-D spectral estimation technique is presented, in which the 1-D autoregressive properties of the data are exploited independently in both dimensions. For 2-D, multiple complex sinusoids in white noise, a significant improvement in 2-D resolution is obtained.
Low SNR simulations comparing the minimum-free-energy (MFE)
spectral estimation algorithms of Silverstein and Pimbley (1988) with
the reduced rank modified covariance eigenanalysis algorithm of Tufts
and Kumaresan (1982) have been performed. Two different MFE algorithms
are discussed and simulated. The results of the statistical analyses
demonstrate that both MFE algorithms are robust, low-variance spectral
estimators capable of making reliable frequency estimations of closely
spaced sources at very low SNRs, from single snapshot data. They are
applicable to the general domain of spectral estimation, while the
eigenanalysis algorithms are primarily restricted to line spectra
frequency estimation. Nonetheless, for simulations of closely spaced
line spectra at low SNRs, the MFE estimations are considerably more
robust than the Tufts-Kumaresan estimations
This paper gives an exposition of linear prediction in the analysis of discrete signals. The signal is modeled as a linear combination of its past values and present and past values of a hypothetical input to a system whose output is the given signal. In the frequency domain, this is equivalent to modeling the signal spectrum by a pole-zero spectrum. The major part of the paper is devoted to all-pole models. The model parameters are obtained by a least squares analysis in the time domain. Two methods result, depending on whether the signal is assumed to be stationary or nonstationary. The same results are then derived in the frequency domain. The resulting spectral matching formulation allows for the modeling of selected portions of a spectrum, for arbitrary spectral shaping in the frequency domain, and for the modeling of continuous as well as discrete spectra. This also leads to a discussion of the advantages and disadvantages of the least squares error criterion. A spectral interpretation is given t
First Page of the Article
The purpose of this paper is Co present a multidimensional MEM algorithm, valid for non-uniformly sampled arrays, which satisfies a "corrrelation-approximatily" constraint. To this end, the correlation matching.equality constraints of the usual MEM are replaced by a single inequality constraint whose form is based on a measure of the noise in the given acf. In this way, one can incorporate into the model knowledge of the noisy nature of the "given" acf, since the "given" acf is usually estimated from samples of the wavefield. The algorithm has been tested with 1-D synthetic data representing multiple sinusoids buried in additive white noise. The performance of this modified MEM algorithm is compared to a traditional MEM algorithm for extendible acfs and for different SNRs.
First Page of the Article
Simplified linear versions of the nonlinear minimum free energy
(MFE) noise suppression algorithms are introduced. The LMFE algorithms
result from the addition of a smoothness penalty function to the linear
prediction cost function. The autoregressive (AR) parameters are chosen
commensurate with a global minimum of the modified cost function. It is
shown that this constrained optimization procedure reduces to a form of
matrix regularization. The LMFE algorithms are calculable in real time
as they require a negligible increase in complexity over the
conventional autoregressive algorithms, including fast computational
versions thereof. Linear MFE extensions are applicable to all
conventional AR algorithms, and should in each case substantially
increase the useful SNR range of these algorithms. Simulation results
illustrating the single snapshot performance of these algorithms are
given for both narrowband sources and combinations of narrowband and
broadband sources subjected to various levels of Gaussian white noise
It is demonstrated that linear prediction can be successfully
utilized in conjunction with otherwise conventional narrow-band
line-array beamforming for aperture extrapolation (APEX), where there
are significant SNR (signal-to-noise ratio) and resolution advantages
compared to the conventional approach, aperture interpolation, where
faulty sensor data are replaced, and a combined technique (ALPINEX). The
performance of all three methods is investigated in connection with both
simulated and real data. The APEX technique is shown to be amenable to a
simple theoretical treatment. Further, the approach is demonstrated to
offer a practical performance comparable to the well-known
high-resolution technique, MUSIC, when the observation time is short
The maximum entropy method provides an estimate of the power spectral density which maximizes the entropy of a stationary random process from the first N lags of the autocorrelation function. The method is extended to accommodate weighted errors in the measured autocorrelation function.
Maximum-entropy processing is a method for computing the power density spectrum from the first N lags of the autocorrelation function. Unlike the discrete Fourier transform, maximum-entropy processing does not assume that the other lag values are zero. Instead, one mathematically ensures that the fewest possible assumptions about unmeasured data are made by choosing the spectrum that maximizes the entropy for the process. The use of the maximum entropy approach to spectral analysis was introduced by Burg [1]. In this correspondence, the authors derive the maximum-entropy spectrum by obtaining a spectrum that is forced to maximize the entropy of a stationary random process.
Methods of multidimensional power spectral estimation are reviewed. Seven types of estimators are discussed: Fourier, separable, data extension, MLM, MEM, AR, and Pisarenko estimators. Particular emphasis is given to MEM where current research is quite active. Theoretical developments are reviewed and computational algorithms are discussed.
Quadratic regularization
- A Thompson
A. Thompson, " Quadratic regularization, " in Maximum Entropy and Bayesian Methods, J. Skilling, Ed.
Applications of digital signal processing to radar Maximum entropy image restoration in astronomy Maximum entropy signal processing in practical NMR spectroscopy
- J H Mcclellan
- R J Purdy
- R Narayan
- R Nityananda
- S Sibiski
J. H. McClellan and R. J. Purdy, " Applications of digital signal processing to radar, " in Applications of Digital Signal Processing, A. V. Oppenheim, Ed. 131 R. Narayan and R. Nityananda, " Maximum entropy image restoration in astronomy, " Annu. Rev. Astron. Astrophys., vol. 24, pp. 127-170, 1986. 141 S. Sibiski et al., " Maximum entropy signal processing in practical NMR spectroscopy, " Nature, vol. 311, pp. 446-447, Oct. 1984.
Application of a maximum entropy frequency analysis to synthetic aperture radar
- P L Jackson
- L S Joyce
- G B Feldkamp
P. L. Jackson, L. S. Joyce, and G. B. Feldkamp, " Application of a maximum entropy frequency analysis to synthetic aperture radar, " in Proc. RADC Spectrum Estimation Workshop (Rome, NY), May 1978.
Line array beamfonning using linear prediction for aperture interpolation and extrapolation New results on data adaptive high resolution spectral analysis techniques
- L S Joycesi
- D N Swingler
- R S Walker
L. S. Joyce, " A separable 2-D autoregressive-spectral estimation algo-rithm, " in Proc. Int. Conf. Acoust., Speech, Signal Processing (Washing-ton, DC), Apr. 1979. [SI D. N. Swingler and R. S. Walker, " Line array beamfonning using linear prediction for aperture interpolation and extrapolation, " IEEE Trans. Acoust., Speech, Signal Processing, vol. 37, no. I, pp. 16-30, Jan. 1989. 191 C. H. Chen, A. H. Costa, and C. Sue, " New results on data adaptive high resolution spectral analysis techniques, " in Proc. 1988 IEEE Int. Conf. Syst., Man, Cyhern. (Shenyang, China), Aug. 1988.
Haykin Advances in Spectral Analysis and Array Processing
- S D Silverstein
- J M Pimbley