Article

# Compressive Sampling and Lossy Compression

IEEE Signal Processing Magazine (Impact Factor: 3.37). 04/2008; DOI:10.1109/MSP.2007.915001
Source: IEEE Xplore

ABSTRACT Recent results in compressive sampling have shown that sparse signals can be recovered from a small number of random measurements. This property raises the question of whether random measurements can provide an efficient representation of sparse signals in an information-theoretic sense. Through both theoretical and experimental results, we show that encoding a sparse signal through simple scalar quantization of random measurements incurs a significant penalty relative to direct or adaptive encoding of the sparse signal. Information theory provides alternative quantization strategies, but they come at the cost of much greater estimation complexity.

0 0
·
1 Bookmark
·
91 Views
• Source
##### Article: Data compression and harmonic analysis
[hide abstract]
ABSTRACT: In this paper we review some recent interactions between harmonic analysis and data compression. The story goes back of course to Shannon's R(D) theory in the case of Gaussian stationary processes, which says that transforming into a Fourier basis followed by block coding gives an optimal lossy compression technique; practical developments like transform-based image compression have been inspired by this result. In this paper we also discuss connections perhaps less familiar to the information theory community, growing out of the field of harmonic analysis. Recent harmonic analysis constructions, such as wavelet transforms and Gabor transforms, are essentially optimal transforms for transform coding in certain settings. Some of these transforms are under consideration for future compression standards. We discuss some of the lessons of harmonic analysis in this century. Typically, the problems and achievements of this field have involved goals that were not obviously related to practical data compression, and have used a language not immediately accessible to outsiders. Nevertheless, through an extensive generalization of what Shannon called the “sampling theorem”, harmonic analysis has succeeded in developing new forms of functional representation which turn out to have significant data compression interpretations. We explain why harmonic analysis has interacted with data compression, and we describe some interesting recent ideas in the field that may affect data compression in the future
IEEE Transactions on Information Theory 11/1998; · 2.62 Impact Factor
• Source
##### Article: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information
[hide abstract]
ABSTRACT: This paper considers the model problem of reconstructing an object from incomplete frequency samples. Consider a discrete-time signal f∈C<sup>N</sup> and a randomly chosen set of frequencies Ω. Is it possible to reconstruct f from the partial knowledge of its Fourier coefficients on the set Ω? A typical result of this paper is as follows. Suppose that f is a superposition of |T| spikes f(t)=σ<sub>τ∈T</sub>f(τ)δ(t-τ) obeying |T|≤C<sub>M</sub>·(log N)<sup>-1</sup> · |Ω| for some constant C<sub>M</sub>>0. We do not know the locations of the spikes nor their amplitudes. Then with probability at least 1-O(N<sup>-M</sup>), f can be reconstructed exactly as the solution to the ℓ<sub>1</sub> minimization problem. In short, exact recovery may be obtained by solving a convex optimization problem. We give numerical values for C<sub>M</sub> which depend on the desired probability of success. Our result may be interpreted as a novel kind of nonlinear sampling theorem. In effect, it says that any signal made out of |T| spikes may be recovered by convex programming from almost every set of frequencies of size O(|T|·logN). Moreover, this is nearly optimal in the sense that any method succeeding with probability 1-O(N<sup>-M</sup>) would in general require a number of frequency samples at least proportional to |T|·logN. The methodology extends to a variety of other situations and higher dimensions. For example, we show how one can reconstruct a piecewise constant (one- or two-dimensional) object from incomplete frequency samples - provided that the number of jumps (discontinuities) obeys the condition above - by minimizing other convex functionals such as the total variation of f.
IEEE Transactions on Information Theory 03/2006; · 2.62 Impact Factor
• Source
##### Article: Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies?
[hide abstract]
ABSTRACT: Suppose we are given a vector $f$ in $\R^N$. How many linear measurements do we need to make about $f$ to be able to recover $f$ to within precision $\epsilon$ in the Euclidean ($\ell_2$) metric? Or more exactly, suppose we are interested in a class ${\cal F}$ of such objects--discrete digital signals, images, etc; how many linear measurements do we need to recover objects from this class to within accuracy $\epsilon$? This paper shows that if the objects of interest are sparse or compressible in the sense that the reordered entries of a signal $f \in {\cal F}$ decay like a power-law (or if the coefficient sequence of $f$ in a fixed basis decays like a power-law), then it is possible to reconstruct $f$ to within very high accuracy from a small number of random measurements.
IEEE Transactions on Information Theory. 01/2006; 52:5406-5425.