Page 1

Motivation Almost Lossless Analog CompressionRényi’s Information Dimension TheoremsConclusion

Fundamental Limits of

Almost Lossless Analog Compression

Yihong Wu and Sergio Verdú

Department of Electrical Engineering

Princeton University

June 29, 2009

Y. Wu and S. Verdú Almost Lossless Analog Compression1

Page 2

MotivationAlmost Lossless Analog CompressionRényi’s Information DimensionTheorems Conclusion

Dimension compression rate: discrete sources

Let X be a random variable on a discrete alphabet A.

Let Xn= [X1,...,Xn]Tbe i.i.d. with Xi∼ X.

An (n,k)-code:

Encoder f : An→ Ak.

Decoder g : Ak→ An.

Block error probability: ?n= P{g(f(Xn)) ?= Xn}.

Dimension compression rate: R =k

n.

Fundamental limit:

H(X)

log|A|.

Y. Wu and S. VerdúAlmost Lossless Analog Compression2

Page 3

Motivation Almost Lossless Analog Compression Rényi’s Information Dimension TheoremsConclusion

As |A| → ∞

Question

What is the fundamental limit when the alphabet becomes

continuum,

H(X)

log|A|

Y. Wu and S. Verdú Almost Lossless Analog Compression3

Page 4

Motivation Almost Lossless Analog Compression Rényi’s Information DimensionTheoremsConclusion

As |A| → ∞

Question

What is the fundamental limit when the alphabet becomes

continuum,

H(X) ? ∞

log|A| ? ∞

Y. Wu and S. Verdú Almost Lossless Analog Compression3

Page 5

MotivationAlmost Lossless Analog CompressionRényi’s Information DimensionTheorems Conclusion

Dimension compression rate: analog sources

Let X be a random variable on R.

Let Xn= [X1,...,Xn]Tbe i.i.d. with Xi∼ X.

An (n,k)-code:

Encoder f : Rn→ Rk.

Decoder g : Rk→ Rn.

Block error probability: ?n= P{g(f(Xn)) ?= Xn}.

Dimension compression rate: R =k

Fundamental limit:

n.

Y. Wu and S. VerdúAlmost Lossless Analog Compression4