Rateless codes on noisy channels

Abstract

This paper studies the performance of two classes of rateless codes (LTand Raptor codes) on noisy channels such as the BSC and the AWGNC. We find that Raptor codes outperform LT codes, and have good performance on a wide variety of noisy channels.

Full text

Page 1

MITSUBISHI ELECTRIC RESEARCH LABORATORIES
http://www.merl.com
Rateless Codes on Noisy Channels
Ravi Palanki and Jonathan S. Yedidia
TR-2003-124April 2004
Abstract
We study the performance of the newly invented rateless codes (LT- and Raptor codes)
on noisy channels such as the BSC and the AWGN channel. We find that Raptor codes
outperform LT codes, and have good performance on a wide variety of noisy channels.
This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in
whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such
whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research
Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions
of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment
of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved.
Copyright c ? Mitsubishi Electric Research Laboratories, Inc., 2004
201 Broadway, Cambridge, Massachusetts 02139

Page 2

1. First printing, October 2003. 2. Minor corrections, November 2003. 3. More minor corrections,
April 2004. 4. See also MERL TR2004-037 for a shorter version of this technical report published
in the Proceedings of the Conference on Information Sciences and Systems, 2004.

Page 3

Rateless codes on noisy channels
Ravi Palanki
Department of Electrical Engineering
California Institute of Technology
Pasadena, CA 91125, USA
email: ravi@systems.caltech.edu
Jonathan S. Yedidia
Mitsubishi Electric Research Laboratories
Cambridge, MA 02139, USA
email: yedidia@merl.com
Abstract
We study the performance of the newly invented rateless codes
(LT and Raptor codes) on noisy channels such as the BSC and the
AWGNC. We find that Raptor codes outperform LT codes, and have
good performance on a wide variety of noisy channels.
1 Introduction
Recent advances in coding theory, especially the invention of regular [1] and
irregular [2] low density parity check (LDPC) codes, have shown that very
efficient error correction schemes are possible. LDPC codes, decoded using
the belief propagation algorithm, can achieve capacity on the binary era-
sure channel (BEC) [2, 3] and achieve rates very close to capacity on other
channels such as the binary symmetric channel (BSC) and the additive white
Gaussian noise channel (AWGNC)[4]. Because of this, one could say that the
problem of reliable communication over many practical channels has been
solved. However, such a statement comes with a caveat: both the transmit-
ter and the receiver must know the exact channel statistics a-priori. While
this assumption is valid in many important cases, it is clearly not true in
many other equally important cases. For example, on the internet (which is
1

Page 4

modeled as a BEC), the probability p that a given packet is dropped varies
with time, depending on traffic conditions in the network. A code designed
for a good channel (low p) would result in decoding failure when used over a
bad channel (high p). Conversely, a code designed for a bad channel would
result in unnecessary packet transmissions when used over a good channel.
We can solve this problem by using rateless codes. Instead of encoding
the k information bits to a pre-determined number of bits using a block code,
the transmitter encodes them into a potentially infinite stream of bits and
then starts transmitting them. Once the receiver gets a sufficient number of
symbols from the output stream, it decodes the original k bits. The number
of symbols required for successful decoding depends on the quality of the
channel. If decoding fails, the receiver can pick up a few more output symbols
and attempt decoding again. This process can be repeated until successful
decoding. The receiver can then tell the transmitter over a feedback channel
to stop any further transmission.
The use of such an “incremental redundancy” scheme is not new to coding
theorists. In 1974, Mandelbaum [5] proposed puncturing a low rate block
code to build such a system. First the information bits are encoded using a
low rate block code. The resulting codeword is then punctured suitably and
transmitted over the channel. At the receiver the punctured bits are treated
as erasures. If the receiver fails to decode using just the received bits, then
some of the punctured bits are transmitted. This process is repeated till
every bit of the low rate codeword has been transmitted. If the decoder still
fails, the transmitter begins to retransmit bits till successful decoding. It is
easy to see such a system is indeed a rateless code, since the encoder ends
up transmitting a different number of bits depending on the quality of the
channel. Moreover, if the block code is a random (or random linear) block
code, then the rateless code approaches the Shannon limit on every binary
input symmetric channel (BISC) as the rate of the block code approaches
zero. Thus such a scheme is optimal in the information theoretic sense.
Unfortunately, it does not work as well with practical codes. Mandelbaum
originally used RS codes for this purpose and other authors have investigated
the use of punctured low rate convolutional [6] and turbo [7] codes. In addi-
tion to many code-dependent problems, all these schemes share a few common
problems. Firstly, the performance of the rateless code is highly sensitive to
the performance of the low rate block code i.e., a slightly sub-optimal block
code can result in a highly sub-optimal rateless code. Secondly, the rateless
code has very high decoding complexity, even on a good channel. This is
2

Page 5

because on any channel, the decoder is decoding the same low-rate code, but
with varying channel information. The complexity of such a decoding grows
at least as O(k/R) where R is the rate of the low rate code.
In a recent landmark paper, Luby [8] circumvented these problems by de-
signing rateless codes which are not obtained by puncturing standard block
codes. These codes, known as Luby Transform (LT) codes, are low density
generator matrix codes which are decoded using the same message passing
decoding algorithm (belief propagation) that is used to decode LDPC codes.
Also, just like LDPC codes, LT codes achieve capacity on every BEC. Un-
fortunately, LT codes also share the error floor problem endemic to capacity
achieving LDPC codes. Shokrollahi [9, 10] showed that this problem can be
solved using raptor codes, which are LT codes combined with outer LDPC
codes. These codes have no noticeable error floors on the BEC though their
rate is slightly bounded away from capacity.
The aim of this paper is to study the performance of LT and raptor codes
on channels other than the BEC. Since LDPC codes designed for the BEC
perform fairly well on other channels, one might conjecture that such a result
holds for LT codes as well. We test this conjecture using simulation studies
and density evolution [11].
2 LT codes
The operation of an LT encoder is very easy to describe. From k given infor-
mation bits, it generates an infinite stream of encoded bits, with each such
encoded bit generated as follows:
1. Pick a degree d at random according to a distribution µ(d).
2. Choose uniformly at random d distinct input bits.
3. The encoded bit’s value is the XOR-sum of these d bit values.
The encoded bit is then transmitted over a noisy channel, and the decoder
receives a corrupted version of this bit. Here we make the non-trivial assump-
tion that the encoder and decoder are completely synchronized and share a
common random number generator i.e., the decoder knows which d bits are
used to generate any given encoded bit, but not their values. On the inter-
net, this sort of synchronization is easily achieved because every packet has
an uncorrupted packet number. More complicated schemes are required on
other channels; here we shall just assume that some such scheme exists and
works perfectly in the system we’re studying. In other words, the decoder
3