
Amin ShokrollahiSwiss Federal Institute of Technology in Lausanne | EPFL · Mathematics Section
Amin Shokrollahi
PhD Computer Science, University of Bonn
About
219
Publications
42,196
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
17,306
Citations
Introduction
Additional affiliations
October 2000 - February 2009
Digital Fountain, Inc
Position
- Principal Investigator
July 1998 - October 2000
September 1997 - February 1998
Publications
Publications (219)
Employing advanced communication and circuit techniques to augment data -transfer bandwidth. This article presents a class of signaling methods that improves the data rate of communication over copper channels without compromising its sensitivity to inter symbol interference (151). Chord signaling (CS) [1], [2] puts correlated data over multiple wi...
An energy-efficient (1.02 pJ/b) and high-speed (20.83 Gb/s/wire, 417 Gb/s/mm) link for ultra-short reach (USR) applications (up to 6-dB channel loss at the Nyquist frequency of 12.5 GHz) is presented. Correlated non-return to zero (CNRZ) signaling with low sensitivity to inter-symbol interference (ISI) has been developed to improve the link budget....
An online fountain code is defined as a fountain code for which an optimal encoding strategy can be found efficiently given any instantaneous decoding state. This property is important for data distribution in practical networks. In this paper, we formalize the problem of online fountain code construction, and propose new online fountain codes that...
In bus communications methods and apparatus, a first set of physical signals representing the information to be conveyed over the bus is provided, and mapped to a codeword of a spherical code, wherein a codeword is representable as a vector of a plurality of components and the bus uses at least as many signal lines as components of the vector that...
A communications system includes an encoder that produces a plurality of redundant symbols. For a given key, an output symbol is generated from a combined set of symbols including the input symbols and the redundant symbols. The output symbols are generally independent of each other, and an effectively unbounded number of output symbols (subject to...
A method is disclosed for storing information on non-volatile memory which can rewrite memory cells multiple times before a block needs to be erased. The information to be stored is transformed into a suitable form which has better robustness properties with respect to common sources of error, such as leakage of charge, or imperfect read/write unit...
The task of a neural associative memory is to retrieve a set of previously
memorized patterns from their noisy versions using a network of neurons. An
ideal network should have the ability to 1) learn a set of patterns as they
arrive, 2) retrieve the correct patterns from noisy queries, and 3) maximize
the pattern retrieval capacity while maintaini...
Binary codes over 2D arrays are very useful in data storage, where each array column represents a storage device or unit that may suffer failure. In this paper, we propose a new framework for probabilistic construction of codes on 2D arrays. Instead of a pure combinatorial erasure model used in traditional array codes, we propose a mixed combinator...
Recent advances in associative memory design through structured pattern sets and graph-based inference algorithms have allowed reliable learning and recall of an exponential number of patterns that satisfy certain subspace constraints. Although these designs correct external errors in recall, they assume neurons that compute noiselessly, in contras...
The continuing demand for higher bandwidth in serial interconnects has pushed the symbol rate of differential lanes into the high-insertion-loss region of channels. Multi-level signaling such as differential PAM-4 [1] has been used to mitigate the loss of electrical channels by lowering the signal spectrum. Such an approach suffers from lower SNR t...
Recent advances in associative memory design through structured pattern sets and graph-based inference algorithms allow reliable learning and recall of exponential numbers of patterns. Though these designs correct external errors in recall, they assume neurons compute noiselessly, in contrast to highly variable neurons in hippocampus and olfactory...
We consider the problem of neural association for a network of non-binary
neurons. Here, the task is to first memorize a set of patterns using a network
of neurons whose states assume values from a finite number of integer levels.
Later, the same network should be able to recall previously memorized patterns
from their noisy versions. Prior work in...
The task of a neural associative memory is to retrieve a set of previously memorized pat-terns from their noisy versions by using a net-work of neurons. Hence, an ideal network should be able to 1) gradually learn a set of patterns, 2) retrieve the correct pattern from noisy queries and 3) maximize the number of memorized patterns while maintaining...
In this paper we study a particular class of generalized Reed-Solomon codes and introduce encoding and decoding algorithms for such codes that speed up current hardware implementations by a factor p wherein p can be any divisor of the size of the multiplicative group of the underlying field. In many cases, p can be chosen to be 3, for example. In s...
Recent advances in associative memory design through strutured pattern sets
and graph-based inference algorithms have allowed the reliable learning and
retrieval of an exponential number of patterns. Both these and classical
associative memories, however, have assumed internally noiseless computational
nodes. This paper considers the setting when i...
We propose a novel architecture to design a neural associative memory that is
capable of learning a large number of patterns and recalling them later in
presence of noise. It is based on dividing the neurons into local clusters and
parallel plains, very similar to the architecture of the visual cortex of
macaque brain. The common features of our pr...
We consider irregular product codes.In this class of codes, each codeword is
represented by a matrix. The entries in each row (column) of the matrix should
come from a component row (column) code. As opposed to (standard) product
codes, we do not require that all component row codes nor all component column
codes be the same. As we will see, relaxi...
We consider the problem of neural association for a network of non-binary neurons. Here, the task is to recall a previously memorized pattern from its noisy version using a network of neurons whose states assume values from a finite number of non-negative integer levels. Prior works in this area consider storing a finite number of purely random pat...
A fountain coding framework is proposed that endows receivers with the ability to monitor and control the decoding progress given the instantaneous network conditions. These online features allow an optimal recovery from losses manifested by adversarial or other not purely random processes. A uni-partite graph structure and accompanying algorithms...
We consider the problem of neural association, which deals with the retrieval of a previously memorized pattern from its noisy version. The performance of various neural networks developed for this task may be judged in terms of their pattern retrieval capacities (the number of patterns that can be stored), and their error-correction (noise toleran...
This document describes a Fully-Specified Forward Error Correction
(FEC) scheme, corresponding to FEC Encoding ID 6, for the RaptorQ FEC
code and its application to reliable delivery of data objects.
RaptorQ codes are a new family of codes that provide superior
flexibility, support for larger source block sizes, and better coding
efficiency than R...
Suppose that we have two users each of which has a k-dimensional vector over a field FQ. Their goal is to communicate their vectors to a common receiver. At the time of reception, the receiver is given side information consisting of some of the entries of the first vector, some entries of the second vector, and the knowledge that some other entries...
Probabilistic construction of codes on two- dimensional arrays is proposed and analyzed. Instead of a pure combinatorial erasure model used in traditional array codes, we propose a mixed combinatorial-probabilistic model of limiting the number of column failures, with assuming a binary erasure channel in each failing column. In addition, motivated...
The last two decades have witnessed a full revival of graph based codes. The advent of Turbo codes [1] in the early 1990’s,
the revival of Gallager’s LDPC codes in the 1990’s [12,11,7,8], and a decade long research on their properties [16] have brought
fundamental changes to coding theory in general, and to the practical design of codes in particul...
This monograph describes the theory behind Raptor codes, and elucidates elements of the processes behind the design of two of the most prominent members of this class of codes: R10 and RaptorQ (RQ). R10 has already been adopted by a number of standards’ bodies, and RQ is in the process of entering various standards at the time of writing of this mo...
We consider ensembles of binary linear error correcting codes, obtained by sampling each column of the generator matrix G or parity check matrix H independently from the set of all binary vectors of weight d (of appropriate dimension). We investigate the circumstances under which the mutual information between a randomly chosen codeword and the vec...
We consider the problem of multicasting data from a source to receivers that possess arbitrary subsets of the data apriori as side information. Fountain codes, which are an ideal solution to the standard multicasting problem without any side information, have also been proposed as a potential approach for the side information problem in multiple in...
A wiretap protocol is a pair of randomized encoding and decoding functions such that knowledge of a bounded fraction of the encoding of a message reveals essentially no information about the message, while knowledge of the entire encoding reveals the message using the decoder. In this paper we study the notion of efficiently invertible extractors a...
We analyze the second moment of the ripple size during the LT decoding process and prove that the standard deviation of the ripple size for an LT-code with length k is of the order of radick. Together with a result by Karp et. al (2004) stating that the expectation of the ripple size is of the order of k, this gives bounds on the error probability...
In this paper, we introduce a class of decoding algorithms for binary Raptor codes used for transmission over q-ary channels, where q = 2<sup>m</sup>. The algorithms provide a tradeoff between complexity and decoding capability. Whereas the running time of the q-ary belief-propagation algorithms is m2<sup>m</sup> times that of its binary counterpar...
This paper studies the stability of some reconstruction algorithms for compressed sensing in terms of the bit precision. Considering the fact that practical digital systems deal with discretized signals, we motivate the importance of the total number of accurate bits needed from the measurement outcomes in addition to the number of measurements. It...
We present some simple information inequalities on binary sums of Bernoulli random variables that appear to be new. Consequences for information across binary input memoryless symmetric channels are also presented.
We consider the problem of uniform sampling of points on an algebraic variety. Specifically, we develop a randomized algorithm that, given a small set of multivariate polynomials over a sufficiently large finite field, produces a common zero of the polynomials almost uniformly at random. The statistical distance between the output distribution of t...
While linear programming (LP) decoding provides more flexibility for finite-length performance analysis than iterative message-passing (IMP) decoding, it is computationally more complex to implement in its original form, due to both the large size of the relaxed LP problem, and the inefficiency of using general-purpose LP solvers. This paper explor...
Digital media have become an integral part of modern lives. Whether surfing the web, making a wireless phone call, watching satellite TV, or listening to digital music, a large part of our professional and leisure time is filled with all things digital. The replacement of analog media by their digital counterparts and the explosion of Internet use...
This monograph describes the theory behind Raptor codes, and elucidates elements of the processes behind the design of two of the most prominent members of this class of codes: R10 and RaptorQ (RQ). R10 has already been adopted by a number of standards’ bodies, and RQ
is in the process of entering various standards at the time of writing of
this mo...
This paper explores ideas for fast linear programming (LP) decoding of low-density parity-check (LDPC) codes. We first propose a modification of adaptive LP decoding, and prove that it performs LP decoding by solving a number of linear programs that contain at most one linear constraint derived from each of the parity-check constraints. Then, as a...
In this paper, two fixed per-information symbol complexity lossless source coding algorithms are modified for estimation and incremental LT decoding over piecewise stationary memoryless channels (PSMC's) with a bounded number of abrupt changes in channel statistics. In particular, as a class of PSMC's, binary symmetric channels are considered with...
Summary form only given. This talk describes the development of various versions of Raptor codes starting from the first one developed at the end of 2000 to the more advanced versions that are being used in many commercial systems today. We will focus on practical aspects and how these have pushed the theoretical frontiers. The talk will conclude w...
In the Information Embedding Problem one is given a piece of data which can be altered only conditionally, for example only at certain places. One is then asked to embed an arbitrary message into the data by only applying admissible changes to the data. These changes lead to a distortion which is to be kept low. In this short note, we introduce an...
Application Layer Forward Error Correction (AL-FEC) is an innovative way to provide reliability in mobile broadcast systems. Conventional data such as multimedia files or multimedia streams are extended with repair information which can be used to recover lost data at the receiver. AL-FEC is integrated into content delivery protocols (CDPs) to supp...
This document describes a Fully-Specified Forward Error Correction
(FEC) scheme, corresponding to FEC Encoding ID 1, for the Raptor
forward error correction code and its application to reliable
delivery of data objects.
Raptor is a fountain code, i.e., as many encoding symbols as needed
can be generated by the encoder on-the-fly from the source sy...
In this paper, I will give a brief introduction to the theory of low-density parity-check codes, and their decoding. I will
emphasize the case of correcting erasures as it is still the best understood and most accessible case. At the end of the paper,
I will also describe more recent developments.
A Fountain code is a code of fixed dimension and a limitless block-length. This is a class of codes with many interesting properties and applications. In this talk I will introduce several classes of probabilistic Fountain codes, including LT-and Raptor codes, show tools for their design and analysis, and discuss how they are used today to solve va...
In this paper we propose a notion of EXIT functions for LT-codes and raptor codes, and discuss some of their useful properties that can be used to design good codes. We will further use these results to derive interesting conclusions on ranks of random matrices. New expressions for the asymptotic ranks of certain classes of random binary matrices a...
In this paper we propose a new structure for multiplication using optimal normal bases of type 2. The multiplier uses an efficient linear transformation to convert the normal basis representations of elements of \(\mathbb{F}_{q^{n}}\) to suitable polynomials of degree at most n over \(\mathbb{F}_{q}\). These polynomials are multiplied using any met...
We present a structural attack against the Sidelnikov cryptosystem [V. M. Sidelnikov, “A public-key cryptosystem based on binary Reed-Muller codes”, Discrete Math. Appl. 4, No. 3, 191–207 (1994; Zbl 0872.94040)]. The attack creates a private key from a given public key. Its running time is subexponential and is effective if the parameters of the Re...
The workshop on Coding Theory has brought together leading researchers in several key areas of mathematical coding theory. On the side of many mathematicians there were computer scientist and electrical engineers present. Participants came from many countries and the group included both senior and junior researchers.
Ever since its conception in th...
In this paper we will investigate the performance of Raptor codes using Gallager's majority decoding algorithm on the binary symmetric channels. We obtain equations which relate the error probability to the outputnode degree distribution and then we design good degree distributions using the differential evolution (DE) method
LT-codes are a new class of codes introduced by Luby for the purpose of scalable and fault-tolerant distribution of data over computer networks. In this paper, we introduce Raptor codes, an extension of LT-codes with linear time encoding and decoding. We will exhibit a class of universal Raptor codes: for a given integer k and any real epsiv>0, Rap...
In this paper, we will investigate the performance of Raptor codes on arbitrary binary input memoryless symmetric channels (BIMSCs). In doing so, we generalize some of the results that were proved before for the erasure channel. We will generalize the stability condition to the class of Raptor codes. This generalization gives a lower bound on the f...
We introduce "derandomized" versions of the tensor product and the zig-zag product, extending the ideas in the derandomized squaring operation of Rozenman and Vadhan. These enable us to obtain graphs with smaller degrees than those obtained using their non-derandomized counterparts, though at the cost of slightly worse expansion. In this paper we g...
In this paper we describe some practical aspects of the design process of good Raptor codes for finite block lengths over arbitrary binary input symmetric channels. In particular we introduce a simple model for the finite-length convergence behavior of the iterative decoding algorithm based on density evolution, and propose a practical design proce...
We present a new model for LT codes which simplifies the analysis of the error probability of decoding by belief propagation. For any given degree distribution, we provide the first rigorous expression for the limiting error probability as the length of the code goes to infinity via recent results in random hypergraphs [Darling-Norris 2005]. For a...
Summary form only given. This paper presents an algorithm in a purely lossless text compression setting based on fountain codes and the Burrows-Wheeler transform (BWT). The scheme consists of five stages, each of which is briefly described in the paper. The algorithm offers encouraging compression rate performance for large files. A summary of the...
In this paper, we describe one solution to the two-user Slepian-Wolf problem in a certain part of the achievable region using fountain codes. Symmetric case of memoryless compression of two correlated sources is considered and modeled by a BSC channel. The compression is done by two separate compressors without any exchange of information between t...
We outline a procedure for using pseudorandom generators to construct binary codes with good properties, assuming the existence of sufficiently hard functions. Specifically, we give a polynomial time algorithm, which for every integers $n$ and $k$, constructs polynomially many linear codes of block length $n$ and dimension $k$, most of which achiev...
We analyze a generalization of a recent algorithm of Bleichenbacher et al. for decoding interleaved codes on the Q-ary symmetric channel for large Q. We will show that for any m and any ε the new algorithms can decode up to a fraction of at least \(\frac{\beta m}{\beta m+1}(1-R-2Q^{-1/2m}) - \epsilon\) errors, where \(\beta = \frac{ln(q^m - 1)}{ln(...
In this paper we extend the double verification algorithm of Luby and Mitzenmacher to the class of Raptor codes, analyze it, and design Raptor codes that perform very well with respect to this algorithm
We consider a simple network, where a source and destination node are connected with a line of erasure channels. It is well known that in order to achieve the min-cut capacity, the intermediate nodes are required to process the information. We propose coding schemes for this setting, and discuss each scheme in terms of complexity, delay, achievable...
We show, using a probabilistic argument, that for any y with 0 < y < 1 there exist repeat-accumulate codes that approach the asymptotic Gilbert-Varshamov bound, and that are encodable in time O(n <sup>1+y</sup>)
We consider a simple network, where a source and destination node are connected with a line of erasure channels. It is well known that in order to achieve the min-cut capacity, the intermediate nodes are required to process the information. We propose coding schemes for this setting, and discuss each scheme in terms of complexity, delay, achievable...
We prove that for any given R between 0 and 1 the best threshold value for a regular LDPC code of rate R with common variable degree v and common check degree c occurs when v is at least 3 and is minimal subject to the condition R=1-v/c.
The design of practical and highly powerful codes for protection against erasures in digital communication can be reduced to optimizing solutions of a highly nonlinear constraint satisfaction problem (Luby et al. 1997). In this section we will attack this problem using the differential evolution approach (Storn and Price 1997) and significantly imp...
Fountain codes are a new class of codes with finite dimension and infinite block-length. An appropriately designed Fountain code eliminates the need for a sender to get a guess of the quality of the channel to a particular receiver, and enables the design of scalable transmission of data in a broadcast or multicast setting. In this paper we discuss...
We give a short survey of several techniques to construct codes on GF(q) that approach the capacity of the q-ary symmetric channel. The q-ary symmetric channel represents the next level of difficulty after the binary erasure channel (BEC). Since the channel is more complex than the BEC, one may hope that codes and decoding algorithms that approach...
This paper proposes a universal variable-length lossless compression algorithm based on fountain codes. The compressor concatenates the Burrows-Wheeler block sorting transform (BWT) with a fountain encoder, together with the closed-loop iterative doping algorithm. The decompressor uses a belief propagation algorithm in conjunction with the iterativ...
We show that the error probability for decoding interleaved Reed-Solomon Codes with the decoder found by Bleichenbacher et al. (Ref.1) is upper bounded by O(1/q), independently of n. The decoding algorithm presented here is similar to that of standard RS codes. It involves computing the error-locator polynomial. These polynomials are found by compu...
LDPC codes are one of the hottest topics in coding theory today. Originally invented in the early 1960’s, they have experienced
an amazing comeback in the last few years. Unlike many other classes of codes, LDPC codes are already equipped with very fast
(probabilistic) encoding and decoding algorithms. The question is that of the design of the code...
We consider a multicast configuration with two sources, and translate the network code design prob-lem to vertex coloring of an appropriately defined graph. This observation enables to derive code design algorithms and alphabet size bounds, as well as establish a connection with a number of well-known results from discrete mathe-matics that increas...
Transmission of packets over computer networks is subject to packet-level errors, which appear as "bursts" of bit-level errors and are not well modeled by memoryless binary channels. A standard scrambling technique is used for transmission of packets by the q-ary symmetric channel (q-SC) with alphabet size q and error probability p. Furthermore, si...
This paper investigates the use of algebraic-geometric codes for data transmission over a packet network, by comparing their encoding/decoding speeds to those of the ubiquitous Reed-Solomon Codes. We take advantage of the fact that AG codes allow the construction of longer codes over a given alphabet, which in turn means we can create an [n, k]-cod...
This paper provides an efficient method for analyzing the error probability of the belief propagation (BP) decoder applied to LT Codes. Each output symbol is generated independently by sampling from a distribution and adding the input symbols corresponding to the support of the sampled vector.
This paper extends the construction and analysis of Raptor codes originally designed in A. Shokrollahi (2004) for the erasure channel to general symmetric channels. We explicitly calculate the asymptotic fraction of output nodes of degree one and two for capacity-achieving Raptor codes, and discuss techniques to optimize the output degree distribut...
About two thirds or 480 pages of Knuth’ [Knuth 1998] volume on Seminumerical algorithms are devoted to Chapter 4, entitled Arithmetic. He states Research on seminumerical algorithms continues at a phenomenal rate. They are called seminumerical because they lie on the borderline between numeric and symbolic calculation. Each algorithm not only compu...
We describe an authentication scheme whose security is based on the hardness of finding roots of systems of sparse polynomial equations in many variables and of high degree. One of the new ideas is the use of many keys. In one authentication session, a small amount of information about only one of them, chosen randomly, is released; this may be use...
Multiple antennas can greatly increase the data rate and
reliability of a wireless communication link in a fading environment.
Their success, however, depends on the design of cedes that achieve
these promises. It is well known that unitary matrices can be used to
design differentially modulated space-time codes. These codes have a
particularly eff...
In this paper we will survey some of the most recent results on low-density parity-check codes. Our emphasis will be primarily
on the asymptotic theory of these codes. For the most part, we will introduce the main concepts for the easier case of the
erasure channel. We will also give an application of these methods to reliable content delivery.
We investigate the average erasure probability of the belief propagation algorithm over the binary erasure channel (BEC) for various finite-length low-density parity-check (LDPC) ensembles. In particular, we give tight upper bounds on the "error floor", i.e., on the contribution to the erasure probability stemming from relatively small deficiencies...
It is well-known that multiple transmit and receiving antennas can significantly improve the performance of wireless networks.
The design of good modulation schemes for the model of multiple antenna wireless transmission in a fast fading environment
(e.g., mobile communication) leads to an interesting packing problem for unitary matrices. Surprisin...
Multiple antennas can greatly increase the data rate and
reliability of a wireless communication link in a fading environment,
but the practical success of using multiple antennas depends crucially
on our ability to design high-rate space-time constellations with low
encoding and decoding complexity. It has been shown that full
transmitter diversit...
Multiple antennas can greatly increase the data rate and reliability of a wireless communication link in a fading environment, but the practical success of using multiple antennas depends crucially on our ability to design high-rate space-time constellations with low encoding and decoding complexity. It has been shown that full transmitter diversit...
We design low-density parity-check (LDPC) codes that perform at
rates extremely close to the Shannon capacity. The codes are built from
highly irregular bipartite graphs with carefully chosen degree patterns
on both sides. Our theoretical analysis of the codes is based on the
work of Richardson and Urbanke (see ibid., vol.47, no.2, p.599-618,
2000)...
We introduce a simple erasure recovery algorithm for codes derived
from cascades of sparse bipartite graphs and analyze the algorithm by
analyzing a corresponding discrete-time random process. As a result, we
obtain a simple criterion involving the fractions of nodes of different
degrees on both sides of the graph which is necessary and sufficient...
. The polynomial time algorithm of Lenstra, Lenstra, and Lov'asz [17] for
Using methods originating in numerical analysis, we will develop a unified framework for derivation of efficient algorithms for decoding several classes of algebraic codes. We will demonstrate our method by accelerating Sudan's list decoding algorithm for Reed-Solomon codes [22], its generalization to algebraic-geometric codes by Shokrollahi and Wa...