Richard Blahut

• University of Illinois Urbana-Champaign

We derive the Schulz-Snyder phase retrieval algorithm as an alternating minimization method, and discuss its advantages and drawbacks. An annealing-type Schulz-Snyder algorithm is proposed to avoid convergence to nonglobal solutions.

The Körner-Marton modulo two-sum problem is described as a variation of the lossless two-help-one source coding problem.

The history of the cutoff rate is reviewed. It is argued that the cutoff rate is a fundamental constant associated with each discrete memoryless channel. It is the largest rate for which maximum-likelihood (or minimum-distance) decoding is tractable for that channel. This statement is the appropriate backdrop against which to see recent progress in...

In Shannon theory literature, nuances of closely related, yet formally dissimilar, vanishing criteria have not been widely studied. For example, the distinctions between the vanishing-distortion and the vanishing-error criteria, and between the vanishing-rate and the zero-rate coding are still not well understood. In this paper, with the help of ou...

Even after three decades of research, the Berger-Tung problem remains open. The usual method attempts to identify the eponymous inner and outer bounds, tacitly bypassing the natural reference of the unwieldy operational definition. In this context, we show that the inner and outer bounds sometimes differ, making the usual approach untenable. As an...

At the heart of any modern communication system is the modem, connecting the data source to the communication channel. This first course in the mathematical theory of modem design introduces the theory of digital modulation and coding that underpins the design of digital telecommunications systems. A detailed treatment of core subjects is provided,...

Principles behind lossless and lossy coding are usually considered related, yet distinct. In contrast, we show that the direct statements of the rate-distortion theorem and the lossless coding theorem are consequences of a common distortion-abstracted phenomenon. Significantly, we extend such distortion abstraction to a more general multiterminal f...

In this correspondence, the stopping set of turbo codes with iterative decoding in the binary erasure channel is defined. Block and bit erasure probabilities of turbo codes are studied by using the stopping set analysis. It is found that block and bit erasure probabilities of turbo codes with iterative decoding are higher than those with maximum-li...

Consider the two-terminal partial side information problem, where one source is decoded under a distortion measure, while the other acts as a helper. There are two well known inner bounds on the (convex) achievable region: (i) a bound due to Berger et al., and (ii) a suitable specialization of the general Berger-Tung bound. While the former bound a...

Algebraic geometry is often employed to encode and decode signals transmitted in communication systems. This book describes the fundamental principles of algebraic coding theory from the perspective of an engineer, discussing a number of applications in communications and signal processing. The principal concept is that of using algebraic curves ov...

In this paper, we consider a class of multiterminal source coding problems, where all sources, except one, are reconstructed perfectly. Our framework generalizes the hitherto open single helper problems due to Csiszar and Korner, by adding complete side information at the decoder, and lossy reconstruction of the remaining source. In this setup we o...

Euclidean distance is a discrepancy measure between two real-valued functions. Divergence is a discrepancy measure between two positive functions. Corresponding to these two well-known discrepancy measures, there are two inference principles; namely, the least-squares principle for choosing a real-valued function subject to linear constraints, and...

We propose the signal-to-noise ratio (SNR) transfer characteristic band (TCB) chart of extrinsic information (EI) as a basic tool for the probabilistic convergence analysis of iterative decoding. The proposed SNR TCB chart is a general version of the widely used SNR transfer chart, and it can model the behavior of EI in iterative decoding with any...

In this letter, we propose an estimate for the block erasure probability of turbo codes with an iterative decoding in a waterfall region, which is nonlinearly scaled by the information blocklength. This estimate can be used to predict efficiently the block erasure probability of the finite-length turbo codes over a binary erasure channel

The issue of source/channel duality is as old as information theory itself. Whereas type covering has been identified as a basis for source coding and sphere packing as the basis for channel coding, very little effort has been directed at tying covering and packing together. In this paper, a covering result has been derived which gives asymptotical...

We will illustrate the connection between the Ising problem in statistical mechanics and the problem of computing the constrained capacity of an array of the same dimension. Using this connection, we show that for a given amount of violation, a soft constrained capacity can be computed. The classical Shannon capacity of a constrained channel is mer...

Csiszár's I-divergence is adopted as a discrepancy measure for the nonnegative impulse response intersymbol interference (ISI) channel demodulation. Both blind and nonblind I-divergence demodulation algorithms are discussed and numerically studied. Simulation results show that they have bit error rate (BER) performance close to the Viterbi algorit...

Both blind and nonblind demodulation algorithms based on the Richardson-Lucy algorithm for the one-dimensional intersymbol interference channel with nonnegative impulse response function are proposed and compared numerically with the Viterbi demodulation algorithm and the demodulation algorithms based on the Wiener filter. Simulation results show t...

We present a single letter characterization of the secrecy capacity of the single-input multiple-outputs (SIMO) channel under Gaussian (and possibly colored) noise. To do so, we transform the channel into a scalar Gaussian wiretap channel using standard techniques of communications theory. The result is used to study the impact of slow fading on th...

We present a generalized extrinsic information transfer characteristics of the iterative turbo decoding algorithm. By using this tool, we obtain a lower bound on the bit error rate performance of the finite-length turbo codes showing a gentle waterfall over a wide waterfall region.

Multimode optical fiber links for local area networks have been challenged by the ever increasing demand for higher data rates. The understanding of their fundamental limits thus becomes crucial. Recent research results show that multimode optical fiber channels have much higher channel capacity from the information-theoretic point of view. A modul...

We propose a tool for the analysis of the finite-length iterative turbo decoding algorithm. The proposed tool is a generalized EXIT (extrinsic information transfer) chart based on the mutual information transfer characteristics of the extrinsic information in the iterative turbo decoding algorithm. The proposed tool can describe the probabilistic c...

In this paper, we propose a novel method to obtain the lower bound on the average BER of the finite-length turbo codes in an additive white Gaussian noise (AWGN) channel considering the transition probability of the average BER curve between two asymptotes. For this purpose, we use the SNR TCB chart based on the Gaussian approximation of extrinsic...

A novel method of estimating the BER performance of finite length turbo codes in a binary AWGN channel is proposed. We consider the asymptotes of BER performance at low and high E_b/N₀ resulted from the small and large extrinsic values, respectively. The BER performance curve transitions between two asymptotes in the waterfall...

A selected set of papers documenting accomplishments and new findings are provided in the appendix. The paper compilation in the appendix is meant to be representative of the overall scope of the work and is not intended to be exhaustive. Complete references are given in the Publications section. Most of the papers listed may be downloaded for user...

The SNR transfer characteristic band of the extrinsic value in the turbo decoding is proposed. The proposed method can explain the bimodal Gaussian average histogram of the extrinsic value in the waterfall region of the finite length turbo decoding. The proposed method is also useful to discuss the probabilistic convergence behavior of the finite l...

Multimode optical fiber links for local area networks have been challenged by the ever-increasing demand for higher bit rates. As a result, numerous research efforts have been devoted to such communication channels. The understanding of the fundamental limits of the multimode fiber channels thus becomes crucial. To this end, the capacity of an idea...

Foreword by James L. Massey. Codes, Graphs, and Systems is an excellent reference for both academic researchers and professional engineers working in the fields of communications and signal processing. A collection of contributions from world-renowned experts in coding theory, information theory, and signal processing, the book provides a broad per...

This paper presents the first integrated circuit implementation of a Hermitian decoder thereby proving its practical viability. Hermitian codes provide much larger block lengths (n=4080) compared to that of the popular Reed-Solomon (RS) codes (n=256) over the same field (GF(256)). This translates to a coding gain of 0.6 dB for the same rate. Howeve...

An algorithm is described for demodulating full-surface two-dimensional data, such as two-dimensional on-off keying, in the presence of two-dimensional intersymbol interference, a topic that is becoming important in the field of optical recording

The emergent role of information theory in image formation is surveyed. Unlike the subject of information-theoretic communication theory, information-theoretic imaging is far from a mature subject. The possible role of information theory in problems of image formation is to provide a rigorous framework for defining the imaging problem, for defining...

Sudan (see Journal of Complexity, vol.13, no.1, p.180-93, 1997) published a new algorithm that can decode low rate Reed-Solomon codes beyond their packing radii; he also gave an expression for the error-correcting capability and an upper bound on the number of outputs. We give a simpler derivation which leads to simpler expressions for the error-co...

The class of Hermitian codes is a popular and well-studied class of codes on planar curves over GF(2^m). These codes have large blocklength, and may soon find engineering importance. In contrast to Reed-Solomon codes, which were discovered by engineers, Hermitian codes were introduced by mathematicians as an example of algebraic geometry...

We define a checkerboard code as a two-dimensional binary code that satisfies some constraint, e.g., every binary one must be surrounded by eight zeros, and then investigate the capacity for each of several different constraints. Using a recursive construction we develop a series of loose bounds on the capacity. These bounds, in turn, lead to conje...

The (24,12,8) Golay code over GF(2) is constructed as the subfield-subcode of a three symbol extended (21,16,4) code constructed on the Klein quartic over GF(8) in a nontraditional way

Massey’s theorem is used to determine the minimum distance of the Golay codes. The same method can be used to determine the minimum distance of other cyclic codes. This suggests that Massey’s theorem may be a powerful tool whose uses are not yet fully uncovered.

Minimum-Shift-Keying is a digital modulation which is of particular interest for both practical applications and theoretical study. It is of interest for theoretical study since it can be examined from (at least) five points of view, each one giving different insight and suggesting alternative implementations. While four of the five points of view...

A method of investigating the minimum distance of binary cyclic codes of composite blocklength is described; only the case of blocklength 63 is discussed in any detail. Moreover, the usefulness of the method is left as an unanswered question.

Algebraic structures and number theory underlie much of the development of modern techniques of digital signal processing and digital communications, so it behooves us to begin our study with a formal survey of these mathematical fundamentals. Although the familiar real and complex number systems satisfy most of our everyday needs, we will find pra...

Traditionally, digital signal processing deals with vectors whose components are elements of the complex number system or the real number system. The discrete Fourier transform is commonly defined in the complex field and it forms a fundamental tool in the subject of digital signal processing. Likewise, many of the tasks of error-control codes can...

An n×n Toeplitz matrix is a square matrix in which element a, ij = a i- j . An n×n circulant matrix is a square matrix in which element a, ij = a ((i- j) ). A circulant matrix is a Toeplitz matrix. A Toeplitz system of equations is given by the matrix-vector equation Af = g. The computational task of solving the Toeplitz system of equations is the...

The most important structure in digital signal processing is the nonrecur-sive digital filter known as the finite impulse response (FIR) filter. The FIR filter is simply a tapped delay line. It convolves the input datastream with the filter taps. An incoming datastream of digital samples enters the filter and an outgoing datastream of digital sampl...

Just as the FIR filter, which is closely related to convolutions, Fourier transforms, and Toeplitz matrices, is a central topic of digital signal processing, so the cyclic code is a central topic of error-control codes. A cyclic code is simply an error-control code that satisfies a certain convolution property in the time domain or a certain spectr...

A fast Fourier transform (FFT) algorithm is an efficient and nonobvious procedure for computing the (discrete) Fourier transform that is considerably more efficient than is the obvious form of the computation. Notice that the terminology carefully distinguishes between the Fourier transform as a function and the fast Fourier transform as a computat...

Cyclic codes have been defined from a spectral point of view based on the Fourier transform. Toeplitz matrices are directly related to spectral analysis, so it will not be surprising that the algorithms for inverting Toeplitz systems of equations can be applied to the decoding of cyclic codes. This chapter will make this connection explicit by deve...

The Gleason-Prange theorem describes a nontrivial automorphism of an extended quadratic residue code. By using the Fourier transform in an appropriate extension field, methods of digital signal processing are used to make the proof transparent and the automorphism intuitive. The theorem is proved both in a field of characteristic tow and in an arbi...

This survey paper is intended to integrate the subjects of digital signal processing and error control codes by studying their common dependence on the properties of the discrete Fourier transform. The two subjects are traditionally studied in different algebraic fields. Usually, the computations of digital signal processing are done using the comp...

Efficient signal processing algorithms are important for embedded and power-limited applications since, by reducing the number of computations, power consumption can be reduced significantly. Similarly, efficient algorithms are also critical to very large scale applications such as video processing and four-dimensional medical imaging. This self-co...

Algorithms for computation are a central part of digital signal processing and of decoders for error-control codes. When restricted to the study of their computational algorithms, there is not much to distinguish those two subjects. Only the arithmetic field is different; in one case the real or complex field, and a Galois field in the other. Even...

Two architectures for universal Reed-Solomon decoders are given. These decoders, called time-domain decoders, work directly on the raw data word as received without the usual syndrome calculation or power-sum-symmetric functions. Up to the limitations of the working registers, the decoders can decode any Reed-Solomon codeword or BCH codeword in the...

A number of efficient decoding algorithms for Reed-Solomon codes of blocklength n have been developed; decoders with order of n**2 computational steps. The purpose of these lectures is to sharpen the attack on the complexity of the decoders. One measure of computational complexity used is the number of multiplications in the computation. A secondar...

By using the theory of finite field Fourier transforms, the subject of error control codes is described in a language familiar to the field of signal processing. The many important uses of spectral techniques in error control are summarized. Many classes of linear codes are given a spectral interpretation and some new codes are describe. Several al...

Communication theory and signal processing are closely related subjects that have been developed largely by engineers. Analysis and synthesis problems in these fields depend heavily on reasoning in the frequency domain. Thus, in the study of real- or complex-valued signals, the Fourier transform plays a basic role. When the time variable is discret...

The performance of Channel block codes for a general channel is studied by examining the relationship between the rate of a code, the joint composition of pairs of codewords, and the probability of decoding error. At fixed rate, lower bounds and upper bounds, both on minimum Bhattacharyya distance between codewords and on minimum equivocation dista...

A large class of lower bounds relating to the performance of hypothesis testers, channel codes, and source compression codes is developed. These are extensions of Fano's inequality on the one hand, and of the discrimination inequality of Kullback on the other. The hypothesis testing and channel coding bounds are interesting primarily for small bloc...

The testing of binary hypotheses is developed from an information-theoretic point of view, and the asymptotic performance of optimum hypothesis testers is developed in exact analogy to the asymptotic performance of optimum channel codes. The discrimination, introduced by Kullback, is developed in a role analogous to that of mutual information in ch...

By defining mutual information as a maximum over an appropriate space, channel capacities can be defined as double maxima and rate-distortion functions as double minima. This approach yields valuable new insights regarding the computation of channel capacities and rate-distortion functions. In particular, it suggests a simple algorithm for computin...

In this work we study codes characterized b y t h e p r operty that their parity check matrix is circulant, i.e., that rows are obtained as all the distinct cyclic shifts of the rst row. For these codes, we give simple expressions for their dimension and for a lower bound on their minimum distance. We also present an Ond algorithm to solve the prob...

Thesis (Ph. D.)--Cornell University, August, 1972.