Brian Marcus

University of British Columbia - Vancouver | UBC · Department of Mathematics

We propose two modified versions of the classical gradient ascent method to compute the capacity of finite-state channels with Markovian inputs. For the case that the channel mutual information rate is strongly concave in a parameter taking values in a compact convex subset of some Euclidean space, our first algorithm proves to achieve polynomial a...

We propose a modified version of the classical gradient descent method to compute the capacity of finite-state channels with Markovian input. Under some concavity assumption, our algorithm proves to achieve a polynomial accuracy in a polynomial time for general finite-state channels. Moreover, for some special families of finite-state channels, our...

We develop a new pressure representation theorem for nearest-neighbour Gibbs interactions and apply this to obtain the existence of efficient algorithms for approximating the pressure in the $2$-dimensional ferromagnetic Potts, multi-type Widom-Rowlinson and hard-core models. For Potts, our results apply to every inverse temperature but the critica...

Given an equilibrium state $\mu$ for a continuous function $f$ on a shift of finite type $X$, the pressure of $f$ is the integral, with respect to $\mu$, of the sum of $f$ and the information function of $\mu$. We show that under certain assumptions on $f$, $X$ and an invariant measure $\nu$, the pressure of $f$ can also be represented as the integ...

We introduce a concept of independence entropy for symbolic dynamical systems. This notion of entropy measures the extent to which one can freely insert symbols in positions without violating the constraint defined by the shift space. We show that for a certain class of one-dimensional shift spaces X, the independence entropy coincides with the lim...

We prove that under mild positivity assumptions, the entropy rate of a continuous-state hidden Markov chain, observed when passing a finite-state Markov chain through a discrete-time continuous-output channel, is analytic as a function of the transition probabilities of the underlying Markov chain. We further prove that the entropy rate of a contin...

For any stationary $\mZ^d$-Gibbs measure that satisfies strong spatial mixing, we obtain sequences of upper and lower approximations that converge to its entropy. In the case, $d=2$, these approximations are efficient in the sense that the approximations are accurate to within $\epsilon$ and can be computed in time polynomial in $1/\epsilon$.

We consider a memoryless channel with an input Markov process supported on a mixing finite-type constraint. We continue the development of asymptotics for the entropy rate of the output hidden Markov chain and deduce that, at high signal-to-noise ratio, the mutual information rate of such a channel is concave with respect to “almost” all input Mark...

In this paper we show that any one-dimensional stationary, finite-valued Markov Random Field (MRF) is a Markov chain, without any mixing condition or condition on the support. Our proof makes use of two properties of the support $X$ of a finite-valued stationary MRF: 1) $X$ is non-wandering (this is a property of the support of any finite-valued st...

For a class of $\zz^2$ Markov Random Fields (MRFs) $\mu$, we show that the sequence of successive differences of entropies of induced MRFs on strips of height $n$ converges exponentially fast (in $n$) to the entropy of $\mu$. These strip entropies can be computed explicitly when $\mu$ is a Gibbs state given by a nearest-neighbor interaction on a st...

We consider a finite-state memoryless channel with i.i.d. channel state and the input Markov process supported on a mixing finite-type constraint. We discuss the asymptotic behavior of entropy rate of the output hidden Markov chain and deduce that the mutual information rate of such a channel is concave with respect to the parameters of the input M...

A method for computing lower bounds on capacities of two-dimensional (2D) constraints having a symmetric presentation in either the horizontal or the vertical direction is presented. The method is a generalization of the method of Calkin and Wilf (SIAM J. Discrete Math. , 1998). Previous best lower bounds on capacities of certain constraints are im...

We derive an asymptotic formula for entropy rate of a hidden Markov chain under certain parameterizations. We also discuss applications of the asymptotic formula to the asymptotic behaviors of entropy rate of hidden Markov chains as outputs of certain channels, such as binary symmetric channel, binary erasure channel, and some special Gilbert-Ellio...

The aim of this chapter is to present, in appropriate perspective, some selected topics in the theory of variable-length codes. One of the domains of applications is lossless data compression. The main aspects covered include optimal prefix codes and finite automata and transducers. These are a basic tool for encoding and decoding variable-length c...

We consider a finite-state memoryless channel with i.i.d. channel state and the input Markov process supported on a mixing finite-type constraint. We discuss the asymptotic behavior of entropy rate of the output hidden Markov chain and deduce that the mutual information rate of such a channel is concave with respect to the parameters of the input M...

In this note, we show that small complex perturbations of positive matrices are contractions, with respect to a complex version of the Hilbert metric, on the standard complex simplex. We show that this metric can be used to obtain estimates of the domain of analyticity of entropy rate for a hidden Markov process when the underlying Markov chain has...

A method for computing lower bounds on capacities of 2-dimensional constraints having a symmetric presentation in either the horizontal or the vertical direction is presented. The method is a generalization of the method of Calkin and Wilf (SIAM J. Discrete Math., 1998). Previous best lower bounds on capacities of certain constraints are improved u...

The maximum insertion rate of a one-dimensional constrained system over a finite alphabet is defined to be the maximum density of positions that can be freely, and independently, filled in with arbitrary symbols of the alphabet and still satisfy the constraint. In this paper, this concept is extended to higher dimensional constraints, that is, to c...

We generalize a result in [8] and derive an asymptotic formula for entropy rate of a hidden Markov chain around a "weak Black Hole". We also discuss applications of the asymptotic formula to certain channels.

We study the classical problem of noisy constrained capacity in the case of the binary symmetric channel (BSC), namely, the capacity of a BSC whose inputs are sequences chosen from a constrained set. Motivated by a result of Ordentlich and Weissman [In Proceedings of IEEE Information Theory Workshop (2004) 117--122], we derive an asymptotic formula...

In most recording channels, modulation codes are employed to transform user data to sequences that satisfy some desirable constraint. Run-length-limited (RLL) and maximum tran- sition run (MTR) systems are examples of constraints that improve timing and detection performance. When used in conjunction with error correction codes (ECC), schemes that...

Consider a hidden Markov chain obtained as the observation process of an ordinary Markov chain corrupted by noise. Recently Zuk et al showed how, in principle, one can explicitly compute the derivatives of the entropy rate of at extreme values of the noise. Namely, they showed that the derivatives of standard upper approximations to the entropy rat...

In this paper, we generalize a result in [18] and derive an asymptotic formula for the entropy rate of a hidden Markov chain, observed when a Markov chain passes through a binary symmetric channel. And we prove an asymptotic formula for the capacity of a binary symmetric channel with input process supported on an irreducible finite type constraint.

We prove that under mild positivity assumptions the entropy rate of a hidden Markov chain varies analytically as a function of the underlying Markov chain parameters. A general principle to determine the domain of analyticity is stated. An example is given to estimate the radius of convergence for the entropy rate. We then show that the positivity...

Maximum transition run (MTR) constrained systems are used to improve detection performance in storage channels. Recently, there has been a growing interest in time-varying MTR (TMTR) systems, after such codes were observed to eliminate certain error events and thus provide high coding gain for EⁿPR4 channels for n=2,3. In this work, TMTR...

We give relaxed sufficient conditions (compared to D. Blackwell (1957)) for analyticity of the entropy rate of a hidden Markov chain. Several special cases of the relaxed conditions are discussed. A general principle to calculate the domain of analyticity is stated. An example is given to estimate the radius of convergence for the entropy rate. Fin...

We introduce a new method for analyzing and constructing combined modulation and error-correcting codes (ECCs), in particular codes that utilize some form of reverse concatenation and whose ECC decoding scheme requires easy access to soft information. We expand the work of Immink and Wijngaarden and also of Campello, Marcus, New, and Wilson, in whi...

We prove that under mild assumptions a hidden Markov chain varies analytically, in a strong sense, as a function of the underlying Markov chain parameters. In particular, we show that, under these assumptions, the entropy rate of a hidden Markov chain is an analytic function of the parameters. We give examples to show how this can fail in some case...

We consider a class of encoders for constrained systems, which we call block-type-decodable encoders. For a constrained system presented by a deterministic graph, we design a block-type-decodable encoder by selecting a subset of states of the graph to be used as encoder states. Such a subset is known as a set of principal states. Our goal is to fin...

In digital storage systems where the input to the noisy channel is required to satisfy a modulation constraint, the constrained code and error-control code (ECC) are typically designed and decoded independently. The achievable rate for this situation is evaluated as the rate of average intersection of the constraint and the ECC. The gap from the ca...

Recently introduced methods to compute the capacity of a constrained code over a noisy channel are used to estimate the potential improvement in performance when the error-control code (ECC) is designed in conjunction with the constraint, as compared with an average ECC for the channel. This constraint gain is computed for various (d, k)-RLL constr...

A channel with inter-symbol interference and random perturbations to timing is approximated using a finite-state Markov model. The inter-symbol interference is approximated by a finite-impulse response and the timing perturbations are approximated by a first-order random walk. A Viterbi algorithm is then applied to find the maximum-likelihood seque...

A constrained system is presented by a finite-state labeled graph. For such systems, we focus on block-type-decodable encoders, comprising three classes known as block, block-decodable, and deterministic encoders. Franaszek (1968) gives a sufficient condition which guarantees the equality of the optimal rates of block-decodable and deterministic en...

It is shown via a combinatorial approach that an infinite cascade of reverse concatenation with independent decoding of the error control code (ECC) and constraint yields a capacity equal to the rate of average intersection.

This paper investigates the performance of the Kalman filter as a timing recovery system in tracking mode, subject to a wide range of operating conditions. Our simulation compares the Kalman filter and the phase-locked loop based on the number of divergences for various values of timing disturbance and SNR. They are shown to work equally well in lo...

We develop methods for analyzing and constructing combined modulation/error-correcting codes (ECC codes), in particular codes that employ some form of reversed concatenation and whose ECC decoding scheme requires easy access to soft information (e.g., turbo codes, low-density parity-check (LDPC) codes or parity codes). We expand on earlier work of...

A constrained system or constraint S=SG is the set of all finite sequences generated by a labeled graph G by reading the labels along walks on the graph. We say that G is a presentation of S. A graph G is deterministic if at each state, the outgoing edges are labeled distinctly. Some well-known constraints include runlength limited RLL(d,k) and max...

A rate p : q block encoder is a dataword-to-codeword assignment from 2^p p-bit datawords to 2^p q-bit codewords, and the corresponding block decoder is the inverse of the encoder. When designing block encoders/decoders for constrained systems, often, more than 2^p codewords are available. In this paper, as our main con...

"Volume 150, number 710 (first of 5 numbers)" Incluye bibliografía

The sum-product and min-sum algorithms are used to decode codes defined by trellises. In this paper, we discuss the behavior of these and related algorithms on tail-biting (TB) trellises. The convergence of the sum-product algorithm on tail-biting trellises was analyzed recently by Anderson and Hladik [2] and was shown to give approximate a posteri...

Holographic storage has the potential to become a digital data storage technology with fast readout and high density. Computer users have come to expect, however, that data retrieved from their storage devices will be retrieved error-free (with a probability of error <10 -12 ). In both conventional storage devices and holographic data storage, achi...

We present an overview of our research effort on volume holographic digital data storage. Innovations, developments, and new insights gained in the design and operation of working storage platforms, novel optical components and techniques, data coding and signal processing algorithms, systems tradeoffs, materials testing and tradeoffs, and photon-g...

Time-varying encoders for constrained systems are introduced. The approach generalizes the state-splitting (ACH) algorithm in a way that yields encoders consisting of multiple phases, with encoding proceeding cyclically from one phase to the next. The framework is useful for design of high-rate codes with reduced decoder error propagation and reduc...

nd. Researchers recently demonstrated the storage of 10,000 superimposed holograms, 1 the use of digital-processing techniques for data extraction, 3 nonmechanical data access, 2 and high areal density (10 bitsymm 2 ) in thin materials. 4 These experiments demonstrated the potential for either high capacity or fast readout but not both, because no...

A method is presented for designing lossless sliding-block compression schemes that map constrained sequences onto unconstrained ones. The new compression scheme is incorporated into a coding technique for noisy constrained channels, which has applications to magnetic and optical storage. As suggested previously by Immink (see ibid., vol.43, p.1389...

We present an initial experimental evaluation of coding and signal processing tradeoffs in high-density holographic data storage. Block-based and low-pass modulation codes, predistortion of holographic pages during recording (pre- processing), and conventional equalization (post-processing) are compared using a few recorded holograms. The relative...

We give necessary and sufficient conditions for a Markov chain to factor onto a Bernoulli shift (i) as an eventual right-closing factor, (ii) by a right-closing factor map, (iii) by a one-to-one a.e. right-closing factor map, and (iv) by a regular isomorphism. We pass to the setting of polynomials in several variables to represent the Bernoulli shi...

We describe a framework for designing encoders that transform arbitrary data sequences into two-dimensional arrays satisfying certain constraints, in particular, constraints that guarantee arrays with limited high spatial frequency content. We also exhibit specific codes that produce such arrays. Such codes are useful for holographic recording syst...

We compare performance and error propagation of DFE with and without a d=1 RLL code, at 2.67 user density and with a single coefficient FIR phase equalizer. Performance without error propagation is slightly better with d=1 in spite of the rate loss, because precursor ISI can be completely eliminated. We develop a model to estimate the effects of er...

We describe a generalization of the state-splitting algorithm (also known as the Adler-Coppersmith-Hassner (1983) algorithm) for constructing encoders which encode arbitrary data into constrained systems of sequences. In the generalized algorithm, we replace approximate eigenvectors with approximate eigenmatrices to yield a framework for designing...

We describe a digital holographic storage system for the study of noise sources and the evaluation of modulation and error-correction codes. A precision zoom lens and Fourier transform optics provide pixel-to-pixel matching between any input spatial light modulator and output CCD array over magnifications from 0.8 to 3. Holograms are angle multiple...

We completely classify one-sided Markov chains up to measure-theoretic isomorphism. The classification is effective and computable.

Finite-state encoders that encode n-ary data into a constrained system S are considered. The anticipation, or decoding delay, of such an (S,n)-encoder is the number of symbols that a state-dependent decoder needs to look ahead in order to recover the current input symbol. Upper bounds are obtained on the smallest attainable number of states of any...

In the past decade, complexity issues have become increasingly prominent in coding research. A number of mutually reinforcing sub-themes, springing from multiple distinct research communities have come together to generate a powerful research tide in this area. This special issue presents several articles showing a powerful surge of research on com...

To be competitive with alternate technologies, holographic data storage and retrieval systems will need improved components, signal-processing methods, and storage materials.

It is known that each Markov chain has associated with it a polytope and a family of Markov measures indexed by the interior points of the polytope. Measure-preserving factor maps between Markov chains must preserve the associated families. In the present paper, we augment this structure by identifying measures corresponding to points on the bounda...

The existence and uniqueness of a canonical minimal encoder for any given sliding block decoder are proven. The structure of this encoder is given in terms of an explicit sequence of state splittings. The universality of the state splitting algorithm for code construction is clarified.

An input-constrained channel is defined as the set S of finite sequences generated by a finite labeled directed graph which defines the channel. A construction based on a result of Adler, Goodwyn, and Weiss (1977) is presented for finite-state encoders for input-constrained channels. Let G=(V, E) denote a smallest deterministic presentation of S. F...

These lecture notes address aspects of the theory and design of constrained codes that find applications in digital data storage devices. The notes provide background for the material presented in a lecture in the short course on ‘coding theory’ held in conjunction with the AMS meeting in San Francisco, California, January 1995. The notes are large...

Several constructions are presented for extending a bounded-to-one sliding-block code to a bounded-to-one surjection onto its range, while preserving nice properties of the original code.

First Page of the Article

Let M denote the distribution of an irreducible Markov chain supported by a finite directed graph L, and let P(n) denote the empirical type of the first n transitions. Csiszar, Cover, and Choi examined the large deviation properties of P(n) and proved conditional limit theorems subject to linear inequality constraints. We consider linear equality c...

Nonconstructive existence results are obtained for block error-correcting codes whose codewords lie in a given constrained system. Each such system is defined as a set of words obtained by reading the labels of a finite directed labeled graph. For a prescribed constrained system and relative minimum distance δ, the new lower bounds on the rate of s...

We prove several theorems about automorphisms of Markov chains, using the weight-per-symbol polytope.

The authors provide a self-contained exposition of modulation code design methods based upon the state splitting algorithm. They review the necessary background on finite state transition diagrams, constrained systems, and Shannon (1948) capacity. The state splitting algorithm for constructing finite state encoders is presented and summarized in a...

A new class of constrained systems average runlength constraints (ARCs), is defined by requiring that the sum of n consecutive run lengths be bounded above by a linear function of n. In particular, the running average runlength of every sequence in the system is bounded above by a constant. A general result is given on the capacity of ARC systems....

The authors obtain general lower bounds on the number of states in any encoder for a given constrained system and rate. Lower bounds on the number of states are exhibited in a fixed-rate finite-state encoder that maps unconstrained n-ary sequences into a given set of constrained sequences, defined by a finite labeled graph G. In particular, one sim...

We prove some results related to the question of the existence of factor maps and eventual factor maps between shifts of finite type. Our main result is that if A and B are integral eventually positive (IEP) matrices, and A eventually factors finite-to-one onto B, then there exists an IEP matrix C such that A eventually factors onto C by left closi...

We study Markov chains via invariants constructed from periodic orbits. Canonical extensions, based on these invariants, are used to establish a constraint on the degree of finite-to-one block homomorphisms from one Markov chain to another. We construct a polytope from the normalized weights of periodic orbits. Using this polytope, we find canonica...

In digital data transmission (respectively, storage systems), line codes (respectively, recording codes) are used to tailor the spectrum of the encoded sequences to satisfy constraints imposed by the channel transfer characteristics or other system requirements. For instance, pilot tone insertion requires codes with zero mean and zero spectral dens...

We classify finite-to-one factor maps between shifts of finite type up to almost topological conjugacy.

We characterize the sofic systems which have minimal subshift-of-finite-type covers and derive some consequences.

this paper, we will completely answer the following questions for an arbitrary Markov chain and a Bernoulli shift. (1) Does the Markov chain eventually factor onto the Bernoulli shift by right-closing maps? (2) Does the Markov chain factor onto the Bernoulli shift by a right-closing map? By a right-closing map of degree 1? (3) Is the Markov chain r...

Incluye bibliografía e índice