Content uploaded by Kees Schouhamer Immink

Author content

All content in this area was uploaded by Kees Schouhamer Immink on Apr 11, 2019

Content may be subject to copyright.

IEEE TRANSACTIONS ON INFORMATION THEORY,VOL. IT-29, NO. 5, SEPTEMBER 1983 751

TABLE I

THE(~~,&)= (~,~)FI~ED(T= I)NONCATASTROPHICCOWOLUTIONAL

ENCODERWITHMEMORY

M

= ~ANDMAXIMUMFREEDISTANCE

TOGETHERWITHSOMEBETTERTIME-VARYINGENCODERS'

T G$ GT G2* G3 G4* d, Nm L

1

3 2 2 1 3 I 2 4

2 E A C 7 3/2 3

3 B 7

3 3A 1c 00 7 4/3 8/3

OE 3B 00

03 2D 30

4 E9 CO 7 3/2 13/4

3B 70

OD EC

03 B7

5 3A7 000 7 e/5 13/5

OED 300

037 1co

OOD 3B0

003 2DC

‘d,

The fixed encoder has maximum free distance and is optimum in

the sense of smallest number of weight d, paths per time instant N, and

in the sense of smallest average number of information bit errors along

these paths I,, The encoders are specified by the

(n,,T, k,,T)

matrices G,*

of (7); the rows of G,* are written in hexadecimal form; e.g., the upper row

of G,f for

T

= 2 is specified as “E” and hence is the row [ 1 1 1 01.

where G,? for 0 < j 4 M* is the (Tk,) X (Tn 0) matrix

GjT (0) GjT+l(l) ... G,T+T-I(T- 1)

GjT- I (0)

G,,(l) "' G,T+T-#- l>

G;=

. __

G,,-T+,(O) G,;-T+z(~) ... Gj,(T- 1)

(7)

where by way of convention G,(u) = 0 for j > it4 and for j < 0

and where

M* is the smallest integer equal to or greater than

M/T, i.e.,

M* = [M/T].

(8)

Thus, every (n,, k,) periodic convolutional encoder with period

T and memory M can be considered to be an (n*,, ki) fixed

convolutional encoder with memory M* given by (8) and with

k,* = Tk, (94

n; = Tn,. (9b)

This equivalence permits upper bounds on code distance, as well

as software, developed for FCE’s to be applied to PCE(T)‘s.

III.

NEW CODES

We now report some positive results from a computer-aided

search for noncatastrophic PCE( T)‘s with (n,, k,) = (2,l) that

are superior to the best noncatastrophic FCE with the same

parameters n,, k,, and M. The search was concentrated on the

case M = 4 as this is the smallest M such that Heller’s upper

bound [4] on d,, namely d, = 8, is not achieved by any FCE.

The results of the search are given in the Table I. The T =

1

entry is the fixed convolutional encoder found by Oldenwalder

[5] that has maximum d,, namely dm = 7, and is optimum both

in the sense of minimum ypo and m the sense of minimum 1,.

For T > 1, the codes are time-varying but are specified by the

corresponding fixed encoding matrices Cl*, 0 < j < M*, defined

by (7).

In Table I, the encoders with period T = 2 and 3 were found

by an exhaustive search to be optimal in the sense of minimizing

N,; the codes for T = 4 and 5 were the best found in a heuristic

nonexhaustive search. The codes given in Table I for T = 2, 3, 4,

and 5 are all superior to the best fixed code (T = 1) both in the

sense of smaller N, and also in the sense of smaller 1,.

It is somewhat disappointing that no time-varying codes with

larger d, than the best fixed code were found. It seems likely

that no such superiority is possible for M = 4 when (n,, k,) =

(2, I); the next M for which such superiority is possible is M = 7

where Heller’s bound gives d, < 11 but the best fixed code has

d, = 10. It is encouraging, however, that periodic codes superior

to the best fixed codes could be found at all, as no such instances

could be found in- the prior literature.

ACKNOWLEDGMENT

The author is very grateful to Prof. James L. Massey who not

only suggested this work but also devoted much time and pa-

tience in supervising it.

[II

[21

[31

[41

[51

REFERENCES

J. L. Massey, “Error bounds for tree codes, trellis codes and convolutional

codes with encoding and decoding procedures,” in Coding

and Complexity,

G. Longo, Ed., ‘C.I.S.M. Courses and Lectures No. 216. New York:

Springer-Verlag, 1974, pp. l-57.

A. J. Viterbi, “Error bounds for convolutional codes and an asymptoti-

cally optimum decoding algorithm,”

IEEE Trans. Inform. Theory,

vol.

IT-13, pp. 260-269, Apr. 1967.

-, “Convolutional codes and their performance in communication

systems,”

IEEE Trans. Commun. Technol., vol.

COM-19, pp. 751-771,

Oct. 1971.

J. A. Heller, “Sequential decoding: Short constraint length convolutional

codes,” Jet Propulsion Lab., California Inst. of Technology, Pasadena,

Space Program Summary 37-54, vol. 3, pp. 171-174, Dec. 1968.

J. P. Oldenwalder, “Optimal decoding of convolutional codes,” Ph.D.

dissertation, School of Engineering and Applied Science, University of

California, Los Angeles, CA, 1970.

A Generalized Method for Encoding and Decoding

Run-Length-Limited Binary Sequences

G. F. M. BEENKER AND K. A. SCHOUHAMER IMMINK

Abstract-Many modulation systems used in magnetic and optical re-

cording are based on binary run-length-limited codes. We generalize the

concept of dk-limited sequences of length n introduced by Tang and Bahl

by imposing constraints on the maximum number of consecutive zeros at

the beginning and the end of the sequences. It is shown that the encoding

and decoding procedures are similar to those of Tang and Bahl. The

additional constraints allow a more efficient merging of the sequences. We

demonstrate two constructions of run-length-limited codes with merging

rules of increasing complexity and efficiency and compare them to Tang

and Bahl’s method.

I.

INTRODUCTION

Many baseband modulation systems applied in magnetic and

optical recording are based on binary run-length-limited codes

[l], [2], [3]. A string of bits is said to be run-length-limited if the

Manuscript received April 5, 1982; revised November 30, 1982. This work

was partially presented at the IEEE International Symposium on Information

Theory, Les Arcs, France, June 21-25, 1982.

The authors are with the Philips Research Laboratories, 5600 MD

Eindhoven, The Netherlands.

0018-9448/83,‘0900-075 l$Ol .OO 01983 IEEE

152 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. IT - 29, NO. 5, SEPTEMBER 1983

number of consecutive zeros between adjacent ones is bounded

between a certain minimum and a certain maximum value. The

upper run-length constraint guarantees a transition within a

specified time interval needed for clock regeneration at the re-

ceiver. The lower run-length constraint is imposed to reduce the

intersymbol interference. The latter constraint appears to have a

bearing on the spectral properties of the sequences [4].

In this cvrrespondence we consider an encoding and decoding

procedure for a special class of run-length-limited codes. The

constraints we consider can be defined as followed. Let n, d, k, I,

and r be given, where 1 6 k and r < k. A binary sequence of

length n is called a dklr-sequence if it satisfies the following

constraints:

a) d-constraint: any two ones are separated by a run of at least

d consecutive zeros;

b) k-constraint: any run of consecutive zeros has a maximum

length k;

c) /-constraint: the number of consecutive leading zeros of the

sequence is at most 1;

d) r-constraint: the number of consecutive zeros at the end of

the sequence is at most r.

A sequence satisfying the d- and k-constraints is called a dk-

sequence.

Given certain desired run-length constraints, it is not trivial

how to map uniquely the input data stream onto the encoded

output data stream. A systematic design procedure on the basis

of fixed length sequences has been given by Tang and Bahl [l].

Their method is based on mapping dk-sequences of length n onto

consecutive integers and vice versa. In this correspondence we

intend to generalize their results to dklr-sequences. We are also

going to show that the application of dklr-sequences enables the

sequences of length n to be merged efficiently without violation

of the d- and k-constraints at the boundaries.

We assume r > d. Theorems 1 and 2 of this correspondence

remain valid for r < d and Theorem 3 can be generalized accord-

ingly. The proofs, however, are less elegant, the reason being that

for r < d there are no dklr-sequences (x,,- , , . . , x0) having a one

at positions j where r < j < d.

II. ENCODING AND DECODING

In this section we consider a way of mapping the set of all

dklr-sequences onto a set of consecutive integers and vice versa.

Our results are similar to those obtained by Tang and Bahl for

dk-sequences.

Let A,, be the set of all dklr-sequences of length n. A, can be

embedded in a larger set 6?,, consisting of the all-zero sequence of

length n and of all binary sequences of length n satisfying the d- ,

k- , and r-constraints where the number of consecutive leading

zeros is allowed to be greater than k.

The set a,, can be ordered lexicographically as follows: if

x = ( x!,~ , , . , x,,) and y = (y,,_ , , . ’ . , yO) are elements of @n

theny is called less than x, y -+ x, if there exists an i, 0 < i < n,

such that y, < x, and x, ,= y, for i < j < n. The position of x in

the lexicographical ordermg of a,,, is denoted by r(x); i.e., r(x) is

the number of ally in @n withy + x. Consequently r(0) = 0.

For the sake of convenience we introduce the residual of a

vector. Lety = (y,,-,;..,~a) E a,,,, y * 0, and let t be such that

yt = 1 and y, = 0 if t < j < n. Then the residual of y, res( y), is

defined as follows:

res( y): = y - A,, where At, =

1,

ifi=

t;

0, elsewhere,

res (0): = 0.

It can easily be seen that y E &,, implies res ( y) E a,,,. The

following observation is basic to the proof of Theorem 1. Let

X, u E W,, and assume that x, = u, = 0

(t

< j < n) and x, = nt

= 1 (0 <

t

< n). Then it is not difficult to show that r(n) -

r(x) = r(res(u)) - r(res(x)).

Let N,.(i), i > 0, be the number of dklr-sequences with I = k of

length i and let N,.(O) = 1.

Theorem 1: Let x = (x,~,;.., xc,) E @,. Then

n-l

r(x) = c x,W(j).

j=O

Proof: Let the nonzero coordinates of x be indexed by

i, < i, < . . < i,, i.e., x, = 1 if and only if i E (i,; . ., iq}. Let

I( be the smallest element of @,, with the property that ulq = 1.

Then it is not difficult to see that the second 1 of I( occurs at

position i, - k - 1, if i, - k - 1 2 0 (otherwise u has only one

1, if r > i,, or also u0 = 1 if r < i4). Here we recall that the

all-zero sequence of length n is a dklr-sequence if and only if

n < min(l, r}. Lete(i,, r) = 1 ifr > i,,andc(i,, r) = Oifr < i,.

Then we obtain

r(u) - r(res (u)) = the number of dklr-sequences with I = k of

length i, with their leftmost 1 at position j where

max(O, i, - k - l} < j < i, + c(i,, r) = N,.(i,).

Furthermore, on the basis of the above mentioned observation it

holds that r(x) = r(u) + r(x) -r(u)

= r(u) + r(res(x)) - r(res(u))

= r(res(x)) + Nr(iq).

The theorem then follows by induction. Q.E.D.

We have found a simple method for mapping the elements of

&,, onto consecutive integers. For practical applications we need a

mapping of the elements of A, onto the set (0, 1, . , ]An] - l},

where ]A,,] is the cardinality of A,,. Obviously the set A, consists

of the lA,1l largest elements of @,. In addition it is clear that the

number a of elements of &,, that are smaller than all the elements

of A,, is equal to r(u); i.e., cr = r(a), where a is the smallest

element of A,,. In this way we have proved the following theorem.

Theorem 2: The transformation t: A,, + N

U

(0) defined by

t(x)

= r(x) - (Y for all x E A,, is a one-to-one mapping from A,,

onto the set (0, 1, . . . , lAnl - l} which preserves the ordering of

A,,, i.e., x ~0 y if and only if t(x) < t(y).

The number (Y can also be expressed in another way. In order

to do so we define N,!)(j), j > 0, to be the number of dk-

sequences of length j with their leftmost element equal to 1 and

satisfying the r-constraint, NrO(0): = 1. It is not difficult to show

that n-l-l n-l-l

a = c N:(j) + 1 = c NY(j).

j=l j=O

The numbers N,(j) and N!‘(j) are easily computed. They can be

found by a straightforward computer search. A more sophisti-

cated approach to finding these numbers can be based on the

finite state transition matrix corresponding to the dklr-sequences

]51.

The conversion from integers to dklr-sequences of length n is

aho

analogous to Tang and Bahl’s method and can be carried out

as follows. Let T,, . . ., T,- , be integers defined by

T = the number of elements y in W, smaller than the smallest

element u in & with u, = 1 and u, = 0 for j > i,

i.e.,

T, = i N:(j) + 1 = i N:(j).

j=l j=O

IEEE TRANSACTIONS ON INFORMATION THEORY,VOL. IT-29, NO. 5, SEPTEMBER 1983 753

TABLE I

MERGINGOF~~~T-SEQUENCESWHERE OJ STANDS

FOR~CONSECUTIVE

ZEROS

$3 t

Merging Bits

s+t+dik+l Od

s+t+d>k+l

ifs<d o”-“lo”- I

ifs > d 1od-

From the definition and our assumption r 2 d it immediately

follows that T, < T, < . . . < T,- ,. These integers are used for

mapping consecutive integers onto dklr-sequences of length n as

shown in the following theorem.

Theorem 3: Let x = (x,- ,; . ., x0) be a dk/r-sequence of

length n. Then a) x, = 1 and xj = 0 for

t

< j < n e T, < r(x) <

T,, , Furthermore, if x, = 1 and x, = 0 for

t

< j < n, then b)

T-,_, d r(x) - N,(t) < Ted.

Proof: a) This statement follows from the definition of 7;. b)

Let x be a dklr-sequence of length n with x, = 1 and x, = 0 for

t

cj < n. Then Trek-, < r(res(x)) < Tr-d. Hence b) follows,

since r(x) -

r(res(x)) = N,(t).

Q.E.D.

The conversion from integers to dk-sequences can therefore

also be generalized to the conversion from. integers to dklr-

sequences. The following simple encoding algorithm for dklr-

sequences can be derived from this theorem. Given an integer I in

the set R = {r(x)lx E A,} (1 = 0 if x = 0), we first locate the

largest possible

t,

0 <

t

<

n,

such that T < I < I;+, and we

make x, = 1. Subtracting the contribution of x, in I, we get a new

integer I -

N,(t).

Theorem 3 can be used again to find the next

nonzero component of x. The second part of Theorem 3 assures

us that x, will be followed by at least d, but no more than k zeros.

III. THE EFFICIENCY OF dklr-SEQUENCES

In modulation systems the dklr-sequences of length n

cannot in

general be cascaded without violating the dk-constraint at

the

boundaries. Inserting a number /3 of merging bits between adja-

cent n-sequences makes it possible to preserve the & and k-con-

straints for the cascaded output sequence. The dk-sequences need

p = d + 2 merging bits [l], whereas only /3 = d merging bits are

required for dklr-sequences, provided that the parameters J and r

are suitably chosen. Hence this method is more efficient, espe-

cially for small values of n. We shall now demonstrate two

constructions of codes with merging rules of increasing complex-

ity and efficiency.

Construction I: Choose d, k, r, and n such that r + d < k. Let

I= k - d - r and p = d. Then the dklr-sequences of length n

can be cascaded without violating the d- and k-constraints if the

merging bits are all set to zero.

Construction 2: Choose d, k, and n such that 2d - 1 < k. Let

r = I = k - d and /? = d. Then the dklr-sequences of length n

can be cascaded without violating the d- and k-constraints if the

merging bits are determined by the following rules. Let an

n-sequence end with a run of s zeros (s < r) while the next

n-sequence start with

t (t

< f) leading zeros. Table I shows the

merging rule for the /3 = d merging bits.

The number m of data bits that can be represented uniquely by

a dk[r-sequence of length n is given simply by

m = [log2 IA,I~ f

where Lx] is the greatest integer not greater than x. The ratio R

of the number of data bits and the number of needed channel

bits is called the information rate of the code. For example, the

information rate of the codes based on the two above-mentioned

constructions equals R = m/(n + d). The asymptotic informa-

tion rate is the capacity C of Shannon’s discrete noiseless run-

TABLE II

BLOCKCODESBASEDONCONSTRUCTION

1

k n R C q = R/C

7 12 8/13 0.68 0.91

17 14 8/16 0.55 0.91

14 17 8/20 0.46 0.87

18 19 8/23 0.40 0.87

TABLE III

BLOCKCODESBASEDONCONSTRUCTION

2

k n R c q = R/C

5 12 8/13 0.65 0.95

10

14

8/16 0.54 0.92

IO 17 8/20 0.45 0.90

12 19 8/23 0.39 0.90

TABLE IV

BLOCKCODESBASEDONTANGANDBAHL'SCONSTRUCTION

d k n R C q = R/C

1 5 12 8/15 0.65 0.82

2 9 14 8/18 0.54 0.83

3 8 17 a/22 0.43 0.86

4 10 19 8/25 0.38 0.85

length-limited channel [6], [7],

log, IAnI

C= lim p.

n-tm

n

The efficiency q can be defined as the ratio of the information

rate R and the capacity C of the noiseless run-length-limited

channel, q = R/C.

In order to get some insight into the efficiency of the codes

based on Constructions 1 and 2 we have considered some exam-

ples. For m = 8 and for d = 1, 2, 3, 4 and k = 2d;. ., 20 we

have determined n in such a way that the information rate R was

maximized. In order to compare our two constructions to Tang

and Bahl’s method we have calculated the corresponding capaci-

ties C and efficiencies r~. The capacity of the noiseless run-

length-limited channels was calculated by a method given in [ 11.

Our results can be summarized as follows. For small values of

k, i.e., 2d $ k < 3d, Construction 2 is only slightly better than

Tang and Bahl’s method (approximately 5 percent), while the

efficiency of Construction 1 was worse (5 to 10 percent). For

larger values of k, however, Constructions 1 and 2 are clearly

better. For those values of k the gain of Construction 2 compared

to Tang and Bahl’s method is most significant for d =

1,2

(12 to

15 percent), while for d = 3,4 the gain is equal to 9 percent. For

large values of k, Constructions 1 and 2 have the same efficiency;

for the other values of k, Construction 2 has a better efficiency

than Construction 1. Tables II, III, and IV give the results for

m = 8 and d = 1, 2, 3, and 4; in order to limit the length of the

tables, we have restricted k and n to those values which maximize

the information rate R. We note that rates up to 95 percent of the

channel capacity can be achieved. On average we observe a slight

difference in the rates obtained by Constructions 1 and 2, ap-

proximately 5 percent in favor of Construction 2.

IV. CONCLUSION

Methods are described for the construction of run-

length-limited codes on the basis of sequences of fixed length.

Additional constraints on the maximum number of zeros at the

beginning and end of a sequence, a generalization of Tang and

154 IEEE TRANSACTIONS ON INFORMATION THEORY,

VOL. IT - 29, NO. 5, SEPTEMBER

1983

Bahl’s work, allow a more efficient merging of the sequences. For

short lengths in particular, our method yields better efficiencies redundancy R s over A,

than those of Tang and Bahl. R, = Qy$A ?~,H(f’i: QN),

E

REFERENCES

where H( P,,$ : QN) is the relative entropy between Ps and QN,

111

D. T. Tang and L. R. Bahl, “Block codes for a class of constrained

represented by

noiseless channels,” Inform.

Contr., vol.

17, pp. 436-461, 1970.

PI H. Kobayashi, “A survey of coding schemes for transmission of recording

of digital data,”

IEEE Trans. Comm. Tech.,

vol. COM-19, pp. 1087-l 100,

H(Pi: QN) = N-’ C P~(x)log(P{(x)/Q,(X)).

1971.

XEAN

[31 K. A. Immink, “Modulation systems for digital audio disks with optical

read out,” in

Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal

Henceforth, all the logarithms are of base 2. The interpretation of

Processing,

pp. 587-589, 1981.

the source matching approach, as an approximation to the

[41 M. G. Peichat and J. M. Geist, “Surprising properties of two-level band-

minimax solution over the set of uniquely decodable codes, is

width compaction codes,”

IEEE Trans. Corm-z.,

vol. COM-23, pp.

well demonstrated in [2, th. 21.

878-883, 1973.

[51 P. A. Franaszek, “Sequence-state methods for run-length-limited coding,”

A numerical approach to the source matching solution is based

IBMJ. Res. Dev.,

vol. 14, pp. 376-383, 1970.

on the next relationship [2, th. 31,

161

C. E. Shannon, “A mathematical theory of communication,”

Be// Syst.

Tech. J.,

vol. 27, pp. 379-423, 1948.

[71 C. V. Freiman and A. D. Wyner, “Optimum block codes for noiseless

input restricted channels,”

Inform.

Contr.,

vol. 7, pp. 398-415, 1964.

where Qc( x) is given by

Q;(x)

= J1,9”:‘-4 Me), for all x E AN. (2)

Note that the right side of (I) is merely the channel capacity

between the source parameter space A and the source output

space AN and hence the source matching problem can be con-

verted to the channel capacity problem. It is also known [3, p. 961

On The Source Matching Approach for Markov Sources

that the number of values of the source parameter 0 with nonzero

DONG H. LEE

probabilities achieving the channel capacity is no larger than the

cardinality of the source output space, which is (t + l)N for

alphabet AN. The capacity of a finite-input to finite-output

A&m&-The source matching approach is a universal noiseless source

channel can be obtained numerically by applying the Blahut-

coding scheme used to approximate the solution of minimax coding. A

Arimoto algorithm [4], [5]. Furthermore, if any sufficient statistic

numeric solution of the source matching approach is presented for the class

of AN is found, its application will reduce the computational

of binary first-order Markov sources.

complexity of the numerical solution since its cardinality is much

I.

INTRODUCTION

smaller than (t + l)N in most cases.

Noiseless source coding, as applied to sources whose statistics III. MARKET

SOURCES

are either partially or completely unknown, is called universal

noiseless source coding. Based on previous results, Davisson [l] In this section we consider the source matching approach for

formulated game-theoretic definitions for the problems of univer- the class of first-order Markov sources with binary alphabet

sal coding. According to him, the problem of minimax coding is A = (0, 1) It is assumed that Markov source is stationary with

stochastic matrix

to find a code minimizing the maximal redundancy over a given

class of sources. In [2], Davisson and Leon-Garcia presented a

coding scheme called “source matching” to approximate the

solutions of minimax codes over an arbitrary class of stationary [z ::]=[l;,eo l8”fJ

sources. A numerical solution for this source matching approach where B,, for i, j = 0,l represents the transition probability from

was obtained for the extended class of Bernoulli-like sources. the previous state i to the present state j. The stationary pdf

This correspondence presents a numerical solution for the source r = ( ro, a,) is uniquely expressed as

matching approach for the class of binary first-order Markov

sources. 7r =

(e,(eo

+ 6,)-l,

e,(e,

+

8,)-I).

II.

SOURCE MATCHING APPROACH

The domain A of the source parameter 0 = (0,, 19,) becomes the

In this section, the results of [2] are briefly reviewed for Cartesian product [0, I]

X

[0, I] over the closed interval [0, I].

reference. Let A = (0, I,. . , t} be a source alphabet set. We We now present a sufficient statistic defined on AN. Let n be

consider fixed-length to variable-length binary block encoding on the Hamming weight of I’s of the source message block

x

=

source message blocks x = (x, , x2,. . . , xN) on N-tuple alphabet (x,9. “9 xN). With n fixed, let u, for i = I, 2; . ., n be the num-

space AN. Let Pi(x) be the probability density function (pdf) of ber of runs of l’s with run length i. As an example, N = 6, n = 4,

x conditioned for each 6 taking values in some index set A. Let aI = 2, u2 = I, and us = a4 = 0 for x = (101011). The total

W be the set of all the possible pdf’s w on A. number of all the runs of I’s is upper-bounded as

Let A be the set of all the real-valued (t + l)N-dimensional

pdf’s QN. The source matching approach finds the minimax

Q, = t

a,<N-n+l.

I=1

Manuscript received July 6, 1982; revised January 11, 1983. This paper is a

With x, and xN, the first and last digits of x = (x,; . ., x,),

part of a dissertation submitted to the University of Maryland, College Park, in

respectively, (n, D,, , x, , xN) uniquely specifies Pi(x) and is an

partial fulfillment of the requirements for Ph.D. degree.

eligible sufficient statistic. Realizing that the probability of x with

The author is with Hewlett-Packard Laboratories, Palo Alto, CA 94304.

(n, D,,, 0, I) is identical to that with (iz, D,,, I, 0), the sufficient

001 S-9448/83/0900-0754$01 .OO 01983 IEEE