Content uploaded by Kees Schouhamer Immink
Author content
All content in this area was uploaded by Kees Schouhamer Immink on Apr 11, 2019
Content may be subject to copyright.
IEEE TRANSACTIONS ON INFORMATION THEORY,VOL. IT-29, NO. 5, SEPTEMBER 1983 751
TABLE I
THE(~~,&)= (~,~)FI~ED(T= I)NONCATASTROPHICCOWOLUTIONAL
ENCODERWITHMEMORY
M
= ~ANDMAXIMUMFREEDISTANCE
TOGETHERWITHSOMEBETTERTIME-VARYINGENCODERS'
T G$ GT G2* G3 G4* d, Nm L
1
3 2 2 1 3 I 2 4
2 E A C 7 3/2 3
3 B 7
3 3A 1c 00 7 4/3 8/3
OE 3B 00
03 2D 30
4 E9 CO 7 3/2 13/4
3B 70
OD EC
03 B7
5 3A7 000 7 e/5 13/5
OED 300
037 1co
OOD 3B0
003 2DC
‘d,
The fixed encoder has maximum free distance and is optimum in
the sense of smallest number of weight d, paths per time instant N, and
in the sense of smallest average number of information bit errors along
these paths I,, The encoders are specified by the
(n,,T, k,,T)
matrices G,*
of (7); the rows of G,* are written in hexadecimal form; e.g., the upper row
of G,f for
T
= 2 is specified as “E” and hence is the row [ 1 1 1 01.
where G,? for 0 < j 4 M* is the (Tk,) X (Tn 0) matrix
GjT (0) GjT+l(l) ... G,T+T-I(T- 1)
GjT- I (0)
G,,(l) "' G,T+T-#- l>
G;=
. __
G,,-T+,(O) G,;-T+z(~) ... Gj,(T- 1)
(7)
where by way of convention G,(u) = 0 for j > it4 and for j < 0
and where
M* is the smallest integer equal to or greater than
M/T, i.e.,
M* = [M/T].
(8)
Thus, every (n,, k,) periodic convolutional encoder with period
T and memory M can be considered to be an (n*,, ki) fixed
convolutional encoder with memory M* given by (8) and with
k,* = Tk, (94
n; = Tn,. (9b)
This equivalence permits upper bounds on code distance, as well
as software, developed for FCE’s to be applied to PCE(T)‘s.
III.
NEW CODES
We now report some positive results from a computer-aided
search for noncatastrophic PCE( T)‘s with (n,, k,) = (2,l) that
are superior to the best noncatastrophic FCE with the same
parameters n,, k,, and M. The search was concentrated on the
case M = 4 as this is the smallest M such that Heller’s upper
bound [4] on d,, namely d, = 8, is not achieved by any FCE.
The results of the search are given in the Table I. The T =
1
entry is the fixed convolutional encoder found by Oldenwalder
[5] that has maximum d,, namely dm = 7, and is optimum both
in the sense of minimum ypo and m the sense of minimum 1,.
For T > 1, the codes are time-varying but are specified by the
corresponding fixed encoding matrices Cl*, 0 < j < M*, defined
by (7).
In Table I, the encoders with period T = 2 and 3 were found
by an exhaustive search to be optimal in the sense of minimizing
N,; the codes for T = 4 and 5 were the best found in a heuristic
nonexhaustive search. The codes given in Table I for T = 2, 3, 4,
and 5 are all superior to the best fixed code (T = 1) both in the
sense of smaller N, and also in the sense of smaller 1,.
It is somewhat disappointing that no time-varying codes with
larger d, than the best fixed code were found. It seems likely
that no such superiority is possible for M = 4 when (n,, k,) =
(2, I); the next M for which such superiority is possible is M = 7
where Heller’s bound gives d, < 11 but the best fixed code has
d, = 10. It is encouraging, however, that periodic codes superior
to the best fixed codes could be found at all, as no such instances
could be found in- the prior literature.
ACKNOWLEDGMENT
The author is very grateful to Prof. James L. Massey who not
only suggested this work but also devoted much time and pa-
tience in supervising it.
[II
[21
[31
[41
[51
REFERENCES
J. L. Massey, “Error bounds for tree codes, trellis codes and convolutional
codes with encoding and decoding procedures,” in Coding
and Complexity,
G. Longo, Ed., ‘C.I.S.M. Courses and Lectures No. 216. New York:
Springer-Verlag, 1974, pp. l-57.
A. J. Viterbi, “Error bounds for convolutional codes and an asymptoti-
cally optimum decoding algorithm,”
IEEE Trans. Inform. Theory,
vol.
IT-13, pp. 260-269, Apr. 1967.
-, “Convolutional codes and their performance in communication
systems,”
IEEE Trans. Commun. Technol., vol.
COM-19, pp. 751-771,
Oct. 1971.
J. A. Heller, “Sequential decoding: Short constraint length convolutional
codes,” Jet Propulsion Lab., California Inst. of Technology, Pasadena,
Space Program Summary 37-54, vol. 3, pp. 171-174, Dec. 1968.
J. P. Oldenwalder, “Optimal decoding of convolutional codes,” Ph.D.
dissertation, School of Engineering and Applied Science, University of
California, Los Angeles, CA, 1970.
A Generalized Method for Encoding and Decoding
Run-Length-Limited Binary Sequences
G. F. M. BEENKER AND K. A. SCHOUHAMER IMMINK
Abstract-Many modulation systems used in magnetic and optical re-
cording are based on binary run-length-limited codes. We generalize the
concept of dk-limited sequences of length n introduced by Tang and Bahl
by imposing constraints on the maximum number of consecutive zeros at
the beginning and the end of the sequences. It is shown that the encoding
and decoding procedures are similar to those of Tang and Bahl. The
additional constraints allow a more efficient merging of the sequences. We
demonstrate two constructions of run-length-limited codes with merging
rules of increasing complexity and efficiency and compare them to Tang
and Bahl’s method.
I.
INTRODUCTION
Many baseband modulation systems applied in magnetic and
optical recording are based on binary run-length-limited codes
[l], [2], [3]. A string of bits is said to be run-length-limited if the
Manuscript received April 5, 1982; revised November 30, 1982. This work
was partially presented at the IEEE International Symposium on Information
Theory, Les Arcs, France, June 21-25, 1982.
The authors are with the Philips Research Laboratories, 5600 MD
Eindhoven, The Netherlands.
0018-9448/83,‘0900-075 l$Ol .OO 01983 IEEE
152 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. IT - 29, NO. 5, SEPTEMBER 1983
number of consecutive zeros between adjacent ones is bounded
between a certain minimum and a certain maximum value. The
upper run-length constraint guarantees a transition within a
specified time interval needed for clock regeneration at the re-
ceiver. The lower run-length constraint is imposed to reduce the
intersymbol interference. The latter constraint appears to have a
bearing on the spectral properties of the sequences [4].
In this cvrrespondence we consider an encoding and decoding
procedure for a special class of run-length-limited codes. The
constraints we consider can be defined as followed. Let n, d, k, I,
and r be given, where 1 6 k and r < k. A binary sequence of
length n is called a dklr-sequence if it satisfies the following
constraints:
a) d-constraint: any two ones are separated by a run of at least
d consecutive zeros;
b) k-constraint: any run of consecutive zeros has a maximum
length k;
c) /-constraint: the number of consecutive leading zeros of the
sequence is at most 1;
d) r-constraint: the number of consecutive zeros at the end of
the sequence is at most r.
A sequence satisfying the d- and k-constraints is called a dk-
sequence.
Given certain desired run-length constraints, it is not trivial
how to map uniquely the input data stream onto the encoded
output data stream. A systematic design procedure on the basis
of fixed length sequences has been given by Tang and Bahl [l].
Their method is based on mapping dk-sequences of length n onto
consecutive integers and vice versa. In this correspondence we
intend to generalize their results to dklr-sequences. We are also
going to show that the application of dklr-sequences enables the
sequences of length n to be merged efficiently without violation
of the d- and k-constraints at the boundaries.
We assume r > d. Theorems 1 and 2 of this correspondence
remain valid for r < d and Theorem 3 can be generalized accord-
ingly. The proofs, however, are less elegant, the reason being that
for r < d there are no dklr-sequences (x,,- , , . . , x0) having a one
at positions j where r < j < d.
II. ENCODING AND DECODING
In this section we consider a way of mapping the set of all
dklr-sequences onto a set of consecutive integers and vice versa.
Our results are similar to those obtained by Tang and Bahl for
dk-sequences.
Let A,, be the set of all dklr-sequences of length n. A, can be
embedded in a larger set 6?,, consisting of the all-zero sequence of
length n and of all binary sequences of length n satisfying the d- ,
k- , and r-constraints where the number of consecutive leading
zeros is allowed to be greater than k.
The set a,, can be ordered lexicographically as follows: if
x = ( x!,~ , , . , x,,) and y = (y,,_ , , . ’ . , yO) are elements of @n
theny is called less than x, y -+ x, if there exists an i, 0 < i < n,
such that y, < x, and x, ,= y, for i < j < n. The position of x in
the lexicographical ordermg of a,,, is denoted by r(x); i.e., r(x) is
the number of ally in @n withy + x. Consequently r(0) = 0.
For the sake of convenience we introduce the residual of a
vector. Lety = (y,,-,;..,~a) E a,,,, y * 0, and let t be such that
yt = 1 and y, = 0 if t < j < n. Then the residual of y, res( y), is
defined as follows:
res( y): = y - A,, where At, =
1,
ifi=
t;
0, elsewhere,
res (0): = 0.
It can easily be seen that y E &,, implies res ( y) E a,,,. The
following observation is basic to the proof of Theorem 1. Let
X, u E W,, and assume that x, = u, = 0
(t
< j < n) and x, = nt
= 1 (0 <
t
< n). Then it is not difficult to show that r(n) -
r(x) = r(res(u)) - r(res(x)).
Let N,.(i), i > 0, be the number of dklr-sequences with I = k of
length i and let N,.(O) = 1.
Theorem 1: Let x = (x,~,;.., xc,) E @,. Then
n-l
r(x) = c x,W(j).
j=O
Proof: Let the nonzero coordinates of x be indexed by
i, < i, < . . < i,, i.e., x, = 1 if and only if i E (i,; . ., iq}. Let
I( be the smallest element of @,, with the property that ulq = 1.
Then it is not difficult to see that the second 1 of I( occurs at
position i, - k - 1, if i, - k - 1 2 0 (otherwise u has only one
1, if r > i,, or also u0 = 1 if r < i4). Here we recall that the
all-zero sequence of length n is a dklr-sequence if and only if
n < min(l, r}. Lete(i,, r) = 1 ifr > i,,andc(i,, r) = Oifr < i,.
Then we obtain
r(u) - r(res (u)) = the number of dklr-sequences with I = k of
length i, with their leftmost 1 at position j where
max(O, i, - k - l} < j < i, + c(i,, r) = N,.(i,).
Furthermore, on the basis of the above mentioned observation it
holds that r(x) = r(u) + r(x) -r(u)
= r(u) + r(res(x)) - r(res(u))
= r(res(x)) + Nr(iq).
The theorem then follows by induction. Q.E.D.
We have found a simple method for mapping the elements of
&,, onto consecutive integers. For practical applications we need a
mapping of the elements of A, onto the set (0, 1, . , ]An] - l},
where ]A,,] is the cardinality of A,,. Obviously the set A, consists
of the lA,1l largest elements of @,. In addition it is clear that the
number a of elements of &,, that are smaller than all the elements
of A,, is equal to r(u); i.e., cr = r(a), where a is the smallest
element of A,,. In this way we have proved the following theorem.
Theorem 2: The transformation t: A,, + N
U
(0) defined by
t(x)
= r(x) - (Y for all x E A,, is a one-to-one mapping from A,,
onto the set (0, 1, . . . , lAnl - l} which preserves the ordering of
A,,, i.e., x ~0 y if and only if t(x) < t(y).
The number (Y can also be expressed in another way. In order
to do so we define N,!)(j), j > 0, to be the number of dk-
sequences of length j with their leftmost element equal to 1 and
satisfying the r-constraint, NrO(0): = 1. It is not difficult to show
that n-l-l n-l-l
a = c N:(j) + 1 = c NY(j).
j=l j=O
The numbers N,(j) and N!‘(j) are easily computed. They can be
found by a straightforward computer search. A more sophisti-
cated approach to finding these numbers can be based on the
finite state transition matrix corresponding to the dklr-sequences
]51.
The conversion from integers to dklr-sequences of length n is
aho
analogous to Tang and Bahl’s method and can be carried out
as follows. Let T,, . . ., T,- , be integers defined by
T = the number of elements y in W, smaller than the smallest
element u in & with u, = 1 and u, = 0 for j > i,
i.e.,
T, = i N:(j) + 1 = i N:(j).
j=l j=O
IEEE TRANSACTIONS ON INFORMATION THEORY,VOL. IT-29, NO. 5, SEPTEMBER 1983 753
TABLE I
MERGINGOF~~~T-SEQUENCESWHERE OJ STANDS
FOR~CONSECUTIVE
ZEROS
$3 t
Merging Bits
s+t+dik+l Od
s+t+d>k+l
ifs<d o”-“lo”- I
ifs > d 1od-
From the definition and our assumption r 2 d it immediately
follows that T, < T, < . . . < T,- ,. These integers are used for
mapping consecutive integers onto dklr-sequences of length n as
shown in the following theorem.
Theorem 3: Let x = (x,- ,; . ., x0) be a dk/r-sequence of
length n. Then a) x, = 1 and xj = 0 for
t
< j < n e T, < r(x) <
T,, , Furthermore, if x, = 1 and x, = 0 for
t
< j < n, then b)
T-,_, d r(x) - N,(t) < Ted.
Proof: a) This statement follows from the definition of 7;. b)
Let x be a dklr-sequence of length n with x, = 1 and x, = 0 for
t
cj < n. Then Trek-, < r(res(x)) < Tr-d. Hence b) follows,
since r(x) -
r(res(x)) = N,(t).
Q.E.D.
The conversion from integers to dk-sequences can therefore
also be generalized to the conversion from. integers to dklr-
sequences. The following simple encoding algorithm for dklr-
sequences can be derived from this theorem. Given an integer I in
the set R = {r(x)lx E A,} (1 = 0 if x = 0), we first locate the
largest possible
t,
0 <
t
<
n,
such that T < I < I;+, and we
make x, = 1. Subtracting the contribution of x, in I, we get a new
integer I -
N,(t).
Theorem 3 can be used again to find the next
nonzero component of x. The second part of Theorem 3 assures
us that x, will be followed by at least d, but no more than k zeros.
III. THE EFFICIENCY OF dklr-SEQUENCES
In modulation systems the dklr-sequences of length n
cannot in
general be cascaded without violating the dk-constraint at
the
boundaries. Inserting a number /3 of merging bits between adja-
cent n-sequences makes it possible to preserve the & and k-con-
straints for the cascaded output sequence. The dk-sequences need
p = d + 2 merging bits [l], whereas only /3 = d merging bits are
required for dklr-sequences, provided that the parameters J and r
are suitably chosen. Hence this method is more efficient, espe-
cially for small values of n. We shall now demonstrate two
constructions of codes with merging rules of increasing complex-
ity and efficiency.
Construction I: Choose d, k, r, and n such that r + d < k. Let
I= k - d - r and p = d. Then the dklr-sequences of length n
can be cascaded without violating the d- and k-constraints if the
merging bits are all set to zero.
Construction 2: Choose d, k, and n such that 2d - 1 < k. Let
r = I = k - d and /? = d. Then the dklr-sequences of length n
can be cascaded without violating the d- and k-constraints if the
merging bits are determined by the following rules. Let an
n-sequence end with a run of s zeros (s < r) while the next
n-sequence start with
t (t
< f) leading zeros. Table I shows the
merging rule for the /3 = d merging bits.
The number m of data bits that can be represented uniquely by
a dk[r-sequence of length n is given simply by
m = [log2 IA,I~ f
where Lx] is the greatest integer not greater than x. The ratio R
of the number of data bits and the number of needed channel
bits is called the information rate of the code. For example, the
information rate of the codes based on the two above-mentioned
constructions equals R = m/(n + d). The asymptotic informa-
tion rate is the capacity C of Shannon’s discrete noiseless run-
TABLE II
BLOCKCODESBASEDONCONSTRUCTION
1
k n R C q = R/C
7 12 8/13 0.68 0.91
17 14 8/16 0.55 0.91
14 17 8/20 0.46 0.87
18 19 8/23 0.40 0.87
TABLE III
BLOCKCODESBASEDONCONSTRUCTION
2
k n R c q = R/C
5 12 8/13 0.65 0.95
10
14
8/16 0.54 0.92
IO 17 8/20 0.45 0.90
12 19 8/23 0.39 0.90
TABLE IV
BLOCKCODESBASEDONTANGANDBAHL'SCONSTRUCTION
d k n R C q = R/C
1 5 12 8/15 0.65 0.82
2 9 14 8/18 0.54 0.83
3 8 17 a/22 0.43 0.86
4 10 19 8/25 0.38 0.85
length-limited channel [6], [7],
log, IAnI
C= lim p.
n-tm
n
The efficiency q can be defined as the ratio of the information
rate R and the capacity C of the noiseless run-length-limited
channel, q = R/C.
In order to get some insight into the efficiency of the codes
based on Constructions 1 and 2 we have considered some exam-
ples. For m = 8 and for d = 1, 2, 3, 4 and k = 2d;. ., 20 we
have determined n in such a way that the information rate R was
maximized. In order to compare our two constructions to Tang
and Bahl’s method we have calculated the corresponding capaci-
ties C and efficiencies r~. The capacity of the noiseless run-
length-limited channels was calculated by a method given in [ 11.
Our results can be summarized as follows. For small values of
k, i.e., 2d $ k < 3d, Construction 2 is only slightly better than
Tang and Bahl’s method (approximately 5 percent), while the
efficiency of Construction 1 was worse (5 to 10 percent). For
larger values of k, however, Constructions 1 and 2 are clearly
better. For those values of k the gain of Construction 2 compared
to Tang and Bahl’s method is most significant for d =
1,2
(12 to
15 percent), while for d = 3,4 the gain is equal to 9 percent. For
large values of k, Constructions 1 and 2 have the same efficiency;
for the other values of k, Construction 2 has a better efficiency
than Construction 1. Tables II, III, and IV give the results for
m = 8 and d = 1, 2, 3, and 4; in order to limit the length of the
tables, we have restricted k and n to those values which maximize
the information rate R. We note that rates up to 95 percent of the
channel capacity can be achieved. On average we observe a slight
difference in the rates obtained by Constructions 1 and 2, ap-
proximately 5 percent in favor of Construction 2.
IV. CONCLUSION
Methods are described for the construction of run-
length-limited codes on the basis of sequences of fixed length.
Additional constraints on the maximum number of zeros at the
beginning and end of a sequence, a generalization of Tang and
154 IEEE TRANSACTIONS ON INFORMATION THEORY,
VOL. IT - 29, NO. 5, SEPTEMBER
1983
Bahl’s work, allow a more efficient merging of the sequences. For
short lengths in particular, our method yields better efficiencies redundancy R s over A,
than those of Tang and Bahl. R, = Qy$A ?~,H(f’i: QN),
E
REFERENCES
where H( P,,$ : QN) is the relative entropy between Ps and QN,
111
D. T. Tang and L. R. Bahl, “Block codes for a class of constrained
represented by
noiseless channels,” Inform.
Contr., vol.
17, pp. 436-461, 1970.
PI H. Kobayashi, “A survey of coding schemes for transmission of recording
of digital data,”
IEEE Trans. Comm. Tech.,
vol. COM-19, pp. 1087-l 100,
H(Pi: QN) = N-’ C P~(x)log(P{(x)/Q,(X)).
1971.
XEAN
[31 K. A. Immink, “Modulation systems for digital audio disks with optical
read out,” in
Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal
Henceforth, all the logarithms are of base 2. The interpretation of
Processing,
pp. 587-589, 1981.
the source matching approach, as an approximation to the
[41 M. G. Peichat and J. M. Geist, “Surprising properties of two-level band-
minimax solution over the set of uniquely decodable codes, is
width compaction codes,”
IEEE Trans. Corm-z.,
vol. COM-23, pp.
well demonstrated in [2, th. 21.
878-883, 1973.
[51 P. A. Franaszek, “Sequence-state methods for run-length-limited coding,”
A numerical approach to the source matching solution is based
IBMJ. Res. Dev.,
vol. 14, pp. 376-383, 1970.
on the next relationship [2, th. 31,
161
C. E. Shannon, “A mathematical theory of communication,”
Be// Syst.
Tech. J.,
vol. 27, pp. 379-423, 1948.
[71 C. V. Freiman and A. D. Wyner, “Optimum block codes for noiseless
input restricted channels,”
Inform.
Contr.,
vol. 7, pp. 398-415, 1964.
where Qc( x) is given by
Q;(x)
= J1,9”:‘-4 Me), for all x E AN. (2)
Note that the right side of (I) is merely the channel capacity
between the source parameter space A and the source output
space AN and hence the source matching problem can be con-
verted to the channel capacity problem. It is also known [3, p. 961
On The Source Matching Approach for Markov Sources
that the number of values of the source parameter 0 with nonzero
DONG H. LEE
probabilities achieving the channel capacity is no larger than the
cardinality of the source output space, which is (t + l)N for
alphabet AN. The capacity of a finite-input to finite-output
A&m&-The source matching approach is a universal noiseless source
channel can be obtained numerically by applying the Blahut-
coding scheme used to approximate the solution of minimax coding. A
Arimoto algorithm [4], [5]. Furthermore, if any sufficient statistic
numeric solution of the source matching approach is presented for the class
of AN is found, its application will reduce the computational
of binary first-order Markov sources.
complexity of the numerical solution since its cardinality is much
I.
INTRODUCTION
smaller than (t + l)N in most cases.
Noiseless source coding, as applied to sources whose statistics III. MARKET
SOURCES
are either partially or completely unknown, is called universal
noiseless source coding. Based on previous results, Davisson [l] In this section we consider the source matching approach for
formulated game-theoretic definitions for the problems of univer- the class of first-order Markov sources with binary alphabet
sal coding. According to him, the problem of minimax coding is A = (0, 1) It is assumed that Markov source is stationary with
stochastic matrix
to find a code minimizing the maximal redundancy over a given
class of sources. In [2], Davisson and Leon-Garcia presented a
coding scheme called “source matching” to approximate the
solutions of minimax codes over an arbitrary class of stationary [z ::]=[l;,eo l8”fJ
sources. A numerical solution for this source matching approach where B,, for i, j = 0,l represents the transition probability from
was obtained for the extended class of Bernoulli-like sources. the previous state i to the present state j. The stationary pdf
This correspondence presents a numerical solution for the source r = ( ro, a,) is uniquely expressed as
matching approach for the class of binary first-order Markov
sources. 7r =
(e,(eo
+ 6,)-l,
e,(e,
+
8,)-I).
II.
SOURCE MATCHING APPROACH
The domain A of the source parameter 0 = (0,, 19,) becomes the
In this section, the results of [2] are briefly reviewed for Cartesian product [0, I]
X
[0, I] over the closed interval [0, I].
reference. Let A = (0, I,. . , t} be a source alphabet set. We We now present a sufficient statistic defined on AN. Let n be
consider fixed-length to variable-length binary block encoding on the Hamming weight of I’s of the source message block
x
=
source message blocks x = (x, , x2,. . . , xN) on N-tuple alphabet (x,9. “9 xN). With n fixed, let u, for i = I, 2; . ., n be the num-
space AN. Let Pi(x) be the probability density function (pdf) of ber of runs of l’s with run length i. As an example, N = 6, n = 4,
x conditioned for each 6 taking values in some index set A. Let aI = 2, u2 = I, and us = a4 = 0 for x = (101011). The total
W be the set of all the possible pdf’s w on A. number of all the runs of I’s is upper-bounded as
Let A be the set of all the real-valued (t + l)N-dimensional
pdf’s QN. The source matching approach finds the minimax
Q, = t
a,<N-n+l.
I=1
Manuscript received July 6, 1982; revised January 11, 1983. This paper is a
With x, and xN, the first and last digits of x = (x,; . ., x,),
part of a dissertation submitted to the University of Maryland, College Park, in
respectively, (n, D,, , x, , xN) uniquely specifies Pi(x) and is an
partial fulfillment of the requirements for Ph.D. degree.
eligible sufficient statistic. Realizing that the probability of x with
The author is with Hewlett-Packard Laboratories, Palo Alto, CA 94304.
(n, D,,, 0, I) is identical to that with (iz, D,,, I, 0), the sufficient
001 S-9448/83/0900-0754$01 .OO 01983 IEEE