Content uploaded by Robert C. Tausworthe

Author content

All content in this area was uploaded by Robert C. Tausworthe on Sep 07, 2015

Content may be subject to copyright.

Random Numbers Generated by Linear

Recurrence Modulo Two

By Robert C. Tausworthe

1. Introduction. Many situations arise in various fields of interest for which the

mathematical model utilizes a random sequence of numbers, events, or both. In

many of these applications it is often extremely advantageous to generate, by some

deterministic means, a sequence which appears to be random, even if, upon closer

and longer observation, certain regularities become evident. For example, electronic

computer programs for generating random numbers to be used in Monte Carlo

experiments have proved extremely useful. This article describes a random number

generator of this type with several outstanding properties. The numbers are gener-

ated by modulo 2 linear recurrence techniques long used to generate binary codes

for communications.

2. Linear Recurrence Relations over GF(2). Let a = ¡a*} be the sequence

of O's and l's generated by the linear recursion relation

ak = ciak-i + c&k-i + • • • + cnak-n (mod 2)

for any given set of integers a (i — 1, 2, • • • , n), each having the value 0 or 1. We,

of course, require c„ = 1, and say that the sequence has degree n.

From the recursion, ak is determined solely (for fixed d) by the n-tuple

(a*_i, a*_2, • • • , Ok-n) of terms preceding it. Similarly, ak+i is a function solely of

(a* , a*_i, • • • , a*_B+i). Each such n-tuple thus has a unique successor governed by

the recursion formula, and the period of a is clearly the same as the period with

which an n-tuple repeats. The period p of a linear recurring sequence obviously can-

not be greater than 2" — 1, for the n-tuple (0, 0, • • • , 0) is always followed by

(0, 0, • • • , 0). The necessary and sufficient condition that p = 2" — 1 is that the

polynomial

fix) = 1 + cix + CiX2 + ■ • • + xn

be primitive over GF(2) [1], [2].

We shall assume in the remainder of this article that f(x) is a primitive nth

degree polynomial over GF(2) ; the sequence a is then a maximal-length linearly re-

curring sequence modulo 2. These sequences have been studied and used as codes in

communications and information-theoretic studies [3], [4]. The properties of interest

to us at present are the following [1], [2] :

(1) ¿„-E+I-JT*.

k— 1 ¿

(2) For every distinct set of (0, 1) integers sx, s2, • • • , s„ , not all zero, there

Received July 10, 1964. This paper presents the results of one phase of research carried at

the Jet Propulsion Laboratory, California Institute of Technology, under Contract No. NAS

7-100, sponsored by the National Aeronautics and Space Administration.

201

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

202 ROBERT C. TAUSWORTHE

exists a unique integer n (0 í n Í p - 1) such that for every k, Si«*_i + s&k-2

+ ■ ■ ■ + snak-n = ak+v (mod 2). This is often referred to as the "cycle-and-add"

property.

(3) Every nonzero (0, 1) binary n-vector (et, c2, • • • , e„) occurs exactly once

per period as n consecutive binary digits in a.

Note that properties (1) and (3) follow directly from the fact that each possible

nonzero binary n-tuple (a*_i, a*_2, ■ • • , a*_„) must occur exactly once per cycle if

a has period p = 2" — 1.

We shall, in what follows, find it convenient to use a slightly different version

of the sequence a. Let us define

a*- (-1)*»- 1 -2a».

Under this transformation, we see that, if ak takes on the values 0 and 1, then ak

takes the values +1 and —1, respectively. The properties (1), (2), and (3) are

then transformed into

(i') E«*=-i-

t-i

(2') For every distinct set of (0, 1) integers si, ••■«„, not all zero, there exists

a unique integer v (0 ^ v ^ p — 1) such that a'k~ia'k-2 ■ ■ • a*-„ = £**+„.

(3') Every ±1 binary n-vector («i, e2, • • • , e„), except the all-ones vector,

occurs exactly once per period as n consecutive elements in a.

3. The Boolean Transform. Let gix) be a ±l-valued Boolean function of

(0, 1 ) variables xi, x2, • ■ ■ ,xn. For any s = (si, sa, • ■ • , sn), 8,: = 0 or 1, define

¿(s, x) = 2-n/2(-l),in+-+,"*\

These 2" functions of x, the Rademacher-Walsh functions [5], form an ortho-

normal basis for 2n-space. Relative to this basis, #(x) has components (r(s) given

by Gis) =r'iE^(s,i).

X

That is, G(s) is the projection of g(x) on </>(s, x), normalized so that

Eg2(s) = i.

Similarly, we have

gix) =2n/2EG(s)<Ks,x).

■

Consider the effect of setting xt = a*_, in gix). As a function of k, a binary

±1 -sequence [7*] = y is generated :

7* = ZG(s)(-l)",4-l+-+,",i-'

= XI ö(s)afciiaii2 • * • ctl-n .

By ( 2; ), we now have the fourth property basic to our analysis :

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

RANDOM NUMBERS GENERATED BY LINEAR RECURRENCE MODULO TWO 203

(4) yk = G(0) + E Gis)*k+m ,

where the mapping vis) of all binary nonzero n-vectors onto {0, 1, 2, • • • , p — 1}

is one-to-one.

4. Random Number Generation. Let a = \ak\ be the (0, 1) sequence

generated by an nth degree maximal-length linear recurrence modulo 2, as described

previously, and define a set of numbers of the form

yk = 0-a9i+r_ia9/t+r-2 ■ ■ ■ aqk+r-L (base 2),

where r is a randomly chosen integer, 0 ^ r | 2" - 1 and L ^ n. That is, yk is the

binary expansion of a number whose binary representation is L consecutive digits

in a; successive yk are spaced q digits apart. For reasons essential to the analysis, we

restrict q ^ L, and iq, 2" — 1) = 1.

We can also express yk by

L

yk = ¿_^2 aqk+r-t.

i-i

Such numbers always lie in the interval 0 < yk < 1. Because of condition (2), the

randomness of the choice of r is equivalent to the statement that the initial value

2/0 is a random choice.

5. Analysis of the Generator. We shall find it convenient to work with a

transformed set of numbers wk rather than the yk . Specifically, let a = {ak} be the

±1 sequence corresponding to a = \aK\, and define

L

Wk = E 2~*aqk+T-i .

¡-i

We see that yk and wk are related by

wk = 1 - 2~L - 2yk .

There is thus an easy translation between wk and yk .

We generally may assume, merely from the applications to which we wish to suit

the numbers, that n is moderately large, so that the numbers yn and wn are ex-

tremely numerous. For example, if n = 35, there are 3.43 X 10 of them. We wish

to consider only a portion of the total number of these, say N of them, and to dis-

cover, for moderately large N, how these are distributed.

6. Correlation Properties. The mean value of wk is easily found as

■j J>-1 -, L p—l

Eiwk) = - E Wi =- E 2~' E Ctqk+r-t

P r-0 p (=1 r=0

= -2 \1 - 2-"/ '

a number very nearly equal to zero for large n.

Define the sample autocorrelation function Ä(m) of wk by

License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

204 ROBERT C. TAUSWORTHE

1 "

Rim) = Tr= EwtWt+m.

N t-i

The expected value of Rim) is the true autocorrelation function J(m) of the

process,

Rim) = #[Ä(m)],

and the value Ä(0) is the mean-squared value of the process Wk .

, p—\ . L L n-1

Ä(0) =-¿,Wt =-LL2 "E «jt+r-i Otgi+r-u •

P r-0 p (-1 u-1 r-0 •

The last sum is — 1 if t j* u, and p if t = u, by (2'). Hence

This shows that w* has essentially the same variance as a uniformly distributed

process.

Now consider Rim), m ?* 0. First, its mean value is

JE. -k- -k- P-l

Ctqk+T-tCtq(k+m)+r—u

Rim) = E[Rim)} = -L E E E 2-(,+u) fc ,

pA7 *-l <-l u~l r-0

1 L L t^

= - E E 2~ ' E «, Ctr+qm+t-u •

P (-1 u-1 r-0

The last sum is again —1 by (2') unless qm + t — u is a multiple of p. Obviously,

qm — L + 1 á P + í - u ^ qm + L — 1.

Hence, if g ^ L and m ^ (p — L)/g, we see that

0 < qm + < — u < p,

bo qm + t — u can never be a multiple of p. These conditions, mentioned earlier,

shall now be assumed as one of our hypotheses. The mean value of fí(m) is then

Rim) = _I(i_ 2~L)2

V

(i - 2-y

(1 - 2~») •

The mean behaviour of the process shows essentially no correlation between wk

and wk+m for any nonzero integer m less in magnitude than (p — L)/q.

The sample autocorrelation function is a function of r, and is itself a random

process ; its mean-squared value for m ?± 0 is

E[R\m)] = Ê E E E 2-«+-+<+'V<u,> ,

i-1 u-1 i-1 j'-l

where /xiuo is defined by

RANDOM NUMBERS GENERATED BY LINEAR RECURRENCE MODULO TWO 205

1 if y p-i

P-txii) = —¡TxT 2-1 2-, 2-, Olr+qk-tetr+(k+m)q-uCtr+lq-i Ot,+(l+m)q-i .

plSc *=1 i=l r=0

Now since we have restricted q ^ L and 1 ^ m ^ (p — L)/q, there exist Vi and

t>2 such that

OCr+qk—tOtr+kq+mq—u = <*r-ft>i ;

^r+lg—î'ttr+ig+mg—j == u¡r+i>2 *

For fixed values of t, u, i, and j, there is at most one value of I for each k such that

vi = vi, since (g, p) = 1. Hence

^■^[%ID -AT2]

produces the result, for m ^ 0,

£[*(*)]* (1-2-^(^1-1),

and the value of the variance of Rim) is likewise bounded,

*«rtlía-rv(í + l-i-»)<¿(i+í).

This indicates that the deviation of the sample autocorrelation function from its

mean value is very small, and decreases inversely proportional to N.

7. The Distribution Properties. We have shown that wk (and, consequently, yk)

has essentially the same mean and variance as a uniform distribution. Now consider

actual distributions of N values of yk on (0, 1). To do this, we consider an arbitrary

interval in (0, 1) and observe what percentage of the N values of yk lie in this range.

Since we are considering binary expansions of numbers, intervals of width 2~d

are most conveniently considered, and these will surely be sufficient to our needs.

This is done efficiently by considering the first d positions of the vectors representing

yk for k = 1, 2, • • • , N, and count the number of these having a specified pattern.

This is equivalent to forming a Boolean function on the first d positions of yk,

whose value is, say, — 1 if yk has this initial pattern and +1 otherwise.

More specifically, let (ei, e2, • • • , e,¡) be the initial pattern of ones and zeros

we seek as a prefix to yk. Then define the (±1) Boolean function gix) by

/ » Í—1 if *i = Ci, »2 = e¡, ' • • , Xd = et,

gK ' == \l otherwise.

The relative number of times T that a number yk takes on the form 0-eie2 • • •

edXx • ■ ■ x, and thus falls in the specified interval, is

where yk has the value

yk = G(0) + E Gis)akq+r+v{.)

206 ROBERT C. TAUSWORTHE

by the Boolean transform. The expected value of t is

T = E[f] = - £ f

P r-0

-i[i-(4¿) «•>+£? «4

But it is easy to see from its definition that

0(0) = Eg(b),

and that

G(0) = 2_nEû(x) = 2_n(2n - 2-2"-")

= 1 - 2-d+1.

Hence, we have

r-J[i-(i + í)(i-r«») + «a>]

,-d . 1

2 + ¿ to(o) - 11.

Thus, the y* are equidistributed in the mean.

The variance about this mean can also be bounded. First, however, we compute

- ¡C E E 7*7! = E E E E G(S)G(U) i E «ril • • • «rW-r.-l ■ ■ ■ «».-»

P r-0 *-l 1-1 *-l )-l » u P r-1

using t = qil — k).Iîs ?* 0, and u/O, then there exist integers Vi and v2 such that

•l «n

ûr-1 • • • «r-n = OV+,1 ,

«1 Un

«r+l-1 " ' ' «r+i_n = 0!r+»2 ,

and for each k there is at most one Z such that vi = y2. Using this fact and

the Schwartz inequality, we see

-f,tt 7*7* ú n2{g\o) (i +1) - i) + ív(i +1).

p r=o *-i i-i 1 \ p) p) \ p)

This calculation then places a bound on the variance of T,

var [fl - \ E ÍL £ t. - (l + -) 0(0) + - o(0)

s J{-B + *»>] Kl + ?) + I(1+09<0,e<0) + » 0 + i)} '

If the negative terms are omitted, the inequality is stronger,

RANDOM NUMBERS GENERATED BY LINEAR RECURRENCE MODULO TWO 207

„*<^+^+aaffif£^<j(1+í)^+í)

and, again, the deviation from expected behavior decreases as N grows larger.

8. Higher-Order Distributions. We have seen that the numbers wk (or yk)

are "white" and uniformly distributed. We now consider the distribution of

iyk , yk-i%, • • • , yk-iM) where 0 = h < l3 < • • • < lM . It can be shown that this

distribution is far from uniform if qilM + 1) > n. For qilM + 1) ^ n, however,

the distribution is uniform over the unit M-cube. To show this is the case, we shall

count the relative number of times iyk , yk-h , • • • , yk-iM) lies in an arbitrary given

2~dl X • • • X 2~d" interval. Let the initial positions in the binary expansion of

yk+i( be O-d', e2', • • • , ej, for i = 1,2, • ■ • , M, and define ^(x) as follows:

a(x\ = /_1 if Xl«+j = e>' îori= 1> 2, ■■• ,M and j = 1, 2, • • • , d,:,

"1+1 otherwise.

Now since qily + 1) ^ n, if we let the Boolean function variables be

Xt = aqk+r—t ,

then we can use the transformed equation

yk = G(0) + E Gis)akq+r+v(t)

¿Te-

to reveal the desired properties. The previous analysis is valid, with d = dx + d2

+ • • • + dM . Therefore, the relative number of times f that iyk, yk_i, • • • , yk-iu)

lies in the specified interval has mean value

T = Eit) = (l + -) 2~ldl+-+dM) + 1 [giO) - 1]

and the variance about this mean is bounded by

~<*<iMÖ+i)-

8. Summary. The conclusions reached by this analysis are stated in the following

Theorem. If \ak) is a (0, 1) binary sequence generated by annthdegree maximal-

length linear recursion relation modulo 2, if for (ç, 2" — 1) = 1 and q ^ L, yk =

0 • akq-iakq-2 ■ ■ • aqk-L is the binary expansion of a real positive number in the interval

(0, 1), and if wk is a real number in the interval ( —1, +1) related to yk by wk =

1 — 2yk — 2~L, then, averaged over all possible iassumed equally likely) initial values

yo iorwo):

1. The mean value p. of the sequence wk

and variance a

208 ROBERT C. TAUSWORTHE

3 L3 \ 1 — 2-** y 1 - 2-" \l-2-»yj

1

~3"

2. TÄc sample autocorrelation function, defined by

Wk wk+„

i\ t-i

has as its mean value Rim), given by

. *<«>--r(fM£) '

«0

for nonzero integral values of \m\ less than (p — L)/q. The variance of Rim) about

Rim) is bounded by

Rim) = 1 £

N t-i

3. The relative number of times f that yk falls in the interval for which the first d

positions of the binary expansion are fixed, i.e., a neighborhood of length 2~d in the

interval (0, 1), has mean

T = E[f] - 2~d [l + jpLy] + \ |,(0) - 1] (^L_)

«2_d

for any number N of points yk. The variance of t is bounded by

™f|íl<í[1 + (2^T)][s + ^«]wS-

4. The relative number of times f that iyk, yk-it , • • • , yk-iM) foils in the interval

of the unit M-cube for which the first d, positions of the binary expansion of yk+i( are

fixed, i.e., in a 2~dl X 2_dj X • • • X 2~ u interval in the unit M-cube, has mean value

T = Bit) = 2-M»+""M*> (l + 2^Tl) + 2--1 (^r^)

Ä 2-(di+<ia+---H«)

/or any number N of points (y*, 2/t-i, , • • • , yk-iM), provided 0 < U < • • • < l\

< n/q — 1. The variance of f is then bounded by

var m<í[s + 2^n][1 + 2^n>¿r

9. Primitive Polynomials. In order to implement the generator, it is necessary

to find a primitive polynomial fix) over GF(2). A complete tabulation up through

degree 34 appears in Peterson [6]. The form easiest to implement is usually one in

RANDOM NUMBERS GENERATED BY LINEAR RECURRENCE MODULO TWO 209

which the recursion relation has fewest terms. Golomb et al. [7] have found primitive

trinomials for most degrees through degree 36.

Watson [8] has published a table giving one primitive polynomial for each degree

up to 100. A degree 35 polynomial fix) = x3b + x2 + 1 is very useful for generating

numbers on an ibm-7094, whose numerical register contains 35 digits. In this case

the period p = 236 — 1 is relatively prime to 35, so q may be set equal to 35 for

maximal precision iL = n) numbers. Preliminary experimental results indicate

that the bounds given here are indeed valid for arbitrary sample sequences yk .

Additional tests have shown that with L = q = 17, the pair iyk, yk+ï) is uniform

on the unit square.

Jet Propulsion Laboratory

California Institute of Technology

Pasadena, California

1. S. W. Golomb, Sequences with Randomness Properties, Martin Co., Baltimore, Md., 1955.

2. Neal Zierlee, "Linear recurring sequences," J. Soc. Indust. Appl. Math., v. 7, 1959,

pp. 31-48. MR 21 #781.

3. L. Baumert, et al., Coding theory and its Applications to Communications Systems,

Report 32-167, Jet Propulsion Laboratory, Pasadena, Calif., 1961.

4. R. C. Titsworth & L. R. Welch, Modulation by Random and Pseudo-Random Sequences,

Report 20-387, Jet Propulsion Laboratory, Pasadena, Calif., 1959.

5. A. Zygmund, Trigonometrical Series, Monogr. Mat., Bd. 5, Warsaw, 1935; reprint,

Dover, New York, 1955; 2nd ed., Chelsea, New York, 1952; Russian transi., Moscow, 1939.

MR 17, 361; MR 17,844.

6. W. W. Peterson, Error-Correcting Codes, M.I.T. Press, Cambridge and Wiley, New

York, 1961, pp. 251-270. MR 22 » 12003.

7. S.W. Golomb, L. R.Welch & A. Hales, On the Factorization of Trinomials Over GF(2),"

Report 20-189, Jet Propulsion Laboratory, Pasadena, Calif., 1959.

8. E. J. Watson, "Primitive polynomials (mod 2)," Math. Comp. v. 16, 1962, pp. 368-369.

MR 26 #5764.