Content uploaded by Pierre L’Ecuyer
Author content
All content in this area was uploaded by Pierre L’Ecuyer on Nov 09, 2017
Content may be subject to copyright.
Random Number Generation
Pierre L’Ecuyer1
D´epartement d’Informatique et de Recherche Op´erationnelle, Universit´e de
Montr´eal, C.P. 6128, Succ. Centre-Ville, Montr´eal (Qu´ebec), H9S 5B8, Canada.
http://www.iro.umontreal.ca/~lecuyer
1 Introduction
The fields of probability and statistics are built over the abstract concepts
of probability space and random variable. This has given rise to elegant and
powerful mathematical theory, but exact implementation of these concepts
on conventional computers seems impossible. In practice, random variables
and other random objects are simulated by deterministic algorithms. The
purpose of these algorithms is to produce sequences of numbers or objects
whose behavior is very hard to distinguish from that of their “truly random”
counterparts, at least for the application of interest. Key requirements may
differ depending on the context. For Monte Carlo methods, the main goal is
to reproduce the statistical properties on which these methods are based, so
that the Monte Carlo estimators behave as expected, whereas for gambling
machines and cryptology, observing the sequence of output values for some
time should provide no practical advantage for predicting the forthcoming
numbers better than by just guessing at random.
In computational statistics, random variate generation is usually made in
two steps: (1) generating imitations of independent and identically distributed
(i.i.d.) random variables having the uniform distribution over the interval
(0,1) and (2) applying transformations to these i.i.d. U(0,1) random variates
in order to generate (or imitate) random variates and random vectors from
arbitrary distributions. These two steps are essentially independent and the
world’s best experts on them are two different groups of scientists, with little
overlap. The expression (pseudo)random number generator (RNG) usually
refers to an algorithm used for step (1).
In principle, the simplest way of generating a random variate Xwith
distribution function Ffrom a U(0,1) random variate Uis to apply the inverse
of Fto U:
X=F−1(U)def
= min{x|F(x)≥U}.(1)
2 Pierre L’Ecuyer
This is the inversion method. It is easily seen that Xhas the desired distri-
bution: P[X≤x] = P[F−1(U)≤x] = P[U≤F(x)] = F(x). Other methods
are sometimes preferable when F−1is too difficult or expensive to compute,
as will be seen later.
The remainder of this chapter is organized as follows. In the next section,
we give a definition and the main requirements of a uniform RNG. Genera-
tors based on linear recurrences modulo a large integer m, their lattice struc-
ture and quality criteria, and their implementation, are covered in Sect. 3. In
Sect. 4, we have a similar discussion for RNGs based on linear recurrences
modulo 2. Nonlinear RNGs are briefly presented in Sect. 5. In Sect. 6, we dis-
cuss empirical statististical testing of RNGs and give some examples. Sect. 7
contains a few pointers to recommended RNGs and software. In Sect. 8, we
cover non-uniform random variate generators. We first discuss inversion and
its implementation in various settings. We then explain the alias, rejection,
ratio-of-uniform, composition, and convolution methods, and provide pointers
to the several other methods that apply in special cases.
Important basic references that we recommend are Knuth (1998); L’Ecuyer
(1994, 1998); Niederreiter (1992), and Tezuka (1995) for uniform RNGs, and
Devroye (1986); Gentle (2003), and H¨ormann et al. (2004) for non-uniform
RNGs.
2 Uniform Random Number Generators
2.1 Physical Devices
Random numbers can be generated via physical mechanisms such as the tim-
ing between successive events in atomic decay, thermal noise in semiconduc-
tors, and the like. A key issue when constructing a RNG based on a physical
device is that a “random” or “chaotic” output does not suffice; the numbers
produced must be, at least to a good approximation, realizations of indepen-
dent and uniformly distributed random variables. If the device generates a
stream of bits, which is typical, then each bit should be 0 or 1 with equal
probability, and be independent of all the other bits. In general, this cannot
be proved, so one must rely on the results of empirical statistical testing to
get convinced that the output values have the desired statistical behavior. Not
all these devices are reliable, but some apparently are. I did test two of them
recently and they passed all statistical tests that I tried.
For computational statistics, physical devices have several disadvantages
compared to a good algorithmic RNG that stands in a few lines of code.
For example, (a) they are much more cumbersome to install and run; (b)
they are more costly; (c) they are slower; (d) they cannot reproduce exactly
the same sequence twice. Item (d) is important in several contexts, including
program verification and debugging as well as comparison of similar systems
Random Number Generation 3
by simulation with common random numbers to reduce the variance (Brat-
ley et al., 1987; Fishman, 1996; Law and Kelton, 2000). Nevertheless, these
physical RNGs can be useful for selecting the seed of an algorithmic RNG,
more particularly for applications in cryptology and for gaming machines,
where frequent reseeding of the RNG with an external source of entropy (or
randomness) is important. A good algorithmic RNG whose seed is selected
at random can be viewed as an extensor of randomness, stretching a short
random seed into a long sequence of pseudorandom numbers.
2.2 Generators Based on a Deterministic Recurrence
RNGs used for simulation and other statistical applications are almost always
based on deterministic algorithms that fit the following framework, taken from
L’Ecuyer (1994): a RNG is a structure (S, µ, f, U, g) where Sis a finite set
of states (the state space), µis a probability distribution on Sused to select
the initial state (or seed)s0,f:S → S is the transition function,Uis the
output space, and g:S → U is the output function. Usually, U= (0,1), and
we shall assume henceforth that this is the case. The state of the RNG evolves
according to the recurrence si=f(si−1), for i≥1, and the output at step i
is ui=g(si)∈ U. The output values u0, u1, u2, . . . are the so-called random
numbers produced by the RNG.
Because Sis finite, there must be some finite l≥0 and j > 0 such that
sl+j=sl. Then, for all i≥l, one has si+j=siand ui+j=ui, because both f
and gare deterministic. That is, the state and output sequences are eventually
periodic. The smallest positive jfor which this happens is called the period
length of the RNG, and is denoted by ρ. When l= 0, the sequence is said to
be purely periodic. Obviously, ρ≤ |S|, the cardinality of S. If the state has a
k-bit representation on the computer, then ρ≤2k. Good RNGs are designed
so that their period length ρis not far from that upper bound. In general, the
value of ρmay depend on the seed s0, but good RNGs are normally designed
so that the period length is the same for all admissible seeds.
In practical implementations, it is important that the output be strictly
between 0 and 1, because F−1(U) is often infinite when Uis 0 or 1. All good
implementations take care of that. However, for the mathematical analysis
of RNGs, we often assume that the output space is [0,1) (i.e., 0 is admissi-
ble), because this simplifies the analysis considerably without making much
difference in the mathematical structure of the generator.
2.3 Quality Criteria
What important quality criteria should we consider when designing RNGs?
An extremely long period is obviously essential, to make sure that no wrap-
around over the cycle can occur in practice. The length of the period must be
guaranteed by a mathematical proof. The RNG must also be efficient (run
fast and use only a small amount of memory), repeatable (able to reproduce
4 Pierre L’Ecuyer
exactly the same sequence as many times as we want), and portable (work the
same way in different software/hardware environments). The availability of
efficient jump-ahead methods that can quickly compute si+νgiven si, for any
large νand any i, is also very useful, because it permits one to partition the
RNG sequence into long disjoint streams and substreams of random numbers,
in order to create an arbitrary number of virtual generators from a single RNG
(Law and Kelton, 2000; L’Ecuyer et al., 2002a). These virtual generators can
be used on parallel processors or to support different sources of randomness
in a large simulation model, for example.
Consider a RNG with state space S={1, . . . , 21000−1}, transition function
si+1 =f(si) = (si+ 1) mod 21000 , and ui=g(si) = si/21000 . This RNG
has period length 21000 and enjoys all the nice properties described in the
preceding paragraph, but is far from imitating “randomness.” In other words,
these properties are not sufficient.
A sequence of real-valued random variables u0, u1, u2, . . . are i.i.d. U(0,1) if
and only if for every integers i≥0 and t > 0, the vector ui,t = (ui, . . . , ui+t−1)
is uniformly distributed over the t-dimensional unit hypercube (0,1)t. Of
course, this cannot hold for algorithmic RNGs because any vector of tsucces-
sive values produced by the generator must belong to the finite set
Ψt={(u0, . . . , ut−1) : s0∈ S},
which is the set of all vectors of tsuccessive output values, from all possible
initial states. Here we interpret Ψtas a multiset, which means that the vectors
are counted as many times as they appear, and the cardinality of Ψtis exactly
equal to that of S.
Suppose we select the seed s0at random, uniformly over S. This can be
approximated by using some physical device, for example. Then, the vector
u0,t has the uniform distribution over the finite set Ψt. And if the sequence is
purely periodic for all s0,ui,t = (ui, . . . , ui+t−1) is also uniformly distributed
over Ψtfor all i≥0. Since the goal is to approximate the uniform distribution
over (0,1)t, it immediately becomes apparent that Ψtshould be evenly spread
over this unit hypercube. In other words, Ψtapproximates (0,1)tas the sample
space from which the vectors of successive output values are drawn randomly,
so it must be a good approximation of (0,1)tin some sense. The design of
good-quality RNGs must therefore involve practical ways of measuring the
uniformity of the corresponding sets Ψteven when they have huge cardinal-
ities. In fact, a large state space Sis necessary to obtain a long period, but
an even more important reason for having a huge number of states is to make
sure that Ψtcan be large enough to provide a good uniform coverage of the
unit hypercube, at least for moderate values of t.
More generally, we may also want to measure the uniformity of sets of the
form
ΨI={(ui1, . . . , uit)|s0∈ S},
where I={i1,···, it}is a fixed set of non-negative integers such that 0 ≤
i1<··· < it. As a special case, we recover Ψt=ΨIwhen I={0, . . . , t −1}.
Random Number Generation 5
The uniformity of a set ΨIis typically assessed by measuring the discrep-
ancy between the empirical distribution of its points and the uniform distri-
bution over (0,1)t(Niederreiter, 1992; Hellekalek and Larcher, 1998; L’Ecuyer
and Lemieux, 2002). Discrepancy measures are equivalent to goodness-of-fit
test statistics for the multivariate uniform distribution. They can be defined
in many different ways. In fact, the choice of a specific definition typically
depends on the mathematical structure of the RNG to be studied and the
reason for this is very pragmatic: we must be able to compute these measures
quickly even when Shas very large cardinality. This obviously excludes any
method that requires explicit generation of the sequence over its entire period.
The selected discrepancy measure is usually computed for each set Iin some
predefined class J, these values are weighted or normalized by factors that
depend on I, and the worst-case (or average) over Jis adopted as a figure of
merit used to rank RNGs. The choice of Jand of the weights are arbitrary.
Typically, Jwould contain sets Isuch that tand it−i1are rather small. Ex-
amples of such figures of merit will be given when we discuss specific classes
of RNGs.
2.4 Statistical Testing
Good RNGs are designed based on mathematical analysis of their properties,
then implemented and submitted to batteries of empirical statistical tests.
These tests try to detect empirical evidence against the null hypothesis H0:
“the uiare realizations of i.i.d. U(0,1) random variables.” A test can be
defined by any function Tthat maps a sequence u0, u1, . . . in (0,1) to a real
number X, and for which a good approximation is available for the distribution
of the random variable Xunder H0. For the test to be implementable, Xmust
depend on only a finite (but perhaps random) number of ui’s. Passing many
tests may improve one’s confidence in the RNG, but never guarantees that
the RNG is foolproof for all kinds of simulations.
Building a RNG that passes all statistical tests is an impossible dream.
Consider, for example, the class of all tests that examine the first (most sig-
nificant) bbits of nsuccessive output values, u0, . . . , un−1, and return a binary
value X∈ {0,1}. Select α∈(0,1) so that αbnis an integer and let Tn,b,α be
the tests in this class that return X= 1 for exactly αbnof the bnpossible
output sequences. We may say that the sequence fails the test when X= 1.
The number of tests in Tn,b,α is equal to the number of ways of choosing αbn
distinct objects among bn. The chosen objects are the sequences that fail the
test. Now, for any given output sequence, the number of such tests that return
1 for this particular sequence is equal to the number of ways of choosing the
other αbn−1 sequences that also fail the test. This is the number of ways of
choosing αbn−1 distinct objects among bn−1. In other words, as pointed
out by Leeb (1995), every output sequence fails exactly the same number of
tests! This result should not be surprising. Viewed from a different angle, it is
essentially a restatement of the well-known fact that under H0, each of the bn
6 Pierre L’Ecuyer
possible sequences has the same probability of occuring, so one could argue
that none should be considered more random than any other (Knuth, 1998).
This viewpoint seems to lead into a dead end. For statistical testing to
be meaningful, all tests should not be considered on equal footing. So which
ones are more important? Any answer is certainly tainted with its share of
arbitrariness. However, for large values of n, the number of tests is huge and
all but a tiny fraction are too complicated even to be implemented. So we
may say that bad RNGs are those that fail simple tests, whereas good RNGs
fail only complicated tests that are hard to find and run. This common-sense
compromise has been generally adopted in one way or another.
Experience shows that RNGs with very long periods, good structure of
their set Ψt, and based on recurrences that are not too simplistic, pass most
reasonable tests, whereas RNGs with short periods or bad structures are usu-
ally easy to crack by standard statistical tests. For sensitive applications, it
is a good idea, when this is possible, to apply additional statistical tests de-
signed in close relation with the random variable of interest (e.g., based on
asimplification of the stochastic model being simulated, and for which the
theoretical distribution can be computed).
Our discussion of statistical tests continues in Sect. 6.
2.5 Cryptographically Strong Generators
One way of defining an ideal RNG would be that no statistical test can dis-
tinguish its output sequence from an i.i.d. U(0,1) sequence. If an unlimited
computing time is available, no finite-state RNG can statisfy this require-
ment, because by running it long enough one can eventually figure out its
periodicity. But what if we impose a limit on the computing time? This can
be analyzed formally in the framework of asymptotic computational complex-
ity theory, under the familiar “rough-cut” assumption that polynomial-time
algorithms are practical and others are not.
Consider a family of RNGs {Gk= (Sk, µk, fk,Uk, gk), k = 1,2, . . .}where
Skof cardinality 2k(i.e., Gkhas a k-bit state). Suppose that the transition
and output functions fand gcan be computed in time bounded by a polyno-
mial in k. Let Tbe the class of statistical tests that run in time bounded by
a polynomial in kand try to differentiate between the output sequence of the
RNG and an i.i.d. U(0,1) sequence. The RNG family is called polynomial-time
perfect if there is a constant > 0 such that for all k, no test in Tcan differ-
entiate correctly with probability larger than 1/2 + e−k. This is equivalent
to asking that no polynomial-time algorithm can predict any given bit of ui
with probability of success larger than 1/2+e−k, after observing u0, . . . , ui−1.
This links unpredictability with statistical uniformity and independence. For
the proofs and additional details, see, e.g. Blum et al. (1986); L’Ecuyer and
Proulx (1989); Lagarias (1993), and Luby (1996). This theoretical framework
has been used to define a notion of reliable RNG in the context of cryptog-
raphy. But the guarantee is only asymptotic; it does not necessarily tell what
Random Number Generation 7
value of kis large enough for the RNG to be secure in practice. Moreover,
specific RNG families have been proved to be polynomial-time perfect only
under yet unproven conjectures. So far, no one has been able to prove even
their existence. Most RNGs discussed in the remainder of this chapter are
known not to be polynomial-time perfect. However, they are fast, convenient,
and have good enough statistical properties when their parameters are chosen
carefully.
3 Linear Recurrences Modulo m
3.1 The Multiple Recursive Generator
The most widely used RNGs are based on the linear recurrence
xi= (a1xi−1+· ·· +akxi−k)mod m, (2)
where mand kare positive integers called the modulus and the order, and
the coefficients a1, . . . , akare in Zm, interpreted as the set {0, . . . , m −1}
on which all operations are performed with reduction modulo m. The state
at step iis si=xi= (xi−k+1, . . . , xi)T. When mis a prime number, the
finite ring Zmis a finite field and it is possible to choose the coefficients aj
so that the period length reaches ρ=mk−1 (the largest possible value)
(Knuth, 1998). This maximal period length is achieved if and only if the
characteristic polynomial of the recurrence, P(z) = zk−a1zk−1− ··· − ak,
is a primitive polynomial over Zm, i.e., if and only if the smallest positive
integer νsuch that (zνmod P(z)) mod m= 1 is ν=mk−1. Knuth (1998)
explains how to verify this for a given P(z). For k > 1, for P(z) to be a
primitive polynomial, it is necessary that akand at least another coefficient
ajbe nonzero. Finding primitive polynomials of this form is generally easy
and they yield the simplified recurrence:
xn= (arxn−r+akxn−k)mod m. (3)
Amultiple recursive generator (MRG) uses (2) with a large value of m
and defines the output as ui=xi/m. For k= 1, this is the classical linear
congruential generator (LCG). In practice, the output function is modified
slightly to make sure that uinever takes the value 0 or 1 (e.g., one may define
ui= (xi+ 1)/(m+ 1), or ui=xi/(m+ 1) if xi>0 and ui=m/(m+ 1)
otherwise) but to simplify the theoretical analysis, we will follow the common
convention of assuming that ui=xi/m (in which case uidoes take the value
0 occasionally).
3.2 The Lattice Structure
Let eidenote the ith unit vector in kdimensions, with a 1 in position iand 0’s
elsewhere. Denote by xi,0, xi,1, xi,2, . . . the values of x0, x1, x2, . . . produced by
8 Pierre L’Ecuyer
the recurrence (2) when the initial state x0is ei. An arbitrary initial state x0=
(z1, . . . , zk)Tcan be written as the linear combination z1e1+···+zkekand the
corresponding sequence is a linear combination of the sequences (xi,0, xi,1, . . .),
with reduction of the coordinates modulo m. Reciprocally, any such linear
combination reduced modulo mis a sequence that can be obtained from some
initial state x0∈ S =Zk
m. If we divide everything by mwe find that for the
MRG, for each t≥1, Ψt=Lt∩[0,1)twhere
Lt=(v=
t
X
i=1
zivi|zi∈Z),
is a t-dimensional lattice in Rt, with basis
v1= (1,0, . . . , 0, x1,k, . . . , x1,t−1)T/m
.
.
..
.
.
vk= (0,0, . . . , 1, xk,k, . . . , xk,t−1)T/m
vk+1 = (0,0, . . . , 0,1, . . . , 0)T
.
.
..
.
.
vt= (0,0, . . . , 0,0, . . . , 1)T.
For t≤k,Ltcontains all vectors whose coordinates are multiples of 1/m. For
t > k, it contains a fraction mk−tof those vectors.
This lattice structure implies that the points of Ψtare distributed accord-
ing to a very regular pattern, in equidistant parallel hyperplanes. Graphical
illustrations of this, usually for LCGs, can be found in a myriad of papers
and books; e.g., Gentle (2003); Knuth (1998); Law and Kelton (2000), and
L’Ecuyer (1998). Define the dual lattice to Ltas
L∗
t={h∈Rt:hTv∈Zfor all v∈Lt}.
Each h∈L∗
tis a normal vector that defines a family of equidistant parallel
hyperplanes, at distance 1/khk2apart, and these hyperplanes cover all the
points of Ltunless his an integer multiple of some other vector h0∈L∗
t.
Therefore, if `tis the euclidean length of a shortest non-zero vector hin L∗
t,
then there is a family of hyperplanes at distance 1/`tapart that cover all
the points of Lt. A small `tmeans thick slices of empty space between the
hyperplanes and we want to avoid that. A large `tmeans a better (more
uniform) coverage of the unit hypercube by the point set Ψt. Computing the
value of 1/`tis often called the spectral test (Knuth, 1998; Fishman, 1996).
The lattice property holds as well for the point sets ΨIformed by values at
arbitrary lags defined by a fixed set of indices I={i1,·· · , it}. One has ΨI=
LI∩[0,1)tfor some lattice LI, and the largest distance between successive
hyperplanes for a family of hyperplanes that cover all the points of LIis 1/`I,
Random Number Generation 9
where `Iis the euclidean length of a shortest nonzero vector in L∗
I, the dual
lattice to LI.
The lattice LIand its dual can be constructed as explained in Couture
and L’Ecuyer (1996) and L’Ecuyer and Couture (1997). Finding the shortest
nonzero vector in a lattice with basis v1, . . . , vtcan be formulated as an integer
programming problem with a quadratic objective function:
Minimize kvk2=
t
X
i=1
t
X
j=1
zivT
ivjzj
subject to z1, . . . , ztintegers and not all zero. This problem can be solved by a
branch-and-bound algorithm (Fincke and Pohst, 1985; L’Ecuyer and Couture,
1997; Tezuka, 1995).
For any given dimension tand mkpoints per unit of volume, there is an
absolute upper bound on the best possible value of `I(Conway and Sloane,
1999; Knuth, 1998; L’Ecuyer, 1999b). Let `∗
t(mk) denote such an upper bound.
To define a figure of merit that takes into account several sets I, in different
numbers of dimensions, it is common practice to divide `Iby an upper bound,
in order to obtain a standardized value between 0 and 1, and then take the
worst case over a given class Jof sets I. This gives a figure of merit of the
form
MJ= min
I∈J `I/`∗
|I|(mk).
A value of MJtoo close to zero means that LIhas a bad lattice structure for
at least one of the selected sets I. We want a value as close to 1 as possible.
Computer searches for good MRGs with respect to this criterion have been
reported by L’Ecuyer et al. (1993); L’Ecuyer and Andres (1997); L’Ecuyer
(1999a), for example. In most cases, Jwas simply the sets of the form I=
{1, . . . , t}for t≤t1, where t1was an arbitrary integer ranging from 8 to
45. L’Ecuyer and Lemieux (2000) also consider the small dimensional sets I
with indices not too far apart. They suggest taking J={{0,1, . . . , i}:i <
t1}∪ {{i1, i2}: 0 = i1< i2< t2}∪ ···∪ {{i1, . . . , id}: 0 = i1< . . . < id< td}
for some positive integers d, t1, . . . , td. We could also take a weighted average
instead of the minimum in the definition of MJ.
An important observation is that for t > k, the t-dimensional vector h=
(−1, a1, . . . , ak,0, . . . , 0)Talways belong to L∗
t, because for any vector v∈Lt,
the first k+1 coordinates of mvmust satisfy the recurrence (2), which implies
that (−1, a1, . . . , ak,0, . . . , 0)vmust be an integer. Therefore, one always has
`2
t≤1 + a2
1+··· +a2
k. Likewise, if Icontains 0 and all indices jsuch that
ak−j6= 0, then `2
I≤1 + a2
1+· ·· +a2
k(L’Ecuyer, 1997). This means that the
sum of squares of the coefficients ajmust be large if we want to have any
chance that the lattice structure be good.
Contructing MRGs with only two nonzero coefficients and taking these
coefficients small has been a very popular idea, because this makes the im-
plementation easier and faster (Deng and Lin, 2000; Knuth, 1998). However,
10 Pierre L’Ecuyer
MRGs thus obtained have a bad structure. As a worst-case illustration, con-
sider the widely-available additive or subtractive lagged-Fibonacci generator,
based on the recurrence (2) where the two coefficients arand akare both
equal to ±1. In this case, whenever Icontains {0, k −r, k}, one has `2
I≤3,
so the distance between the hyperplanes is at least 1/√3. In particular, for
I={0, k −r, k}, all the points of ΨI(aside from the zero vector) are con-
tained in only two planes! This type of structure can have a dramatic effect
on certain simulation problems and is a good reason for staying away from
these lagged-Fibonacci generators, regardless of their parameters.
A similar problem occurs for the “fast MRG” proposed by Deng and Lin
(2000), based on the recurrence
xi= (−xi−1+axi−k)mod m= ((m−1)xi−1+axi−k)mod m,
with a2< m. If ais small, the bound `2
I≤1+a2implies a bad lattice structure
for I={0, k −1, k}. A more detailed analysis by L’Ecuyer and Touzin (2004)
shows that this type of generator cannot have a good lattice structure even
if the condition a2< m is removed. Another special case proposed by Deng
and Xu (2003) has the form
xi=a(xi−j2+· ·· +xi−jt)mod m. (4)
In this case, for I={0, k −jt−1, . . . , k −j2, k}, the vectors (1, a, . . . , a) and
(a∗,1, . . . , 1) both belong to the dual lattice L∗
I, where a∗is the multiplicative
inverse of amodulo m. So neither anor a∗should be small.
To get around this structural problem when Icontains certain sets of
indices, L¨uscher (1994) and Knuth (1998) recommend to skip some of the
output values in order to break up the bad vectors. For the lagged-Fibonacci
generator, for example, one can output ksuccessive values produced by the
recurrence, then skip the next dvalues, output the next k, skip the next
d, and so on. A large value of d(e.g., d= 5kor more) may get rid of the
bad structure, but slows down the generator. See Wegenkittl and Matsumoto
(1999) for further discussion.
3.3 MRG Implementation Techniques
The modulus mis often taken as a large prime number close to the largest
integer directly representable on the computer (e.g., equal or near 231 −1 for
32-bit computers). Since each xi−jcan be as large as m−1, one must be careful
in computing the right side of (2) because the product ajxi−jis typically not
representable as an ordinary integer. Various techniques for computing this
product modulo mare discussed and compared by Fishman (1996); L’Ecuyer
and Tezuka (1991); L’Ecuyer (1999a), and L’Ecuyer and Simard (1999). Note
that if aj=m−a0
j>0, using ajis equivalent to using the negative coefficient
−a0
j, which is sometimes more convenient from the implementation viewpoint.
In what follows, we assume that ajcan be either positive or negative.
Random Number Generation 11
One approach is to perform the arithmetic modulo min 64-bit (double
precision) floating-point arithmetic (L’Ecuyer, 1999a). Under this representa-
tion, assuming that the usual IEEE floating-point standard is respected, all
positive integers up to 253 are represented exactly. Then, if each coefficient
ajis selected to satisfy |aj|(m−1) ≤253, the product |aj|xi−jwill always
be represented exactly and zj=|aj|xi−jmod mcan be computed by the
instructions
y=|aj|xi−j;zj=y−mby/mc.
Similarly, if (|a1|+· · · +|ak|)(m−1) ≤253,a1xi−1+··· +akxi−kwill always
be represented exactly.
A second technique, called approximate factoring (L’Ecuyer and Cˆot´e,
1991), uses only the integer representation and works under the condition
that |aj|=ior |aj|=bm/icfor some integer i < √m. One precomputes
qj=bm/|aj|c and rj=mmod |aj|. Then, zj=|aj|xi−jmod mcan be
computed by
y=bxi−j/qjc;z=|aj|(xi−j−yqj)−yrj;
if z < 0 then zj=z+melse zj=z.
All quantities involved in these computations are integers between −mand
m, so no overflow can occur if mcan be represented as an ordinary integer
(e.g., m < 231 on a 32-bit computer).
The powers-of-two decomposition approach selects coefficients ajthat can
be written as a sum or difference of a small number of powers of 2 (Wu, 1997;
L’Ecuyer and Simard, 1999; L’Ecuyer and Touzin, 2000). For example, one
may take aj=±2q±2rand m= 2e−hfor some positive integers q,r,
e, and h. To compute y= 2qxmo d m, decompose x=z0+ 2e−qz1(where
z0=xmod 2e−q) and observe that
y= 2q(z0+ 2e−qz1)mod (2e−h) = (2qz0+hz1)mod (2e−h).
Suppose now that
h < 2qand h(2q−(h+ 1)2−e+q)< m. (5)
Then, 2qz0< m and hz1< m, so ycan be computed by shifts, masks,
additions, subtractions, and a single multiplication by h. Intermediate results
never exceed 2m−1. Things simplify further if q= 0 or q= 1 or h= 1.
For h= 1, yis obtained simply by swapping the blocks of bits z0and z1
(Wu, 1997). It has been pointed out by L’Ecuyer and Simard (1999) that
LCGs with parameters of the form m= 2e−1 and a=±2q±2rhave
bad statistical properties because the recurrence does not “mix the bits” well
enough. However, good and fast MRGs can be obtained via the power-of-two
decomposition method, as explained in L’Ecuyer and Touzin (2000).
Another interesting idea for improving efficiency is to take all nonzero co-
efficients ajequal to the same constant a(Marsaglia, 1996; Deng and Xu,
12 Pierre L’Ecuyer
2003). Then, computing the right side of (2) requires a single multiplication.
Deng and Xu (2003) provide specific parameter sets and concrete implemen-
tations for MRGs of this type, for prime mnear 231, and k= 102, 120, and
1511.
One may be tempted to take mequal to a power of two, say m= 2e,
because then the “ mod m” operation is much easier: it suffices to keep the e
least significant bits and mask-out all others. However, taking a power-of-two
modulus is not recommended because it has several strong disadvantages in
terms of the quality of the RNG (L’Ecuyer, 1990, 1998). In particular, the
least significant bits have very short periodicity and the period length of the
recurrence (2) cannot exceed (2k−1)2e−1if k > 1, and 2e−2if k= 1 and
e≥4. The maximal period length achievable with k= 7 and m= 231, for
example, is more than 2180 times smaller than the maximal period length
achievable with k= 7 and m= 231 −1 (a prime number).
3.4 Combined MRGs and LCGs
The conditions that make MRG implementations run faster (e.g., only two
nonzero coefficients both close to zero) are generally in conflict with those re-
quired for having a good lattice structure and statistical robustness. Combined
MRGs are one solution to this problem. Consider Jdistinct MRGs evolving
in parallel, based on the recurrences
xj,i = (aj,1xj,i−1+· ·· +aj,k xj,i−k)mod mj(6)
where aj,k 6= 0, for j= 1, . . . , J . Let δ1, . . . , δJbe arbitrary integers,
zi= (δ1x1,i +· ·· +δJxJ,i)mo d m1, ui=zi/m1,(7)
and
wi= (δ1x1,i/m1+· · · +δJxJ,i/mJ)mod 1.(8)
This defines two RNGs, with output sequences {ui, i ≥0}and {wi, i ≥0}.
Suppose that the mjare pairwise relatively prime, that δjand mjhave
no common factor for each j, and that each recurence (6) is purely periodic
with period length ρj. Let m=m1· · · mJand let ρbe the least common
multiple of ρ1, . . . , ρJ. Under these conditions, the following results have been
proved by L’Ecuyer and Tezuka (1991) and L’Ecuyer (1996a): (a) the sequence
(8) is exactly equivalent to the output sequence of a MRG with (composite)
modulus mand coefficients ajthat can be computed explicitly as explained
in L’Ecuyer (1996a); (b) the two sequences in (7) and (8) have period length
ρ; and (c) if both sequences have the same initial state, then ui=wi+i
where maxi≥0|i|can be bounded explicitly by a constant which is very
small when the mjare close to each other.
Thus, these combined MRGs can be viewed as practical ways of imple-
menting an MRG with a large mand several large nonzero coefficients. The
Random Number Generation 13
idea is to cleverly select the components so that: (1) each one is easy to im-
plement efficiently (e.g., has only two small nonzero coefficients) and (2) the
MRG that corresponds to the combination has a good lattice structure. If each
mjis prime and if each component jhas maximal period length ρj=mk
j−1,
then each ρjis even and ρcannot exceed ρ1···ρJ/2J−1. Tables of good pa-
rameters for combined MRGs of different sizes that reach this upper bound
are given in L’Ecuyer (1999a) and L’Ecuyer and Touzin (2000), together with
C implementations.
3.5 Jumping Ahead
The recurrence (2) can be written in matrix form as
xi=Axi−1mod m=
0 1 ··· 0
.
.
.....
.
.
0 0 ··· 1
akak−1··· a1
xi−1mod m.
To jump ahead directly from xito xi+ν, for an arbitrary integer ν, it suffices
to exploit the relationship
xi+ν=Aνximod m= (Aνmod m)ximod m.
If this is to be done several times for the same ν, the matrix Aνmod mcan be
precomputed once for all. For a large ν, this can be done in O(log2ν) matrix
multiplications via a standard divide-and-conquer algorithm (Knuth, 1998):
Aνmod m=(Aν/2mo d m)(Aν/2mo d m)mod mif νis even;
A(Aν−1mod m)mod mif νis odd.
3.6 Linear Recurrences With Carry
These types of recurrences were introduced by Marsaglia and Zaman (1991)
to obtain a large period even when mis a power of two (in which case the
implementation may be faster). They were studied and generalized by Tezuka
et al. (1994); Couture and L’Ecuyer (1994, 1997), and Goresky and Klapper
(2003). The basic idea is to add a carry to the linear recurrence (2). The
general form of this RNG, called multiply-with-carry (MWC), can be written
as
xi= (a1xi−1+· ·· +akxi−k+ci−1)dmod b,
ci=b(a0xi+a1xi−1+· ·· +akxi−k+ci−1)/bc,
ui=
∞
X
`=1
xi+`−1b−`,
14 Pierre L’Ecuyer
where bis a positive integer (e.g., a power of two), a0, . . . , akare arbitrary
integers such that a0is relatively prime to b, and dis the multiplicative inverse
of −a0modulo b. The state at step iis si= (xi−k+1, . . . , xi, ci)T. In practice,
the sum in (9) is truncated to a few terms (it could be a single term if bis
large), but the theoretical analysis is much easier for the infinite sum.
Define m=Pk
`=0 a`b`and let abe the inverse of bin arithmetic modulo m,
assuming for now that m > 0. A major result proved in Tezuka et al. (1994);
Couture and L’Ecuyer (1997), and Goresky and Klapper (2003) is that if the
initial states agree, the output sequence {ui, i ≥0}is exactly the same as
that produced by the LCG with modulus mand multiplier a. Therefore, the
MWC can be seen as a clever way of implementing a LCG with very large
modulus. It has been shown by Couture and L’Ecuyer (1997) that the value
of `tfor this LCG satisfies `2
t≤a2
0+··· +a2
kfor t≥k, which means that
the lattice structure will be bad unless the sum of squares of coefficients ajis
large.
In the original proposals of Marsaglia and Zaman (1991), called add-with-
carry and subtract-with-borrow, one has −a0=±ar=±ak= 1 for some r < k
and the other coefficients ajare zero, so `2
t≤3 for t≥kand the generator
has essentially the same structural defect as the additive lagged-Fibonacci
generator. In the version studied by Couture and L’Ecuyer (1997), it was
assumed that −a0=d= 1. Then, the period length cannot exceed (m−1)/2
if bis a power of two. A concrete implementation was given in that paper.
Goresky and Klapper (2003) pointed out that the maximal period length of
ρ=m−1 can be achieved by allowing a more general a0. They provided
specific parameters that give a maximal period for branging from 221 to 235
and ρup to approximately 22521.
4 Generators Based on Recurrences Modulo 2
4.1 A General Framework
It seems natural to exploit the fact that computers work in binary arithmetic
and to design RNGs defined directly in terms of bit strings and sequences.
We do this under the following framework, taken from L’Ecuyer and Panneton
(2002). Let F2denote the finite field with two elements, 0 and 1, in which the
operations are equivalent to addition and multiplication modulo 2. Consider
the RNG defined by a matrix linear recurrence over F2, as follows:
xi=Axi−1,(9)
yi=Bxi,(10)
ui=
w
X
`=1
yi,`−12−`=.yi,0yi,1yi,2···,(11)
Random Number Generation 15
where xi= (xi,0, . . . , xi,k−1)T∈Fk
2is the k-bit state vector at step i,yi=
(yi,0, . . . , yi,w−1)T∈Fw
2is the w-bit output vector at step i,kand ware
positive integers, Ais a k×ktransition matrix with elements in F2,Bis a
w×koutput transformation matrix with elements in F2, and ui∈[0,1) is the
output at step i. All operations in (9) and (10) are performed in F2.
It is well-known (Niederreiter, 1992; L’Ecuyer, 1994) that when the xi’s
obey (9), for each j, the sequence {xi,j , i ≥0}follows the linear recurrence
xi,j = (α1xi−1,j +· ·· +αkxi−k,j )mod 2,(12)
whose characteristic polynomial P(z) is the characteristic polynomial of A,
i.e.,
P(z) = det(A−zI) = zk−α1zk−1− · ·· − αk−1z−αk,
where Iis the identity matrix and each αjis in F2. The sequences {yi,j , i ≥0},
for 0 ≤j < w, also obey the same recurrence (although some of them may
follow recurrences of shorter order as well in certain situations, depending on
B). We assume that αk= 1, so that the recurrence (12) has order kand is
purely periodic. Its period length is 2k−1 (i.e., maximal) if and only if P(z)
is a primitive polynomial over F2(Niederreiter, 1992; Knuth, 1998).
To jump ahead directly from xito xi+νwith this type of generator, it
suffices to precompute the matrix Aν(in F2) and then multiply xiby this
matrix.
Several popular classes of RNGs fit this framework as special cases, by
appropriate choices of the matrices Aand B. This includes the Tausworthe
or LFSR, polynomial LCG, GFSR, twisted GFSR, Mersenne twister, multiple
recursive matrix generators, and combinations of these (L’Ecuyer and Pan-
neton, 2002; Matsumoto and Nishimura, 1998; Niederreiter, 1995; Tezuka,
1995). We detail some of them after discussing measures of uniformity.
4.2 Measures of Uniformity
The uniformity of point sets ΨIproduced by RNGs based on linear recur-
rences over F2is usually assessed by measures of equidistribution defined
as follows (L’Ecuyer, 1996b; L’Ecuyer and Panneton, 2002; L’Ecuyer, 2004;
Tezuka, 1995). For an arbitrary vector q= (q1, . . . , qt) of non-negative inte-
gers, partition the unit hypercube [0,1)tinto 2qjintervals of the same length
along axis j, for each j. This determines a partition of [0,1)tinto 2q1+···+qt
rectangular boxes of the same size and shape. We call this partition the q-
equidissection of the unit hypercube.
For some index set I={i1, . . . , it}, if ΨIhas 2kpoints, we say that ΨIis
q-equidistributed in base 2 if there are exactly 2qpoints in each box of the
q-equidissection, where k−q=q1+· ·· +qt. This means that among the 2k
points (xj1, . . . , xjt) of ΨI, if we consider the first q1bits of xj1, the first q2
bits of xj2, . . . , and the first qtbits of xjt, each of the 2k−qpossibilities occurs
exactly the same number of times. This is possible only if q≤k.
16 Pierre L’Ecuyer
The q-equidistribution of ΨIdepends only on the first qjbits of xijfor
1≤j≤t, for the points (xi1, . . . , xit) that belong to ΨI. The vector of these
q1+···+qt=k−qbits can always be expressed as a linear function of the k
bits of the initial state x0, i.e., as Mqx0for some (k−q)×kbinary matrix
Mq, and it is easily seen that ΨIis q-equidistributed if and only if Mqhas full
rank k−q. This provides an easy way of checking equidistribution (Fushimi,
1983; L’Ecuyer, 1996b; Tezuka, 1995).
If ΨIis (`, . . . , `)-equidistributed for some `≥1, it is called t-distributed
with `bits of accuracy, or (t, `)-equidistributed (L’Ecuyer, 1996b). The largest
value of `for which this holds is called the resolution of the set ΨIand is
denoted by `I. This value has the upper bound `∗
t= min(bk/tc, w). The
resolution gap of ΨIis defined as δI=`∗
t−`I. In the same vein as for MRGs,
a worst-case figure of merit can be defined here by
∆J= max
I∈J δI,
where Jis a preselected class of index sets I.
The point set ΨIis a (q, k, t)-net in base 2 (often called a (t, m, s)-net
in the context of quasi-Monte Carlo methods, where a different notation is
used (Niederreiter, 1992)), if it is (q1, . . . , qt)-equidistributed in base 2 for all
non-negative integers q1, . . . , qtsumming to k−q. We call the smallest such
qthe q-value of ΨI. The smaller it is, the better. One candidate for a figure
of merit could be the q-value of Ψtfor some large t. A major drawback of
this measure is that it is extremely difficult to compute for good long-period
generators (for which k−qis large), because there are too many vectors qfor
which equidistribution needs to be checked. In practice, one must settle for
figures of merit that involve a smaller number of equidissections.
When δI= 0 for all sets Iof the form I={0, . . . , t −1}, for 1 ≤t≤k,
the RNG is said to be maximally equidistributed or asymptotically random
for the word size w(L’Ecuyer, 1996b; Tezuka, 1995; Tootill et al., 1973). This
property ensures perfect equidistribution of all sets Ψt, for any partition of the
unit hypercube into subcubes of equal sizes, as long as `≤wand the number
of subcubes does not exceed the number of points in Ψt. Large-period max-
imally equidistributed generators, together with their implementations, can
be found in L’Ecuyer (1999c); L’Ecuyer and Panneton (2002), and Panneton
and L’Ecuyer (2004), for example.
4.3 Lattice Structure in Spaces of Polynomials and Formal Series
The RNGs defined via (9)–(11) do not have a lattice structure in the real space
like MRGs, but they do have a lattice structure in a space of formal series,
as explained in Couture and L’Ecuyer (2000); L’Ecuyer (2004); Lemieux and
L’Ecuyer (2003), and Tezuka (1995). The real space Ris replaced by the space
L2of formal power series with coefficients in F2, of the form P∞
`=ωx`z−`for
some integer ω. In that setting, the lattices have the form
Random Number Generation 17
Lt=
v(z) =
t
X
j=1
hj(z)vj(z) such that each hj(z)∈F2[z]
,
where F2[z] is the ring of polynomials with coefficients in F2, and the basis
vectors vjare in Lt
2. The elements of the dual lattice L∗
tare the vectors h(z)
in Lt
2whose scalar product with any vector of Ltis a polynomial (in F2[z]).
We define the mapping ϕ:L2→Rby
ϕ ∞
X
`=ω
x`z−`!=
∞
X
`=ω
x`2−`.
Then, it turns out that the point set Ψtproduced by the generator is equal to
ϕ(Lt)∩[0,1)t. Moreover, the equidistribution properties examined in Sect. 4.2
can be expressed in terms of lengths of shortest vectors in the dual lattice,
with appropriate definitions of the length (or norm). Much of the theory and
algorithms developed for lattices in the real space can be adapted to these
new types of lattices (Couture and L’Ecuyer, 2000; L’Ecuyer, 2004; Lemieux
and L’Ecuyer, 2003; Tezuka, 1995).
4.4 The LFSR Generator
The Tausworthe or linear feedback shift register (LFSR) generator (Taus-
worthe, 1965; L’Ecuyer, 1996b; Tezuka, 1995) is a special case of (9)–(11)
with A=As
0(in F2) for some positive integer s, where
A0=
1
...
1
akak−1. . . a1
,(13)
a1, . . . , akare in F2,ak= 1, and all blank entries in the matrix are zeros. We
take w≤kand the matrix Bcontains the first wlines of the k×kidentity
matrix. The RNG thus obtained can be defined equivalently by
xi=a1xi−1+· ·· +akxi−kmod 2,(14)
ui=
w
X
`=1
xis+`−12−`.(15)
Here, P(z) is the characteristic polynomial of the matrix As
0, not the char-
acteristic polynomial of the recurrence (14), and the choice of sis important
for determining the quality of the generator. A frequently encountered case
is when a single ajis nonzero in addition to ak; then, P(z) is a trinomial
and we say that we have a trinomial-based LFSR generator. These generators
18 Pierre L’Ecuyer
are known to have important statistical deficiencies (Matsumoto and Kurita,
1996; Tezuka, 1995) but they can be used a components of combined RNGs.
LFSR generators can be expressed as LCGs in a space of polynomials
(Tezuka and L’Ecuyer, 1991; Tezuka, 1995; L’Ecuyer, 1994). With this repre-
sentation, their lattice structure as discussed in Sect. 4.3 follows immediately.
4.5 The GFSR and Twisted GFSR
Here we take Aas the pq ×pq matrix
A=
IpS
Ip
Ip
...
Ip
for some positive integers pand q, where Ipis the p×pidentity matrix, Sis
ap×pmatrix, and the matrix Ipon the first line is in columns (r−1)p+ 1 to
rp for some positive integer r. Often, w=pand Bcontains the first wlines of
the pq ×pq identity matrix. If Sis also the identity matrix, the generator thus
obtained is the trinomial-based generalized feedback shift register (GFSR), for
which xiis obtained by a bitwise exclusive-or of xi−rand xi−q. This gives a
very fast RNG, but its period length cannot exceed 2q−1, because each bit
of xifollows the same binary recurrence of order k=q, with characterictic
polynomial P(z) = zq−zq−r−1.
More generally, we can define xias the bitwise exclusive-or of xi−r1,xi−r2,
. . . , xi−rdwhere rd=q, so that each bit of xifollows a recurrence in F2
whose characteristic polynomial P(z) has d+ 1 nonzero terms. However, the
period length is still bounded by 2q−1, whereas considering the pq-bit state,
we should rather expect a period length close to 2pq . This was the main
motivation for the twisted GFSR (TGFSR) generator. In the original version
introduced by Matsumoto and Kurita (1992), w=pand the matrix Sis
defined as the transpose of A0in (13), with kreplaced by p. The characteristic
polynomial of Ais then P(z) = PS(zq+zm), where PS(z) = zp−apzp−1−
· ·· − a1is the characteristic polynomial of S, and its degree is k=pq. If the
parameters are selected so that P(z) is primitive over F2, then the TGFSR has
period length 2k−1. Matsumoto and Kurita (1994) pointed out important
weaknesses of the original TGFSR and proposed an improved version that
uses a well-chosen matrix Bwhose lines differ from those of the identity.
The operations implemented by this matrix are called tempering and their
purpose is to improve the uniformity of the points produced by the RNG.
The Mersenne twister (Matsumoto and Nishimura, 1998; Nishimura, 2000) is
a variant of the TGFSR where kis slightly less than pq and can be a prime
number. A specific instance proposed by Matsumoto and Nishimura (1998) is
Random Number Generation 19
fast, robust, has the huge period length of 219937 −1, and has become quite
popular.
In the multiple recursive matrix method of Niederreiter (1995), the first
row of p×pmatrices in Acontains arbitrary matrices. However, a fast imple-
mentation is possible only when these matrices are sparse and have a special
structure.
4.6 Combined Linear Generators Over F2
Many of the best generators based on linear recurrences over F2are con-
structed by combining the outputs of two or more RNGs having a simple
structure. The idea is the same as for MRGs: select simple components that
can run fast but such that their combination has a more complicated structure
and highly-uniform sets ΨIfor the sets Iconsidered important.
Consider Jdistinct recurrences of the form (9)–(10), where the jth recur-
rence has parameters (k, w, A,B)=(kj, w, Aj,Bj) and state xj,i at step i,
for j= 1, . . . , J . The output of the combined generator at step iis defined by
yi=B1x1,i ⊕ · ·· ⊕ BJxJ,i,
ui=
w
X
`=1
yi,`−12−`,
where ⊕denotes the bitwise exclusive-or operation. One can show (Tezuka,
1995) that the period length ρof this combined generator is the least common
multiple of the period lengths ρjof its components. Moreover, this combined
generator is equivalent to the generator (9)–(11) with k=k1+· · · +kJ,A=
diag(A1, . . . , AJ), and B= (B1, . . . , BJ).
With this method, by selecting the parameters carefully, the combination
of LFSR generators with characteristic polynomials P1(z), . . . , PJ(z) gives yet
another LFSR with characteristic polynomial P(z) = P1(z)···PJ(z) and pe-
riod length equal to the product of the period lengths of the components
(Tezuka and L’Ecuyer, 1991; Wang and Compagner, 1993; L’Ecuyer, 1996b;
Tezuka, 1995). Tables and fast implementations of maximally equidistributed
combined LFSR generators are given in L’Ecuyer (1996b).
The TGFSR and Mersenne twister generators proposed in Matsumoto and
Kurita (1994); Matsumoto and Nishimura (1998) and Nishimura (2000) can-
not be maximally equidistributed. However, concrete examples of maximally
equidistributed combined TGFSR generators with period lengths near 2466
and 21250 are given in L’Ecuyer and Panneton (2002). These generators have
the additional property that the resolution gaps δIare zero for a class of small
sets Iwith indices not too far apart.
20 Pierre L’Ecuyer
5 Nonlinear RNGs
All RNGs discussed so far are based on linear recurrences and their struc-
ture may be deemed too regular. There are at least two ways of getting rid
of this regular linear structure: (1) use a nonlinear transition function for
(2) keep the transition function linear but use a nonlinear output function g.
Several types of nonlinear RNGs have been proposed over the years; see, e.g.,
Blum et al. (1986); Eichenauer-Herrmann (1995); Eichenauer-Herrmann et al.
(1997); Hellekalek and Wegenkittl (2003); Knuth (1998); L’Ecuyer (1994);
Niederreiter and Shparlinski (2002), and Tezuka (1995). Their nonlinear map-
pings are defined in various ways by multiplicative inversion in a finite field,
quadratic and cubic functions in the finite ring of integers modulo m, and other
more complicated transformations. Many of them have output sequences that
tend to behave much like i.i.d. U(0,1) sequences even over their entire period
length, in contrast with “good” linear RNGs, whose point sets Ψtare much
more regular than typical random points (Eichenauer-Herrmann et al., 1997;
L’Ecuyer and Hellekalek, 1998; L’Ecuyer and Granger-Pich´e, 2003; Nieder-
reiter and Shparlinski, 2002). On the other hand, their statistical properties
have been analyzed only empirically or via asymptotic theoretical results. For
specific nonlinear RNGs, the uniformity of the point sets Ψtis very difficult
to measure theoretically. Moreover, the nonlinear RNGs are generally signif-
icantly slower than the linear ones. The RNGs recommended for cryptology
are all nonlinear.
An interesting idea for adding nonlinearity without incurring an excessive
speed penalty is to combine a small nonlinear generator with a fast long-period
linear one (Aiello et al., 1998; L’Ecuyer and Granger-Pich´e, 2003). L’Ecuyer
and Granger-Pich´e (2003) show how to do this while ensuring theoretically
the good uniformity properties of Ψtfor the combined generator. A very fast
implementation can be achieved by using precomputed tables for the nonlinear
component. Empirical studies suggest that mixed linear-nonlinear combined
generators are more robust than the linear ones with respect to statistical
tests, because of their less regular structure.
Several authors have proposed various ways of combining RNGs to pro-
duce streams of random numbers with less regularity and better “randomness”
properties; see, e.g., Collings (1987); Knuth (1998); Gentle (2003); Law and
Kelton (2000); L’Ecuyer (1994); Fishman (1996); Marsaglia (1985), and other
references given there. This includes shuffling the output sequence of one
generator using another one (or the same one), alternating between several
streams, or just adding them in different ways. Most of these techniques are
heuristics. They usually improve the uniformity (empirically), but they can
also make it worse. For random variables in the mathematical sense, certain
types of combinations (e.g., addition modulo 1) can provably improve the uni-
formity, and some authors have used this fact to argue that combined RNGs
are provably better than their components alone (Brown and Solomon, 1979;
Deng and George, 1990; Marsaglia, 1985; Gentle, 2003), but this argument is
Random Number Generation 21
faulty because the output sequences of RNGs are deterministic, not sequences
of independent random variables. To assess the quality of a combined genera-
tor, one must analyze the mathematical structure of the combined generator
itself rather than the structure of its components (L’Ecuyer, 1996b,a, 1998;
L’Ecuyer and Granger-Pich´e, 2003; Tezuka, 1995).
6 Examples of Statistical Tests
As mentioned earlier, a statistical test for RNGs is defined by a random vari-
able Xwhose distribution under H0can be well approximated. When Xtakes
the value x, we define the right and left p-values of the test by
pR=P[X≥x| H0] and pL=P[X≤x| H0].
When testing RNGs, there is no need to prespecify the level of the test. If any
of the right or left p-value is extremely close to zero, e.g., less than 10−15, then
it is clear that H0(and the RNG) must be rejected. When a suspicious p-value
is obtained, e.g., near 10−2or 10−3, one can just repeat this particular test a
few more times, perhaps with a larger sample size. Almost always, things will
then clarify.
Most tests are defined by partitioning the possible realizations of (u0, . . . ,
uτ−1) into a finite number of subsets (where the integer τcan be random or
deterministic), computing the probability pjof each subset junder H0, and
measuring the discrepancy between these probabilities and empirical frequen-
cies from realizations simulated by the RNG.
A special case that immediately comes to mind is to take τ=t(a constant)
and cut the interval [0,1) into dequal segments for some positive integer d,
in order to partition the hypercube [0,1)tinto k=dtsubcubes of volume
1/k. We then generate npoints ui= (uti, . . . , uti+t−1)∈[0,1)t, for i=
0, . . . , n −1, and count the number Njof points falling in subcube j, for
j= 0, . . . , k −1. Any measure of distance (or divergence) between the numbers
Njand their expectations n/k can define a test statistic X. The tests thus
defined are generally called serial tests of uniformity (Knuth, 1998; L’Ecuyer
et al., 2002b). They can be sparse (if n/k 1), or dense (if n/k 1), or
somewhere in between. There are also overlapping versions, where the points
are defined by ui= (ui, . . . , ui+t−1) for i= 0, . . . , n−1 (they have overlapping
coordinates).
Special instances for which the distribution under H0is well-known are
the chi-square, the (negative) empirical entropy, and the number of collisions
(L’Ecuyer and Hellekalek, 1998; L’Ecuyer et al., 2002b; Read and Cressie,
1988). For the latter, the test statistic Xis the number of times a point
falls in a subcube that already had a point in it. Its distribution under H0is
approximately Poisson with mean λ1=n2/(2k), if nis large and λ1not too
large.
22 Pierre L’Ecuyer
A variant is the birthday spacings test, defined as follows (Marsaglia, 1985;
Knuth, 1998; L’Ecuyer and Simard, 2001). Let I(1) ≤ ··· ≤ I(n)be the num-
bers of the subcubes that contain the points, sorted by increasing order. Define
the spacings Sj=I(j+1) −I(j), for j= 1, . . . , n −1, and let Xbe the number
of collisions between these spacings. Under H0,Xis approximately Poisson
with mean λ2=n3/(4k), if nis large and λ2not too large.
Consider now a MRG, for which Ψthas a regular lattice structure. Because
of this regularity the points of Ψtwill tend to be more evenly distributed
among the subcubes than random points. For a well-chosen kand large enough
n, we expect the collision test to detect this: it is likely that there will be too
few collisions. In fact, the same applies to any RNG whose set Ψtis very evenly
distributed. When a birthday spacings test with a very large kis applied to
a MRG, the numbers of the subcubes that contain one point of Ψttend to be
too evenly spaced and the test detects this by finding too many collisions.
These specific interactions between the test and the structure of the RNG
lead to systematic patterns in the p-values of the tests. To illustrate this,
suppose that we take kslightly larger than the cardinality of Ψt(so k≈ρ) and
that due to the excessive regularity, no collision is observed in the collision
test. The left p-value will then be pL≈P[X≤0|X∼Poisson(λ1)] =
exp[−n2/(2k)]. For this p-value to be smaller than a given , we need a sample
size nproportional to the square root of the period length ρ. And after that,
pLdecreases exponentially fast in n2.
Extensive experiments with LCGs, MRGs, and LFSR generators confirms
that this is actually what happens with these RNGs (L’Ecuyer and Hellekalek,
1998; L’Ecuyer, 2001; L’Ecuyer et al., 2002b). For example, if we take =
10−15 and define n0as the minimal sample size nfor which pL< , we find
that n0≈16ρ1/2(plus some noise) for LCGs that behave well in the spectral
test as well as for LFSR generators. For the birthday spacings test, the rule for
LCGs is n0≈16ρ1/3instead (L’Ecuyer and Simard, 2001). So to be safe with
respect to these tests, the period length ρmust be so large that generating
more than ρ1/3numbers is practically unfeasible. This certainly disqualifies
all LCGs with modulus smaller than 2100 or so.
Other types of tests for RNGs include tests based on the closest pairs
of points among npoints generated in the hypercube, tests based on random
walks on the real line or over the integers, tests based on the linear complexity
of a binary sequence, tests based on the simulation of dices or poker hands,
and many others (Gentle, 2003; Knuth, 1998; L’Ecuyer and Simard, 2002;
Marsaglia, 1996; Rukhin et al., 2001; Vattulainen et al., 1995).
When testing RNGs, there is no specific alternative hypothesis to H0.
Different tests are needed to detect different types of departures from H0.Test
suites for RNGs include a selection of tests, with predetermined parameters
and sample sizes. The best known are probably DIEHARD (Marsaglia, 1996)
and the NIST test suite (Rukhin et al., 2001). The library TestU01 (L’Ecuyer
and Simard, 2002) implements a large selection of tests in the C language
Random Number Generation 23
and provides a variety of test suites, some designed for i.i.d. U(0,1) output
sequences and others for strings of bits.
7 Available Software and Recommendations
When we apply test suites to RNGs currently found in commercial software
(statistical and simulation software, spreadsheets, etc.), we find that many of
them fail the tests spectacularly (L’Ecuyer, 1997, 2001). There is no reason to
use these poor RNGs, because there are also several good ones that are fast,
portable, and pass all these test suites with flying colors. Among them we rec-
ommend, for example, the combined MRGs, combined LFSRs, and Mersenne
twisters proposed in L’Ecuyer (1999c,a); L’Ecuyer and Panneton (2002); Mat-
sumoto and Nishimura (1998), and Nishimura (2000).
A convenient ob ject-oriented software package with multiple streams and
substreams of random numbers, is described in L’Ecuyer et al. (2002a)
and is available in Java, C, and C++, at http://www.iro.umontreal.ca/
~lecuyer.
8 Non-Uniform Random Variate Generation
Like for the uniform case, non-uniform variate generation often involves ap-
proximations and compromises. The first requirement is, of course, correct-
ness. This does not mean that the generated random variate Xmust always
have exactly the required distribution, because this would sometimes be much
too costly or even impossible. But we must have a good approximation and,
preferably, some understanding of the quality of that approximation. Robust-
ness is also important: when the accuracy depends on the parameters of the
distribution, it must be good uniformly over the entire range of parameter
values that we are interested in.
The method must also be efficient both in terms of speed and memory
usage. Often, it is possible to increase the speed by using more memory (e.g,
for larger precomputed tables) or by relaxing the accuracy requirements. Some
methods need a one-time setup to compute constants and construct tables.
The setup time can be significant but may be well worth spending if it is
amortized by a large number of subsequent calls to the generator. For example,
it makes sense to invest in a more extensive setup if we plan to make a million
calls to a given generator than if we expert to make only a few calls, assuming
that this investment can improve the speed of the generator sufficiently.
In general, compromises must be made between simplicity of the algorithm,
quality of the approximation, robustness with respect to the distribution pa-
rameters, and efficiency (generation speed, memory requirements, and setup
time).
24 Pierre L’Ecuyer
In many situations, compatibility with variance reduction techniques is
another important issue (Bratley et al., 1987; Law and Kelton, 2000). We may
be willing to sacrifice the speed of the generator to preserve inversion, because
the gain in efficiency obtained via the variance reduction methods may more
than compensate (sometimes by orders of magnitude) for the slightly slower
generator.
8.1 Inversion
The inversion method, defined in the introduction, should be the method of
choice for generating non-uniform random variates in a ma jority of situa-
tions. The fact that X=F−1(U) is a monotone (non-decreasing) function
of Umakes this method compatible with important variance reductions tech-
niques such as common random numbers, antithetic variates, latin hypercube
sampling, and randomized quasi-Monte Carlo methods (Bratley et al., 1987;
Law and Kelton, 2000; L’Ecuyer and Lemieux, 2000).
For some distributions, an analytic expression can be obtained for the
inverse distribution function F−1and inversion can be easily implemented.
As an example, consider the Weibull distribution function with parameters
α > 0 and β > 0, defined by F(x) = 1 −exp[−(x/β)α] for x > 0. It is easy to
see that F−1(U) = β[−ln(1 −U)]1/α. For α= 1, we have the special case of
the exponential distribution with mean β.
For an example of a simple discrete distribution, suppose that P[X=i] =
piwhere p0= 0.6, p1= 0.3, p2= 0.1, and pi= 0 elsewhere. The inversion
method in this case will return 0 if U < 0.6, 1 if 0.6≤U < 0.9, and 2
if U≥0.9. For the discrete uniform distribution over {0, . . . , k −1}, return
X=bkU c. As another example, let Xhave the geometric distribution with
parameter p, so P[X=x] = p(1 −p)xfor x= 0,1,2, . . ., where 0 <p<1.
Then, F(x) = 1−(1 −p)bx+1cfor x≥0 and one can show that X=F−1(U) =
dln(1 −U)/ln(1 −p)e − 1.
There are other distributions (e.g., the normal, Student, chi-square) for
which there is no closed-form expression for F−1but good numerical approx-
imations are available (Bratley et al., 1987; Gentle, 2003; Marsaglia et al.,
1994). When the distribution has only scale and location parameters, we need
to approximate F−1only for a standardized version of the distribution. For
the normal distribution, for example, it suffices to have an efficient method for
evaluating the inverse distribution function of a N(0,1) random variable Z,
since a normal with mean µand variance σ2can be generated by X=σZ +µ.
When shape parameters are involved (e.g., the gamma and beta distributions),
things are more complicated because F−1then depends on the parameters in
a more fundamental manner.
H¨ormann and Leydold (2003) propose a general adaptive and automatic
method that constructs a highly accurate Hermite interpolation method of
F−1. In a one-time setup, their method produces tables for the interpolation
Random Number Generation 25
points and coefficients. Random variate generation using these tables is then
quite fast.
A less efficient but simpler way of implementing inversion when a method
is available for computing Fis via binary search (Cheng, 1998). If the density
is also available and if it is unimodal with known mode, a Newton-Raphson
iteration method can avantageously replace the binary search (Cheng, 1998;
Devroye, 1986).
To implement inversion for general discrete distributions, sequential search
and binary search with look-up tables are the standard methods (Bratley
et al., 1987; Cheng, 1998). For a discrete distribution over the values x1<
··· < xk, one first tabulates the pairs (xi, F (xi)), where F(xi) = P[X≤xi]
for i= 1, . . . , k. To generate X, it then suffices to generate U∼U(0,1), find
I= min{i|F(xi)≥U}, and return X=xI. The following algorithms do
that.
Sequential search (needs O(k) iterations in the worst case);
generate U∼U(0,1); let i= 1;
while F(xi)< U do i=i+ 1;
return xi.
Binary search (needs O(log k) iterations in the worst case);
generate U∼U(0,1); let L= 0 and R=k;
while L < R −1 do
m=b(L+R)/2c;
if F(xm)< U then L=melse R=m;
/* Invariant: at this stage, the index Iis in {L+ 1, . . . , R}. */
return xR.
These algorithms can be modified in many different ways. For example,
if k=∞, in the binary search, one can start with an arbitrary value of R,
double it until F(xR)≥U, and start the algorithm with this Rand L=R/2.
Of course, only a finite portion of the table (a portion that contains most of
the probability mass) would be precomputed in this case, the other values can
be computed only when needed. This can also be done if kis finite but large.
Another class of techniques use indexing or buckets to speed up the search
(Chen and Asau, 1974; Bratley et al., 1987; Devroye, 1986). For example, one
can partition the interval (0,1) into csubintervals of equal sizes and use (pre-
tabulated) initial values of (L, R) that depend on the subinterval in which
Ufalls. For the subinterval [j/c, (j+ 1)/c) the values of Land Rwould be
Lj=F−1(j/c) and Rj=F−1((j+ 1)/c), for j= 0, . . . , c −1. The subinterval
number that corresponds to a given Uis simply J=bcUc. Once we know
that subinterval, we can search it by linear of binary search. With a larger
value of cthe search is faster (on the average) but the setup is more costly
and a larger amount of memory is needed. So a compromise must be made
depending on the situation (e.g., the value of k, the number of variates we
expect to generate, etc.). For c= 1, we recover the basic sequential and binary
26 Pierre L’Ecuyer
search algorithms given above. A well-implemented indexed search with a
large enough cis generally competitive with the alias method (described in the
next paragraph). A combined indexed/binary search algorithm is given below.
An easy adaptation gives the combined indexed/sequential search, which is
generally preferable when k/c is small, because it has smaller overhead.
Indexed search (combined with binary search);
generate U∼U(0,1); let J=bcUc,L=LJ, and R=RJ;
while L < R −1 do
m=b(L+R)/2c;
if F(xm)< U then L=melse R=m;
return xR.
These search methods are also useful for piecewise-linear (or piecewise-
polynomial) distribution functions. Essentially, it suffices to add an inter-
polation step at the end of the algorithm, after the appropriate linear (or
polynomial) piece has been determined (Bratley et al., 1987).
Finally, the stochastic model itself can sometimes be selected in a way that
makes inversion easier. For example, one can fit a parametric, highly-flexible,
and easily computable inverse distribution function F−1to the data, directly
or indirectly (Nelson and Yamnitsky, 1998; Wagner and Wilson, 1996).
There are situations where speed is important and where non-inversion
methods are appropriate. In forthcoming subsections, we outline the main
non-inversion methods.
8.2 The Alias Method
Sequential and binary search require O(k) and O(log k) time, respectively, in
the worst case, to generate a random variate Xby inversion over the finite set
{x1, . . . , xk}. The alias method (Walker, 1974, 1977) can generate such a X
in O(1) time per variate, after a table setup that takes O(k) time and space.
On the other hand, it does not implement inversion, i.e., the transformation
from Uto Xis not monotone.
To explain the idea, consider a bar diagram of the distribution, where each
index ihas a bar of height pi=P[X=xi]. The idea is to “equalize” the bars
so that they all have height 1/k, by cutting-off bar pieces and transfering
them to other bars. This is done in a way that in the new diagram, each bar
icontains one piece of size qi(say) from the original bar iand one piece of
size 1/k −qifrom another bar whose index j, denoted A(i), is called the alias
value of i. The setup procedure initializes two tables, Aand R, where A(i) is
the alias value of iand R(i) = (i−1)/k +qi. See Devroye (1986) and Law
and Kelton (2000) for the details. To generate X, we generate U∼U[0,1],
define I=dkU e, and return X=xIif U < R(I) and X=xA(I)otherwise.
There is a version of the alias method for continuous distributions, called
the acceptance-complement method (Kronmal and Peterson, 1984; Devroye,
Random Number Generation 27
1986; Gentle, 2003). The idea is to decompose the density fof the target
distribution as the convex combination of two densities f1and f2,f=wf1+
(1−w)f2for some real number w∈(0,1), in a way that wf1≤gfor some other
density gand so that it is easy to generate from gand f2. The algorithm works
as follows: Generate Xfrom density gand U∼U(0,1); if Ug(X)≤wf1(Y)
return X, otherwise generate a new Xfrom density f2and return it.
8.3 Kernel Density Estimation and Generation
Instead of selecting a parametric distribution that is hard to invert and es-
timating the parameters, one can estimate the density via a kernel density
estimation method for which random variate generation is very easy (De-
vroye, 1986; H¨ormann and Leydold, 2000). In the case of a gaussian kernel,
for example, on can generate variates simply by selecting one observation at
random from the data and adding random noise generated form a normal dis-
tribution with mean zero. However, this method is not equivalent to inversion.
Because of the added noise, selecting a larger observation does not necessarily
guarantee a larger value for the generated variate.
8.4 The Rejection Method
Now suppose we want to generate Xfrom a complicated density f. In fact
fmay be known only up to a multiplicative constant κ > 0, i.e., we know
only κf. If we know f, we may just take κ= 1. We select another density r
such that κf(x)≤t(x)def
=ar(x) for all xfor some constant a, and such that
generating variates Yfrom the density ris easy. The function tis called a hat
function or majorizing function. By integrating this inequality with respect to
xon both sides, we find that κ≤a. The following rejection method generates
Xwith density f(von Neumann, 1951; Devroye, 1986; Evans and Swartz,
2000):
Rejection method;
repeat
generate Yfrom the density rand U∼U(0,1), independent;
until Ut(Y)≤κf(Y);
return X=Y.
The number Rof turns into the “repeat” loop is one plus a geometric
random variable with parameter κ/a, so E[R] = a/κ. Thus, we want a/κ ≥1
to be as small as possible, i.e., we want to minimize the area between κf and t.
There is generally a compromise between bringing a/κ close to 1 and keeping
rsimple.
When κf is expensive to compute, we can also use squeeze functions q1and
q2that are faster to evaluate and such that q1(x)≤κf (x)≤q2(x)≤t(x) for
all x. To verify the condition U t(Y)≤κf(Y), we first check if Ut(Y)≤q1(Y),
28 Pierre L’Ecuyer
in which case we accept Yimmediately, otherwise we check if Ut(Y)≥q2(Y),
in which case we reject Yimmediately. The value of κf(Y) must be com-
puted only when Ut(Y) falls between the two squeezes. Sequences of imbed-
ded squeezes can also be used, where the primary ones are the least expensive
to compute, the secondary ones are a little more expensive but closer to κf,
etc.
It is common practice to transform the density fby a smooth increasing
function T(e.g., T(x) = log xor T(x) = −x−1/2) selected so that it is easier
to construct good hat and squeeze functions (often piecewise linear) for the
transformed density T(f(·)). By transforming back to the original scale, we
get hat and squeeze functions for f. This is the transformed density rejection
method, which has several variants and extensions (Devroye, 1986; Evans and
Swartz, 2000; H¨ormann et al., 2004).
The rejection method works for discrete distributions as well; it suffices to
replace densities by probability mass functions.
8.5 Thinning for Point Processes with Time-Varying Rates
Thinning is a cousin of acceptance-rejection, often used for generating events
from a non-homogeneous Poisson process. Suppose the process has rate λ(t)
at time t, with λ(t)≤¯
λfor all t, where ¯
λis a finite constant. One can gener-
ate Poisson pseudo-arrivals at constant rate ¯
λby generating interarrival times
that are i.i.d. exponentials of mean 1/¯
λ. Then, a pseudo-arrival at time tis
accepted (becomes an arrival) with probability λ(t)/¯
λ(i.e., if U≤λ(t)/¯
λ,
where Uis an independent U[0,1]), and rejected with probability 1 −λ(t)/¯
λ.
Non-homogeneous Poisson processes can also be generated by inversion (Brat-
ley et al., 1987). The idea is to apply a nonlinear transformation to the time
scale to make the process homogeneous with rate 1 in the new time scale.
Arrival times are generated in this new time scale (which is easy), and then
transformed back to the original time scale. The method can be adapted to
other types of point processes with time-varying rates.
8.6 The Ratio-of-Uniforms Method
If fis a density over the real-line, κan arbitrary positive constant, and the
pair (U, V ) has the uniform distribution over the set
C=n(u, v)∈R2such that 0 ≤u≤pκf (v/u)o,
then V/U has density f(Kinderman and Monahan, 1977; Devroye, 1986;
Gentle, 2003). This interesting property can be exploited to generate Xwith
density f: generate (U, V ) uniformly over Cand return X=V/U. This is
the ratio-of-uniforms method. The key issue is how do we generate a point
uniformly over C. In the cases where this can be done efficienly, we immediately
have an efficient way of generating X.
Random Number Generation 29
The most frequent approach for generating (U, V ) uniformly over Cis the
rejection method: Define a region C2that contains Cand in which it is easy
to generate a point uniformly (for example, a rectangular box or a polygonal
region). To generate X, repeat: generate (U, V ) uniformly over C2, until it
belongs to C. Then return X=V /U . If there is another region C1contained
in Cand for which it is very fast to check if a point (U, V ) is in C1, this C1can
also be used as a squeeze to accelerate the verification that the point belongs
to C. Several special cases and refinements are described in Devroye (1986);
Gentle (2003); Leydold (2000), and other references given there.
8.7 Composition and Convolution
Suppose Fis a convex combination of several distributions, i.e., F(x) =
PjpjFj(x), or more generally F(x) = RFy(x)dH(y). To generate from F,
one can generate J=jwith probability pj(or Yfrom H), then generate X
from FJ(or FY). This method, called the composition algorithm, is useful for
generating from compound distributions such as the hyperexponential or from
compound Poisson processes. It is also frequently used to design specialized
algorithms for generating from complicated densities. The idea is to partition
the area under the complicated density into pieces, where piece jhas surface
pj. To generate X, first select a piece (choose piece jwith probability pj),
then draw a random point uniformly over that piece and project it to the hor-
izontal axis. If the partition is defined so that it is fast and easy to generate
from the large pieces, then Xwill be returned very quickly most of the time.
The rejection method with a squeeze is often used to generate from some of
the pieces.
A dual method to composition is the convolution method, which can be
used when X=Y1+Y2+··· +Yn, where the Yi’s are independent with
specified distributions. With this method, one just generates the Yi’s and sum
up. This requires at least nuniforms. Examples of random variables that
can be expressed as sums like this include the hypoexponential, Erlang, and
binomial distributions.
8.8 Other Special Techniques
Besides the general methods, several specialized and sometimes very ele-
gant techniques have been designed for commonly used distributions such
as the Poisson distribution with small mean, the normal (e.g., the Box-Muller
method), for generating points uniformly on a k-dimensional sphere, for gen-
erating random permutations, and so on. Details can be found, e.g., in Bratley
et al. (1987); Cheng (1998); Devroye (1986); Fishman (1996); Gentle (2003).
Recently, there has been an effort in developping automatic or black box
algorithms for generating variates from an arbitrary (known) density, and re-
liable software that implements these methods (H¨ormann and Leydold, 2000;
H¨ormann et al., 2004; Leydold and H¨ormann, 2002; Leydold et al., 2002).
30 Pierre L’Ecuyer
Acknowledgements
This work has been supported by the Natural Sciences and Engineering Re-
search Council of Canada (NSERC) Grant No. ODGP0110050, NATEQ-
Qu´ebec grant No. 02ER3218, and a Canada Research Chair to the author.
Wolfgang H¨ormann, Josef Leydold, Fran¸cois Panneton, and Richard Simard
made helpful comments and corrections on an earlier draft. The author has
been asked to write chapters on Random Number Generation for several hand-
books and encyclopedia recently. Inevitably, there is a large amount of dupli-
cation between these chapters.
References
Aiello, W., Rajagopalan, S., and Venkatesan, R. (1998). Design of practi-
cal and provably good random number generators. Journal of Algorithms,
29(2):358–389.
Blum, L., Blum, M., and Schub, M. (1986). A simple unpredictable pseudo-
random number generator. SIAM Journal on Computing, 15(2):364–383.
Bratley, P., Fox, B. L., and Schrage, L. E. (1987). A Guide to Simulation.
Springer-Verlag, New York, second edition.
Brown, M. and Solomon, H. (1979). On combining pseudorandom number
generators. Annals of Statistics, 1:691–695.
Chen, H. C. and Asau, Y. (1974). On generating random variates from an
empirical distribution. AIEE Transactions, 6:163–166.
Cheng, R. C. H. (1998). Random variate generation. In Banks, J., editor,
Handbook of Simulation, pages 139–172. Wiley. chapter 5.
Collings, B. J. (1987). Compound random number generators. Journal of the
American Statistical Association, 82(398):525–527.
Conway, J. H. and Sloane, N. J. A. (1999). Sphere Packings, Lattices and
Groups. Grundlehren der Mathematischen Wissenschaften 290. Springer-
Verlag, New York, 3rd edition.
Couture, R. and L’Ecuyer, P. (1994). On the lattice structure of certain linear
congruential sequences related to AWC/SWB generators. Mathematics of
Computation, 62(206):798–808.
Couture, R. and L’Ecuyer, P. (1996). Orbits and lattices for linear random
number generators with composite moduli. Mathematics of Computation,
65(213):189–201.
Couture, R. and L’Ecuyer, P. (1997). Distribution properties of multiply-
with-carry random number generators. Mathematics of Computation,
66(218):591–607.
Couture, R. and L’Ecuyer, P. (2000). Lattice computations for random num-
bers. Mathematics of Computation, 69(230):757–765.
Random Number Generation 31
Deng, L.-Y. and George, E. O. (1990). Generation of uniform variates from
several nearly uniformly distributed variables. Communications in Statis-
tics, B19(1):145–154.
Deng, L.-Y. and Lin, D. K. J. (2000). Random number generation for the new
century. The American Statistician, 54(2):145–150.
Deng, L.-Y. and Xu, H. (2003). A system of high-dimensional, efficient, long-
cycle and portable uniform random number generators. ACM Transactions
on Modeling and Computer Simulation, 13(4):299–309.
Devroye, L. (1986). Non-Uniform Random Variate Generation. Springer-
Verlag, New York.
Eichenauer-Herrmann, J. (1995). Pseudorandom number generation by non-
linear methods. International Statistical Reviews, 63:247–255.
Eichenauer-Herrmann, J., Herrmann, E., and Wegenkittl, S. (1997). A sur-
vey of quadratic and inversive congruential pseudorandom numbers. In
Hellekalek, P., Larcher, G., Niederreiter, H., and Zinterhof, P., editors,
Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing, vol-
ume 127 of Lecture Notes in Statistics, pages 66–97, New York. Springer.
Evans, M. and Swartz, T. (2000). Approximating Integrals via Monte Carlo
and Deterministic Methods. Oxford University Press, Oxford, UK.
Fincke, U. and Pohst, M. (1985). Improved methods for calculating vectors
of short length in a lattice, including a complexity analysis. Mathematics
of Computation, 44:463–471.
Fishman, G. S. (1996). Monte Carlo: Concepts, Algorithms, and Applications.
Springer Series in Operations Research. Springer-Verlag, New York.
Fushimi, M. (1983). Increasing the orders of equidistribution of the leading
bits of the Tausworthe sequence. Information Processing Letters, 16:189–
192.
Gentle, J. E. (2003). Random Number Generation and Monte Carlo Methods.
Springer, New York, second edition.
Goresky, M. and Klapper, A. (2003). Efficient multiply-with-carry random
number genrators with maximal period. ACM Transactions on Modeling
and Computer Simulation, 13(4):310–321.
Hellekalek, P. and Larcher, G., editors (1998). Random and Quasi-Random
Point Sets, volume 138 of Lecture Notes in Statistics. Springer, New York.
Hellekalek, P. and Wegenkittl, S. (2003). Empirical evidence concerning AES.
ACM Transactions on Modeling and Computer Simulation, 13(4):322–333.
H¨ormann, W. and Leydold, J. (2000). Automatic random variate generation
for simulation input. In Joines, J. A., Barton, R. R., Kang, K., and Fishwick,
P. A., editors, Proceedings of the 2000 Winter Simulation Conference, pages
675–682, Pistacaway, NJ. IEEE Press.
H¨ormann, W. and Leydold, J. (2003). Continuous random variate generation
by fast numerical inversion. ACM Transactions on Modeling and Computer
Simulation, 13(4):347–362.
H¨ormann, W., Leydold, J., and Derflinger, G. (2004). Automatic Nonuniform
Random Variate Generation. Springer-Verlag, Berlin.
32 Pierre L’Ecuyer
Kinderman, A. J. and Monahan, J. F. (1977). Computer generation of ran-
dom variables using the ratio of uniform deviates. ACM Transactions on
Mathematical Software, 3:257–260.
Knuth, D. E. (1998). The Art of Computer Programming, Volume 2: Seminu-
merical Algorithms. Addison-Wesley, Reading, Mass., third edition.
Kronmal, R. A. and Peterson, A. V. (1984). An acceptance-complement ana-
logue of the mixture-plus-acceptance-rejection method for generating ran-
dom variables. ACM Transactions on Mathematical Software, 10:271–281.
Lagarias, J. C. (1993). Pseudorandom numbers. Statistical Science, 8(1):31–
39.
Law, A. M. and Kelton, W. D. (2000). Simulation Modeling and Analysis.
McGraw-Hill, New York, third edition.
L’Ecuyer, P. (1990). Random numbers for simulation. Communications of
the ACM, 33(10):85–97.
L’Ecuyer, P. (1994). Uniform random number generation. Annals of Opera-
tions Research, 53:77–120.
L’Ecuyer, P. (1996a). Combined multiple recursive random number genera-
tors. Operations Research, 44(5):816–822.
L’Ecuyer, P. (1996b). Maximally equidistributed combined Tausworthe gen-
erators. Mathematics of Computation, 65(213):203–213.
L’Ecuyer, P. (1997). Bad lattice structures for vectors of non-successive values
produced by some linear recurrences. INFORMS Journal on Computing,
9(1):57–60.
L’Ecuyer, P. (1998). Random number generation. In Banks, J., editor, Hand-
book of Simulation, pages 93–137. Wiley. chapter 4.
L’Ecuyer, P. (1999a). Good parameters and implementations for com-
bined multiple recursive random number generators. Operations Research,
47(1):159–164.
L’Ecuyer, P. (1999b). Tables of linear congruential generators of different sizes
and good lattice structure. Mathematics of Computation, 68(225):249–260.
L’Ecuyer, P. (1999c). Tables of maximally equidistributed combined LFSR
generators. Mathematics of Computation, 68(225):261–269.
L’Ecuyer, P. (2001). Software for uniform random number generation: Dis-
tinguishing the good and the bad. In Proceedings of the 2001 Winter Sim-
ulation Conference, pages 95–105, Pistacaway, NJ. IEEE Press.
L’Ecuyer, P. (2004). Polynomial integration lattices. In Niederreiter, H.,
editor, Monte Carlo and Quasi-Monte Carlo Methods 2002, pages 73–98,
Berlin. Springer-Verlag.
L’Ecuyer, P. and Andres, T. H. (1997). A random number generator based on
the combination of four LCGs. Mathematics and Computers in Simulation,
44:99–107.
L’Ecuyer, P., Blouin, F., and Couture, R. (1993). A search for good multiple
recursive random number generators. ACM Transactions on Modeling and
Computer Simulation, 3(2):87–98.
Random Number Generation 33
L’Ecuyer, P. and Cˆot´e, S. (1991). Implementing a random number pack-
age with splitting facilities. ACM Transactions on Mathematical Software,
17(1):98–111.
L’Ecuyer, P. and Couture, R. (1997). An implementation of the lattice and
spectral tests for multiple recursive linear random number generators. IN-
FORMS Journal on Computing, 9(2):206–217.
L’Ecuyer, P. and Granger-Pich´e, J. (2003). Combined generators with compo-
nents from different families. Mathematics and Computers in Simulation,
62:395–404.
L’Ecuyer, P. and Hellekalek, P. (1998). Random number generators: Selection
criteria and testing. In Hellekalek, P. and Larcher, G., editors, Random and
Quasi-Random Point Sets, volume 138 of Lecture Notes in Statistics, pages
223–265. Springer, New York.
L’Ecuyer, P. and Lemieux, C. (2000). Variance reduction via lattice rules.
Management Science, 46(9):1214–1235.
L’Ecuyer, P. and Lemieux, C. (2002). Recent advances in randomized quasi-
Monte Carlo methods. In Dror, M., L’Ecuyer, P., and Szidarovszki, F., edi-
tors, Modeling Uncertainty: An Examination of Stochastic Theory, Methods,
and Applications, pages 419–474. Kluwer Academic Publishers, Boston.
L’Ecuyer, P. and Panneton, F. (2002). Construction of equidistributed gen-
erators based on linear recurrences modulo 2. In Fang, K.-T., Hickernell,
F. J., and Niederreiter, H., editors, Monte Carlo and Quasi-Monte Carlo
Methods 2000, pages 318–330. Springer-Verlag, Berlin.