A comparative study of some pseudorandom number generators

Abstract

We present results of a test program of a group of pseudorandom number generators which are commonly used in the applications of physics, in particular in Monte Carlo simulations. The generators include public domain programs, manufacturer installed routines and a random number sequence produced from physical noise. We start by traditional standard tests, followed by detailed bit level and visual tests. The computational speed of various algorithms is also scrutinized. Our results allow direct comparisons between the properties of different generators, as well as an assessment of the efficiency of various test methods. Together with recently developed application specific tests, this information provides a good criterion to choose the best generator among the tested ones for a given problem.

Full text

Page 1

arXiv:hep-lat/9304008v2 10 Aug 1993
A Comparative Study of
Some Pseudorandom Number Generators
I. Vattulainen1, K. Kankaala1,2, J. Saarinen1, and T. Ala-Nissila1,3
1Department of Electrical Engineering
Tampere University of Technology
P.O. Box 692
FIN - 33101 Tampere, Finland
2Centre for Scientific Computing
P.O. Box 405, FIN - 02100 Espoo, Finland
3Research Institute for Theoretical Physics
P.O. Box 9 (Siltavuorenpenger 20 C)
FIN - 00014 University of Helsinki, Finland
Abstract
We present results of an extensive test program of a group of pseudo-
random number generators which are commonly used in the applications of
physics, in particular in Monte Carlo simulations. The generators include
public domain programs, manufacturer installed routines and a random num-
ber sequence produced from physical noise. We start by traditional statistical
tests, followed by detailed bit level and visual tests. The computational speed
of various algorithms is also scrutinized. Our results allow direct comparisons
between the properties of different generators, as well as an assessment of the
efficiency of the various test methods. This information provides the best
available criterion to choose the best possible generator for a given problem.
However, in light of recent problems reported with some of these generators,
we also discuss the importance of developing more refined physical tests to
find possible correlations not revealed by the present test methods.
PACS numbers: 02.50.-r, 02.50.Ng, 75.40.Mg.
Key words: Randomness, random number generators, Monte Carlo simulations.
1

Page 2

1Introduction
“Have you generated any
new random numbers today?”
J. K¨ apyaho
Long sequences of random numbers are currently required in numerous applications,
in particular within statistical mechanics, particle physics, and applied mathematics.
The methods utilizing random numbers include Monte Carlo simulation techniques
[5], stochastic optimization [1], and cryptography [6, 12, 61], all of which usually
require fast and reliable random number sources. In practice, the random num-
bers needed for these methods are produced by deterministic rules, implemented as
(pseudo)random number generator algorithms which usually rely on simple aritme-
thic operations. By their definition, the maximum - length sequences produced by
all these algorithms are finite and reproducible, and can thus be “random” only in
some limited sense [8, 9].
Despite the importance of creating good pseudorandom number generators, fairly
little theoretical work exists on random number generation. Thus, the properties of
many generators are not well understood in depth. Some random number generator
algorithms have been studied in the general context of cellular automata [8], and
deterministic chaos [4]. In particular, number theory has yielded exact results on the
periodicity and lattice structure for linear congruential and Tausworthe generators
[11, 40, 30, 55]. These results have led to theoretical methods of evaluating the
algorithms, the most notable being the so called spectral test. However, most of
these theoretical results are derived for the full period of the generator while in
practice the behavior of subsequences of substantially shorter lengths is of particular
importance in applications. In addition, the actual implementation of the random
number generator algorithm may affect the quality of its output. Thus, in situ tests
of implemented programs are usually needed.
Despite this obvious need for in situ testing of pseudorandom number generators,
only relatively few authors have presented results to this end [32, 33, 49, 42]. Most
likely, there are two main reasons for this. The first is the persistence of underlying
fundamental problems in the actual definitions of “randomness” and “random” se-
quences which have given no unique practical recipe for testing a finite sequence of
numbers [31]. Thus various authors have developed an array of different tests which
mostly probe some of the statistical properties of the sequences, or test correlations
e.g. on the binary level. Recently, Compagner and Hoogland [8] have presented a
2

Page 3

somewhat more systematic approach to randomness as embodied in finite sequences.
They propose testing the values of all possible correlation coefficients of an ensem-
ble of a given sequence and all of its “translations” (iterated variations) [10], a task
which nevertheless appears rather formidable for practical purposes. We are not
aware of any attempts to actually carry out their program. The second reason is
probably more practical, namely the gradual evolution of improved pseudorandom
number generator algorithms, which has led to a diversity of generators available in
computer software, public domain and so on. For many of these algorithms (and
their implementations) only a few rudimentary tests have been performed.
In this work we have undertaken an extensive test program [58] of a group of
pseudorandom number generators, which are often employed in the applications
of physics.These generators which are described in detail in Section 2 include
public domain programs GGL, RANMAR, RAN3, RCARRY and R250, a library
subroutine G05FAF, manufacturer installed routines RAND and RANF, and even
a sequence generated from physical noise (PURAN II). Our strategy is to perform
a large set of different tests for all of these generators, whose results can then be
directly compared with each other. There are two main reasons for this. Namely,
there is a difficulty associated with most quantitative tests in the choice of the test
parameters and final criteria for judging the results. Thus, we think that full compar-
ative tests of a large group of generators using identical test parameters and criteria
can yield more meaningful results, in particular when there is a need for a reliable
generator with a good overall performance. Second, performing a large number of
tests also allows a comparison of exactly how efficient each test is in finding certain
kinds of correlations.
As discussed in Section 3, we first employ an array of standard statistical tests,
which measure the degree of uniformity of the distribution of numbers, as well as
correlations between them. Following this, we perform a series of bit level tests, some
of which should be particularly efficient in finding correlations between consecutive
bits in the random number sequences.
pictures of random numbers and their bits on a plane. Finally, for the sake of
completeness we have also included a relative performance test of the generators
in our results. A complete summary of the test results is presented in Section 4.
As our main result we find three generators, namely GGL, G05FAF, and R250
with an overall best performance in all our tests, although some other generators
such as RANF and RANMAR perform almost as convincingly. We also find that
the bit level tests are most efficient in finding local correlations in the random
Third part of our testing utilizes visual
3

Page 4

numbers, but do not nevertheless guarantee good statistical properties, as shown in
the case of RCARRY. Our results also show that visual tests can indeed reveal spatial
correlations not clearly detected in the quantitative tests. Our work thus provides a
rather comprehensive test bench which can be utilized in choosing a random number
generator for a given application.However, choosing a “good quality” random
number generator for all applications may not be trivial as discussed in Section
5, in light of the recent results reporting anomalous correlations in Monte Carlo
simulations [14, 23] using the here almost impeccably performing R250. Thus, more
physical ways of testing random number sequences are probably needed, a project
which is currently underway [59].
4

Page 5

2Generation of Random Numbers
“The generation of random numbers
is too important to be left to chance.”
R. Coveyou
Pseudorandom number sequences needed for high speed applications are usually
generated at run time using an algorithm which often is a relatively simple nonlinear
deterministic map. The implementation of the corresponding recurrence relation
must also ensure that the stream of numbers is reproducible from identical initial
conditions. The deterministic nature of generation means that the designer has to be
careful in the choice of the precise relationship of the recursion, otherwise unwanted
correlations will appear as amply demonstrated in the literature [8].
However, even the best generator algorithm can be defeated by a poor computer
implementation. Whenever an exact mathematical algorithm is translated into a
computer subroutine, different possibilities for its implementation may exist. Only
if the operation of a generator can be exactly specified on the binary level, has
the implementation a chance to be unambiguous; otherwise, machine dependent
features become incorporated into the routine. These include finite precision of real
numbers, limited word size of the computer, and numerical accuracy of mathematical
functions. Furthermore, it would often be desirable that the implemented routine
performed identically in each environment in which it is to be executed, i.e. it would
be portable.
Some of the desired properties of good pseudorandom number generators are easily
defined but often difficult to achieve simultaneously. Namely, besides good “ran-
domness” properties portability, repeatability, performance speed, and a very long
period are often required. Ideally, a random number generator would be designed
for each application, and then tested within that application to ensure that the in-
evitable correlations that do exist in a deterministic algorithm, cause no observable
effects. In practice, this is seldom possible, which is another reason why extensive
tests of pseudorandom number generators are needed.
Most commonly used pseudorandom number generator algorithms are the linear
congruential method, the lagged Fibonacci method, the shift register method, and
combination methods. A special case are nonalgorithmic or physical generators which
are used for creating a non - reproducible sequence of random numbers. These are
usually based on “random” physical events, e.g. changes in physical characteristics
5