Conference PaperPDF Available

Geometric Optimisation using Karva for Graphical Processing Units

Authors:
0
Computational Science Technical Note CSTN-192
Geometric Optimisation using Karva for Graphical Processing Units
Alwyn V. Husselmann and K. A. Hawick
2013
Population-based evolutionary algorithms continue to play an important role in artifically intelligent systems, but can
not always easily use parallel computation. We have combined a geometric (any-space) particle swarm optimisation
algorithm with use of Ferreira’s Karva language of gene expression programming to produce a hybrid that can
accelerate the genetic operators and which can rapidly attain a good solution. We show how Graphical Processing
Units (GPUs) can be exploited for this. While the geometric particle swarm optimiser is not markedly faster that
genetic programming, we show it does attain good solutions faster, which is important for the problems discussed
when the fitness function is inordinately expensive to compute.
Keywords: CUDA; geometric; genetic programming; gpu; parallel; particle swarm
BiBTeX reference:
@INPROCEEDINGS{CSTN-192,
author = {Alwyn V. Husselmann and K. A. Hawick},
title = {Geometric Optimisation using Karva for Graphical Processing Units},
booktitle = {Proc. 15th International Conference on Artificial Intelligence (ICAI’13)},
year = {2013},
number = {CSTN-191},
pages = {ICA2335},
address = {Las Vegas, USA},
month = {22-25 July},
publisher = {WorldComp},
institution = {Computer Science, Massey University, Auckland, New Zealand},
keywords = {CUDA; geometric; genetic programming; gpu; parallel; particle swarm},
owner = {kahawick},
timestamp = {2013.03.19}
}
This is a early preprint of a Technical Note that may have been published elsewhere. Please cite using the information
provided. Comments or quries to:
Prof Ken Hawick, Computer Science, Massey University, Albany, North Shore 102-904, Auckland, New Zealand.
Complete List available at: http://www.massey.ac.nz/~kahawick/cstn
Technical Report CSTN-192
Geometric Optimisation using Karva for Graphical Processing
Units
A.V. Husselmann and K.A. Hawick
Computer Science, Massey University, North Shore 102-904, Auckland, New Zealand
email: k.a.hawick@massey.ac.nz
Tel: +64 9 414 0800 Fax: +64 9 441 8181
May 21, 2013
Abstract
Population-based evolutionary algorithms continue to play
an important role in artifically intelligent systems, but can
not always easily use parallel computation. We have com-
bined a geometric (any-space) particle swarm optimisation
algorithm with use of Ferreira’s Karva language of gene
expression programming to produce a hybrid that can ac-
celerate the genetic operators and which can rapidly attain
a good solution. We show how Graphical Processing Units
(GPUs) can be exploited for this. While the geometric par-
ticle swarm optimiser is not markedly faster that genetic
programming, we show it does attain good solutions faster,
which is important for the problems discussed when the fit-
ness function is inordinately expensive to compute.
Keywords: CUDA; geometric; genetic programming; gpu; par-
allel; particle swarm
1 Introduction
The advent of combinatorial optimisers saw the birth of ge-
netic programming (GP) [17, 30], which is the term now
widely representative of all algorithms intended to opti-
mise in the search space of programs. John Koza first
invented GP in 1995 alongside the pioneering work of
Kennedy and Eberhart towards the Particle Swarm Opti-
miser (PSO) [3, 15, 16]. Combinatorial and parametric op-
timisers have largely evolved in tandem over time, and
a great deal of research effort has been expended in im-
proving them. These have resulted in a great many vari-
eties of parametric optimisers, including the Cooperative
PSO of van den Bergh [38], space exploration advance-
ments [13] and notably, data-parallel optimisers [10]. Com-
binatorial optimisation in the space of programs have also
gained Cartesian GP [23], Gene Expression Programming
(GEP) [4] and a handful of others. The intentions behind
these algorithms are usually to improve upon one or more
aspects of the class of optimiser in question. GP has been
applied to a variety of problem domains including intrusion
detection [2], robotics [20, 21], geography [22], algorithm
Figure 1: A 3D version of the classic Genetic Programming
test environment named the Santa Fe Ant Trail. This is a vi-
sualisation of 2048 “ants” in their competitive coevolution
in gathering the red food items.
discovery [37], image enhancement [31], as well as data
mining [40] and cooperative mult-agent systems [29].
Genetic Programming (GP) is an optimiser intended to suc-
cessively evolve a generation of candidate solution pro-
grams to solve a particular problem. John Koza’s work
toward GP [17] was arguably inspired by the pioneering
work of John Holland [6] earlier in the 20th centuary on the
canonical genetic algorithm. Most variants of the original
algorithm today make use of various representations, oper-
ators and other techniques such as elitism and alternative
selection mechanisms. GP maintains a population of can-
didate solutions and computes a new population of individ-
uals after some genetic operators have been executed based
on relative fitness evaluations of these individuals. These
operators are most commonly crossover, mutation and se-
lection. These are analogous to biological processes, which
was the primary source of inspiration behind them.
1
One modification worthy of note is Ferreira’s Gene Expres-
sion Programming (GEP) [4] algorithm. A common prob-
lem with GP-based algorithms is in choosing a suitable
representation for candidate solutions. The original made
use primarily of abstract syntax trees (ASTs) in pointer-tree
storage and directly for evolution. Linear GP algorithms [1]
store their programs as linear strings of instructions, exe-
cuted one after the other. This unfortunately suffers from
the Halting Problem [28]. GEP represents individuals as
a string of symbols (codons) but this string is known as a
genotype, since it must be interpreted to obtain a tree-based
phenotype. This has a number of advantages, which are
discussed in more detail in Section 2.
Data-parallelism on commodity-priced hardware such as
Graphical Processing Units (GPUs) have gained much in-
terest in the past few years for their relatively inexpen-
sive and formidable processing power [19]. The concept
of executing Evolutionary Algorithms (EAs) on GPUs is
not new however; many modified algorithms (particularly
EAs) have been proposed for a great variety of problems,
but most of these focus on parallelising the fitness evalua-
tion process [18]. This is commonly the most computation-
ally expensive and time consuming aspect of EAs.
Geometrically unified EAs have recently become a field of
interest due to Moraglio and colleagues [24, 27]. It has re-
sulted in the generalisation of a handful of EAs to arbitrary
search spaces. The only caveat of these geometric algo-
rithms is that the user must be able to provide geometric
operators suitable for the search space under consideration.
Geometric Differential Evolution [26, 27] and Geometric
Particle Swarm Optimisation [25] have already been pro-
posed with varying success. While generalisation of these
excellent parametric optimisers incite keen interest, their
efficacy in comparison to Genetic Programming and other
combinatorial optimisers have not been accurately deter-
mined yet [33].
In this article we seek to explore the dynamics of the Ge-
ometric Particle Swarm Optimiser (GPSO) using a modi-
fied representation named k-expressions (short for karva-
expressions) from Ferreira’s Gene Expression Program-
ming (GEP) algorithm with modified genetic operators to
suit this representation. We then accelerate this algo-
rithm using NVidia CUDA-enabled GPUs. NVidia’s Com-
pute Unified Device Architecture (CUDA) allows access
to an effective and highly efficient means of utilising data-
parallelism to many algorithms including agent-based mod-
elling [8, 14] and other situated agent parallelism [9] prob-
lems as well as optimisation [13].
Section 2 contains more details on the GEP k-
expression representation, including the relevant genetic
operators introduced by Ferreira, as well as a brief intro-
duction on CUDA and GPU-based EAs. We also present a
brief summary of the GPSO. Following this, in Section 3
we describe our algorithm and the combination of GEP and
GPSO as well as the modified genetic operators we use. We
then present some performance and convergence results in
Section 5 and compare our results with a Genetic Program-
 
  
 
 
 
Figure 2: The phenotypic AST built from the genotype rep-
resented by the karva-expression Q-+/-abaa+a-bbacda.
ming algorithm. Finally, we discuss our results and con-
clude in Sections 5 and 6 respectively.
2 Background
As mentioned before, the canonical GP maintains a popula-
tion of candidate solutions. In this work, we have elected to
use the k-expression representation for the elegant simplic-
ity it affords to genetic operators, and its inherent support
for introns, or non-coding sections. The linear nature of
this representation is also very desirable for parallelisation,
even taking into account that an interpreter would still be
necessary.
K-expressions are arranged in a string of fixed length, di-
vided into a head section and a tail section. Function sym-
bols can be of any arity, but can only appear in the head sec-
tion of the expression. This serves the purpose of ensuring
that the genotype is always interpreted into a syntactically
correct phenotype tree. Terminal symbols may appear in
both the head or tail sections. An example k-expression is
shown below. When interpreted, this expression results in
the tree shown in Fig. 2. The tree is built by reading the
expression from left-to-right and filling in the arguments of
each codon, level by level.
012345678901234567
Q-+/-abaa+a-bbacda
It is noteworthy that the terminal symbol ddoes not appear
in the phenotype. This is the mechanism by which introns
are supported. Once all arguments have been supplied in
the tree, the rest of the expression is simply ignored, but not
discarded [4]. A mutation in the tail section which swaps
a terminal for a non-terminal could easily reactivate these
ignored codons.
We now provide a brief overview of the original PSO
2
by Kennedy and Eberhart. The PSO is characteristically
known for maintaining a constantly updated global best so-
lution, as well as a constantly updated personal best solu-
tion for each candidate. The behaviour of the algorithm
in geometrically moving a candidate through space, bi-
ased toward the personal best or global best is reminiscent
of crossover and mutation behaviour, as it also contains a
stochastic component. This random aspect of the algorithm
has also been subject to improvements, as it is the main
space exploration mechanism; without which, there could
only be genetic drift [12]. Eqns. 1 and 2 show the recur-
rence relations which underlie the inertial PSO [35].
The stochastic component is introduced by the ephemeral
random constants rpand rg(typically uniform random de-
viates in (0,1)), and user-defined constants !,pand g.
!is known as the inertial constant, whereas the latter two
constants determine a constructive bias towards either the
personal best pior global best g. Together, these equations
attempt to create convergence, and diffusive space explo-
ration similar to that of Brownian motion [5]. Recent work
has led this toward more computationally expensive Lévy
flights [7, 12, 34] for their improved convergence qualities.
vi+1=!vi+prp(pixi)+grg(gxi)(1)
xi+1=xi+vi(2)
It is prudent to also discuss the Geometric PSO (GPSO) and
the implications that it puts forward. In the simplest imple-
mentation, this algorithm requires that a new mutation op-
erator be defined, and a new crossover operator be defined
for multiple parents; so as to mimic geometric movement
toward a personal best solution and/or the global best so-
lution. Considering that the original algorithm shown in
Eqns. 1 and 2 requires geometric movement and specifi-
cally bias, toward either the global or personal best, it be-
comes clear that this new crossover operator must be able to
bias towards one parent or the other, hence being weighted
in some fashion.
Crossover and mutation in the context of linear genetic pro-
grams was investigated by Togelius and others [36] and sev-
eral possible operators were proposed. The authors con-
cede that significant research still remains in finding the
most appropriate operators, but some effective ones pre-
sented include weighted subtree swap, weighted homolo-
gous crossover and weighted one-point crossover. Homol-
ogous crossover ensures that the common region between
two candidates are kept intact [32]. Togelius and colleagues
reports that common regions can sometimes be too small
for this operator to be constructive [36]. The other two op-
erators are more self-explanatory.
Finally, we now provide a brief summary of GPU-based
simulation, especially using NVidia’s (Compute Unified
Device Architecture) CUDA platform [19]. The CUDA ar-
chitecture arose from a potent arrangement of MIMD and
SIMD computing, initially intended for processing large
quantities of pixel data in parallel. Many researchers have
spent years using “ping-pong” buffering with pixel and
fragment shaders to modify textures in order to accomplish
General-purpose GPU simulation (GPGPU). CUDA makes
this process far more accessible.
Essentially, the GPU is divided into several Streaming
Multi-processors (SMs) with a certain number of “CUDA
cores” which process work units known as “blocks”. Each
of these blocks would be sized by the user, up to a max-
imum (at the time of writing) of 1024 threads. When an
SM executes a block, the threads are divided into groups of
16, which are named “warps”. These warps are the small-
est unit of execution in CUDA. They are subsequently ex-
ecuted in a SIMD fashion on the CUDA cores in each SM.
This arrangement is sometimes known as Single-instruction
Multiple-thread (SIMT).
GPUs generally have some idiosyncratic behaviour regard-
ing memory access coalescence, scoping and penalties,
among other aspects which usually require special consid-
eration. The CUDA memory hierarchy provides a range of
memories with varying access times and scope restrictions,
but we omit an extensive discussion on this for brevity. The
process of executing simulations while taking advantage of
the vast computing power of CUDA usually involves copy-
ing data to the GPU global memory from the host, then
performing the GPU-specific code (“CUDA kernel”), and
finally copying the modified data back. There are more ef-
ficient ways of utilising CUDA-enabled GPUs, such as host
page-locked memory, which remove the need for expensive
memory copies between host and device.
GPU-specific code is written by using special syntax which
NVidia released as an addition to the C language. This syn-
tax is parsed and compiled by the NVidia compiler, and
then the rest of the code is passed to the system C/C++
compiler for normal processing. The result is a C or C++
program with additional non-C syntax which is effectively
removed by the nvcc compiler, and the rest is compiled as
a regular program.
3 Method
Our method for combining CUDA, GEP and GPSO is sum-
marised in Alg. 1. We modify the weighted crossover of
The Particle Swarm Programming algorithm of Togelius
and colleagues [36] to operate on k-expressions by fol-
lowing the multi-parent crossover scheme the authors pro-
posed. This is taken from [36] and shown in Eq. 3.
GX((a, wa),(b, wb),(c, wc)) =
GX((GX((a, wa
wa+wb
),(b, wb
wa+wb
)),w
a+wb),(c, wc))
(3)
GX is the crossover operator, and GX is the multi-parent
crossover operator. It is assumed that wa,wband wcare all
positive and sum to 1. Essentially this equation defines the
weighted, multi-parent crossover as two crossovers, the first
being between aand b, where weights are re-normalised to
3
Algorithm 1 The parallel implementation of the GPSO on
GEP k-expressions .
allocate and initialise enough space for ncandidate pro-
grams
allocate space for random deviates
while termination criteria not met do
call CURAND to fill the random number array with
uniform deviates in the range [0,1)
copy candidates and candidate bests to device
CUDA: compute_argument_maps()
CUDA: interpret/execute programs
CUDA: update food locations/fitness
copy back to host
if end-of-generation then then
CUDA: update candidate bests
CUDA: recombine and mutate programs
replace old programs with new ones
end if
visualise the result
end while
sum to 1, and the second is a crossover with c. Togelius, De
Nardi and Moraglio provide more details on this using con-
vex set theory [36]. See also [24, 25, 27, 36] for more details
on the rationale and mathematical aspect of this procedure.
As can be seen in Alg. 1 we parallelise the majority of com-
putations. In order to interpret the programs, we also need
to compute an “argument map” so as to allow the interpreter
to determine which arguments belong to which functions in
program strings.
Now we have determined the mechanism by which we will
ensure that crossover still maintains its geometric prop-
erties as much as possible. What remains to be deter-
mined is precisely how this will be done on the linear k-
expressions . Ferreira [4] defines one-point crossover as
choosing a crossover site or “pivot”, and then exchanging
symbols about this point to obtain two new candidates. In
order to ensure that this crossover is geometric in the sense
that we can compute a multi-parent one-point crossover and
still be able to bias the result towards one parent candidate
or the other, we must ensure that it is weighted.
Our method for accomplishing this recombination is by us-
ing the !,gand pparameters as the weights (wa,wb
and wc) in Eq. 3. We further define the candidate aas the
current candidate under consideration, bas the correspond-
ing personal best of a, and cas the global best candidate
discovered so far. The fitness values of these are not used
in the crossover process. Notice also that unlike GP, we
do not require selection, other than simply P(crossover),
a probability defined by the user, as in GP. GEP crossover
defines a “donor” and a “recipient” tree, which are chosen
randomly also.
Mutation is simple in comparison. Traditionally, mutation
is derived from initialisation methods such as [39]:
1. Grow method
2. Full method
3. Ramped half-and-half
Typically, mutation is simply a replacement of a subtree by
regrowing it using one of these methods. Point mutation
is not the only space exploration operator, but it is the one
given the most consideration [39] since the work of Koza
[17].
Point mutation is simple for k-expressions , apart from the
only restriction being that a tail-section symbol may not be
swapped for a function symbol. This ensures that the re-
sult of the mutation operator is always a valid candidate.
It is worth noting that a symbol in the head section may
be changed into any other function symbol, regardless of
arity [4]. The size of the head and tail are left as a config-
uration parameter, but can be computed to ensure the head
is maximised. From Ferreira’s work [4] the head and tail
section sizes must satisfy the equation shown in Eq. 4:
t=h(n1) + 1 (4)
The symbols in the equation represent the tail length t, head
length hand the maximum arity possible in the function set
n.
Having described our algorithm we now turn our atten-
tion towards a suitable test platform. The Santa Fe Ant
Trail is a classic problem used for evaluating genetic
programming-based algorithms. Essentially the problem
demands an appropriate combination of two function sym-
bols (IfFoodAhead arity 2, ProgN2 arity 2) and three
terminal symbols (Move,Right and Left) for pur-
suing food particles in a spatial 2D environment. The
IfFoodAhead function executes its first argument if there
is food straight ahead of the particle, and the second argu-
ment if not. The ProgN2 function simply executes both of
its arguments in order. In our case, we have elected to use
a 3D version of the problem. Apart from having Left and
Right terminals, we introduce Up and Down terminals.
To the best of our knowledge, there has been no previous
effort to parallelise the GEP (or indeed a variant of this)
algorithm on GPUs. However, to assist in comparison, we
have compared results from this algorithm against an imple-
mentation of Genetic Programming (GP) with tournament
selection and karva-expressions as program representation.
Our analysis of this algorithm involves two aspects. Firstly,
its ability to converge upon a good solution (preferably the
global optimum), and secondly its ability to utilise the par-
allel architecture of a GPU; hence its wall-clock efficiency.
In our experiments we compare against the aforementioned
GP implementation with k-expressions in terms of conver-
gence and speed. The parameters we used for the GP were:
P(Crossover)=0.8,P(Mutate)=0.1. We use the same
crossover and mutation rates for the modified GPSO, and
for the PSO-specific settings, we used: !=0.1,p=0.6,
g=0.3. As for the simulation itself, we restrict angular
4
Figure 3: Convergence results for the GP, with elitism. The
graph shows the average mean value of each generation,
from 100 independent runs. The error bars represent the
average standard deviation of the 100 runs in each genera-
tion.
Figure 4: Convergence results for the GPSO with elitism.
Each data point has been averaged 100 times in indepen-
dent runs, and the error bars represent the average standard
deviation of each generation.
velocities to 0.1units, and initial velocities are initialised
to between 0.16 and 0.16. In order to use a higher muta-
tion rate, Togelius et al recommend using Elitism, whereby
the best candidate is replicated verbatim into the new pop-
ulation following the genetic operators. This is a common
technique used in EAs to bias the population in a particular
direction. We make use of elitism in both the GP and the
GPSO.
4 Algorithm Convergence Results
Fig. 5 shows the convergence results for the modified
GPSO and the GP. Each data point in all the plots shown
have been averaged 100 times in independent runs. It is
therefore conclusive that the PSO is indeed more able to
find a good solution faster, but if computing fitness for more
than about 23 frames is viable, then the GP is more appro-
priate.
Figs. 3 and 4 show the convergence results for the GP and
GPSO respectively. We experimented with elitism, where
the best individual is copied verbatim into the next genera-
tion, Fig. 5 shows conclusively that elitism allows the algo-
rithms to perform better, albeit marginally.
Each data point of the Average Mean has been represents
an average of 100 means from the same generation number.
From these plots, it is clear that the GPSO has more spread
per generation than the GP, which is not very desirable. The
minimum and maximum values are also shown.
Finally, Fig. 6 shows the average compute time, by genera-
tion, for both the GP and the GPSO. The fitness evaluation
consisted of computing 300 frames of the candidate pro-
grams and gathering fitness results from this. Therefore,
each data point represents the average frame compute time
across each of the 300 frames, and then averaged 100 times
by independent runs. The generation compute times are
also shown, although they are somewhat hidden. While the
first observation seems that the GPSO is faster than the GP,
this is somewhat misleading. Essentially, the plots in Fig. 6
would be completely linear, if all the terminal and function
symbols were of the same complexity.
The average new-generation population compute time for
the GP was 420µsec, and for the GPSO it was 440µsec.
Even though this is not comparable to the fitness evaluation
(340,000µsec), it was still worth the effort, as this must
happen in serial following the fitness evaluation phase. The
function IfFoodAhead has a rough complexity of O(f),
where fis the number of food particles, which would ap-
proach O(fN), should all candidates have one of these
symbols in its program. Of course, the worst case here is
that every candidate consists only of these functions and
enough terminals to satisfy the k-expression’s head and
tail sections. Hypothetically, given a maximum expres-
sion length l=8, and a head length h=3(hence a tail
length of 5), then the maximum number of IfFoodAhead
functions would be 3. Extrapolating from this, assume all
Nparticles were formed like this, then evaluation would
be of complexity O(3fN), which could very well exceed
O(N2).
Therefore, following from this argument, we could perhaps
conjecture that at generation 20, the GP increased its use of
the IfFoodAhead function, while the PSO had reached a
steady equilibrium of a certain number of these functions.
This would seem to agree with our suspicion that the GP is
in fact better in preserving population diversity. In Section
5 we explore this in more depth.
Our attempts to improve the GPSO beyond the results we
see here was met with disappointment. Our parameter
tuning effort for phig,phipand !included normalised
combination of respective scores of particles and also nor-
malised weighted scores, but the best parameters were sim-
ply phig=0.3,phi
p=0.6,!=0.1.
Fine-tuning crossover and mutation probabilities had vary-
ing effects on convergence. Removing the crossover phase
with the global best solution reduced mean scores to
0.2, and similar results were obtained from removing the
crossover with the personal best. Randomising slightly the
crossover point with hand-tuned parameters to aid in diver-
5
Figure 5: Convergence results for the GP and GPSO, as
well as the use of elitism for both. Each data point has been
averaged across 100 independent runs to obtain meaningful
statistical data.
sity did not improve scores at all.
Our results indicate that, at the very least, that the GPSO
operating over k-expressions is appropriate for when the
fitness evaluation is extremely computationally expensive.
Given enough time and compute power, however, the GP
operating on k-expressions is more suited to the problem.
5 Discussion
Evolutionary Algorithms such as the GPSO and the GP
we have compared above frequently involve a very man-
ual parameter-tuning effort in order to ensure an unbiased
comparison. We describe a meta-optimiser (also based on
the PSO) in [11]. We found that the PSO was suitable as
a “super-optimiser” or “meta-optimiser” for fitness evalua-
tions which are of relatively low compute expense. In this
case, fitness evaluation was clearly far more expensive, and
for a “meta-optimiser” to be successful in obtaining good
parameters, it would need to be an optimiser which re-
quires very few frames for a good solution. In this case,
we believe that our meta-optimiser could potentially take
months to obtain a result comparable to hand-tuning. Meta-
optimisers, in general, are notoriously expensive to run.
From our experiments it is not immediately clear why the
GPSO is not as effective as the GP over more than approx-
imately 23 generations. We believe that this may be due to
how diversity in population is managed between the GPSO
and GP. Consider the following as the rationale for this:
Fig. 5 shows a clear change in average mean fitness from
approximately generation 10 for the GPSO. Whereas, for
the GP, a very slight decrease in average mean fitness is
shown. This is reminiscent of local minima in parametric
optimisers. This may also be indicative of the inability of
the GPSO to use extensive diversity to its advantage in es-
caping and reaching the global optimum. Consider also the
artifact shown in Fig. 3 in the standard deviation at gen-
eration number 20. The same position in Fig. 4 is fully
linear.
As for the performance data we present, it would be unwise
Figure 6: Compute times for fitness evaluation, and gener-
ation compute time for GPSO and GP.
to favour the GPSO from the observation in Fig. 6 that the
frame compute time is lower. The slightly higher compute
time does, after all, translate into a higher success rate as
shown in Figs. 3 and 5.
6 Conclusions
We have presented a modified Geometric Particle Swarm
Optimiser (GPSO) searching through the space of Fer-
reira’s k-expressions. We have also compared this against a
Geometric version of the canonical Genetic Programming
method for evolutionary optimisation in the space of k-
expressions. Our results show that the GPSO is not clearly
superior over the GP, however, it is able to attain an ac-
ceptable solution faster, more consistently. This could be
a desirable attribute, especially when the fitness function is
inordinately expensive to compute.
We have also shown that Geometric algorithms such as
the GPSO can be parallelised effectively in both the fit-
ness evaluation phase, and the genetic operator phase (mu-
tation, recombination). CUDA is particularly effective in
this case, as evolutionary algorithms lend themselves well
to data-parallelism.
There is scope for other Evolutionary Algorithms with ge-
ometric modification and parallelisation to be investigated
using similar GPU/Hybrid techniques to those we have pre-
sented.
References
[1] Markus Brameier. On Linear Genetic Programming.
PhD thesis, University of Dortmund, 2004.
[2] Mark Crosbie and Eugene H. Spafford. Applying
genetic programming to intrusion detection. Techni-
cal report, Department of Computer Sciences, Purdue
University, West Lafayette, 1995. AAAI Technical
Report FS-95-01.
[3] R. C. Eberhart and J. Kennedy. A new optimizer us-
ing particle swarm theory. In Proc. Sixth Int. Symp.
on Micromachine and Human Science, pages 39–43,
6
Nagoya, Japan, 1995.
[4] Cândida Ferreira. Gene expression programming: A
new adaptive algorithm for solving problems. Com-
plex Systems, 13(2):87–129, 2001.
[5] G.E.Uhlenbeck and L.S.Ornstein. On the theory of the
Brownian motion. Phys.Rev., 36:823–841, Sep 1930.
[6] J. H. Holland. Adaptation in natural and artificial sys-
tems. Ann Arbor: University of Michigan Press, 1975.
[7] Gang Huang, Yuanming Long, and Jinhang Li. Levy
flight search patterns in particle swarm optimization.
In Seventh International Conference on Natural Com-
putation, 2011.
[8] A. V. Husselmann and K. A. Hawick. Spatial agent-
based modelling and simulations - a review. Tech-
nical Report CSTN-153, Computer Science, Massey
University, Albany, North Shore,102-904, Auckland,
New Zealand, October 2011. In Proc. IIMS Postgrad-
uate Student Conference, October 2011.
[9] A. V. Husselmann and K. A. Hawick. Spatial data
structures, sorting and gpu parallelism for situated-
agent simulation and visualisation. In Proc. Int. Conf.
on Modelling, Simulation and Visualization Methods
(MSV’12), pages 14–20, Las Vegas, USA, 16-19 July
2012. CSREA.
[10] A. V. Husselmann and K. A. Hawick. Levy flights
for particle swarm optimisation algorithms on graph-
ical processing units. Parallel and Cloud Computing,
2(2):32–40, 2013. Submitted to J. Parallel and Cloud
Computing.
[11] A. V. Husselmann and K. A. Hawick. Particle swarm-
based meta-optimising on graphical processing units.
In Proc. Int. Conf. on Modelling, Identification and
Control (AsiaMIC 2013), Phuket, Thailand, 10-12
April 2013. IASTED. CSTN-161.
[12] A. V. Husselmann and K. A. Hawick. Random flights
for particle swarm optimisers. In Proc. 12th IASTED
Int. Conf. on Artificial Intelligence and Applications,
Innsbruck, Austria, 11-13 February 2013. IASTED.
[13] Alwyn V. Husselmann and K. A. Hawick. Paral-
lel parametric optimisation with firefly algorithms on
graphical processing units. In Proc. Int. Conf. on Ge-
netic and Evolutionary Methods (GEM’12), number
CSTN-141, pages 77–83, Las Vegas, USA, 16-19 July
2012. CSREA.
[14] A.V. Husselmann and K.A. Hawick. Simulating
species interactions and complex emergence in mul-
tiple flocks of boids with gpus. In T. Gonzalez, ed-
itor, Proc. IASTED International Conference on Par-
allel and Distributed Computing and Systems (PDCS
2011), pages 100–107, Dallas, USA, 14-16 Dec 2011.
IASTED.
[15] Kennedy and Eberhart. Particle swarm optimization.
Proc. IEEE Int. Conf. on Neural Networks, 4:1942–
1948, 1995.
[16] J. Kennedy and R. C. Eberhart. Particle swarm op-
timization. In Proc. IEEE Int. Conf. on Neural Net-
works, pages 1942–1948, Piscataway, NJ. USA, 1995.
[17] John R. Koza. Genetic programming as a means for
programming computers by natural selection. Statis-
tics and Computing, 4(2):87–112, June 1994.
[18] W. B. Langdon. A many-threaded cuda interpreter
for genetic programming. In Ana Isabel Esparcia-
Alcazar, Aniko Ekart, Sara Silva, Stephen Dignum,
and A. Sima Uyar, editors, Proceedings of the 13th
European Conference on Genetic Programming, Eu-
roGP, pages 146–158. Springer, April 2010.
[19] Arno Leist, Daniel P. Playne, and K. A. Hawick. Ex-
ploiting Graphical Processing Units for Data-Parallel
Scientific Applications. Concurrency and Computa-
tion: Practice and Experience, 21(18):2400–2437, 25
December 2009. CSTN-065.
[20] Sean Luke. Genetic programming produced compet-
itive soccer softbot teams for robocup97. In J. R.
Koza, W. Banzhaf, K. Chellapilla, D. Kumar, K. Deb,
M. Dorigo, D.B. Fogel, M.H. Garzon, D.E. Gold-
berg, H. Iba, and R. Riolo, editors, Genetic Program-
ming 1998: Proceedings of the 3rd annual confer-
ence, pages 214–222. Morgan Kaufmann, San Mateo,
California, 1998.
[21] Sean Luke, Charles Hohn, Jonathan Farris, Gary Jack-
son, and James Hendler. Co-evolving soccer soft-
bot team coordination with genetic programming.
Robocup-97: Robot soccer world cup I, 1:398–411,
1998.
[22] Steven M. Manson. Agent-based modeling and ge-
netic programming for modeling land change in the
southern yucatán peninsular region of mexico. Agri-
culture Ecosystems & Environment, 111:47–62, 2005.
[23] Julian F. Miller and Stephen L. Smith. Redundancy
and computational efficiency in cartesian genetic pro-
gramming. IEEE Transactions on Evolutionary Com-
putation, 10(2):167–174, 2006.
[24] A. Moraglio. Towards a Geometric Unification of
Evolutionary Algorithms. PhD thesis, Computer Sci-
ence and Electronic Engineering, University of Essex,
2007.
[25] A. Moraglio, C. Di Chio, and R. Poli. Geometric par-
ticle swarm optimization. In M. Eber et al, editor, Pro-
ceedings of the European conference on genetic pro-
gramming (EuroGP), volume 4445 of Lecture notes
in computer science, pages 125–136, Berlin, 2007.
Springer.
[26] A. Moraglio and S. Silva. Geometric differential evo-
lution on the space of genetic programs. Genetic Pro-
gramming, 6021:171–183, 2010.
[27] A. Moraglio and J. Togelius. Geometric differential
evolution. In Proceedings of GECCO-2009, pages
1705–1712. ACM Press, 2009.
[28] Michael O’Neill, Leonardo Vanneschi, Steven
Gustafson, and Wolfgang Banzhaf. Open issues in
genetic programming. Genetic Programming and
Evolvable Machines, 11:339–363, 2010.
[29] Panait and Luke. Cooperative multi-agent learning:
The state of the art. Autonomous Agents and Multi-
Agent Systems, 11:387–434, 2005.
[30] R. Poli, W.B. Langdon, and N.F. McPhee. A field
7
guide to genetic programming. lulu.com, 2008.
[31] Riccardo Poli and Stefano Cagnoni. Genetic program-
ming with user-driven selection: Experiments on the
evolution of algorithms for image enhancement. In
Genetic Programming 1997: Proceedings of the 2nd
Annual Conference, pages 269–277. Morgan Kauf-
mann, 1997.
[32] Riccardo Poli and Nicholas F. McPhee. Exact schema
theory for gp and variable-length gas with homol-
ogous crossover. In Proceedings of the Genetic
and Evolutionary Computation Conference (GECCO-
2001), 2001.
[33] Riccardo Poli, Leonardo Vanneschi, William B. Lang-
don, and Nicholas Freitag McPhee. Theoretical re-
sults in genetic programming: the next ten years? Ge-
netic Programming and Evolvable Machines, 11:285–
320, 2010.
[34] Toby J. Richer. The levy particle swarm. In IEEE
Congress on Evolutionary Computation, 2006.
[35] Yuhui Shi and Russel Eberhart. A modified particle
swarm optimizer. In Evolutionary Computation Pro-
ceedings, 1998.
[36] Julian Togelius, Renzo De Nardi, and Alberto
Moraglio. Geometric pso + gp = particle swarm pro-
gramming. In 2008 IEEE Congress on Evolutionary
computation (CEC 2008), 2008.
[37] Sjors van Berkel. Automatic discovery of distributed
algorithms for large-scale systems. Master’s thesis,
Delft University of Technology, 2012.
[38] Frans van den Bergh. An Analysis of Particle Swarm
Optimizers. PhD thesis, University of Pretoria, 2001.
[39] Matthew Walker. Introduction to ge-
netic programming. Downloaded
from http://www.cs.montana.edu/ bwal-
l/cs580/introduction_to_gp.pdf February 5, 2013.
[40] Chi Zhou, Weimin Xiao, Thomas M. Tirpak, and Pe-
ter C. Nelson. Evolving accurate and compact classifi-
cation rules with gene expression programming. IEEE
Transactions on Evolutionary Computation, 7:519–
531, December 2003.
8
... We summarise the genetic programming algorithms of interest to us in this work and give some background and references on their main properties relevant to our test-bed system and visual rendering implementation. Our test-bed algorithms include a data-parallel implementation of Genetic Programming using Karva [24] for representation of programs (which we shall denote K-GP) [25], as well as a data-parallel Geometric Particle Swarm Optimiser also using Karva (which we shall denote using K-GPSO) [26]. We present here a brief overview of these algorithms. ...
... Poli and colleagues [30] have conceded that it is too early to assert the efficacy of Geometric EAs over traditional related algorithms; which has inspired interest in more metrics and visualisations, as well as new algorithms such as the K-GP and K-GPSO. Both the K-GP and K-GPSO are implemented on Graphical Processing Units (GPUs) to improve wall-clock performance, but as we are only concerned with convergence performance, we omit a detailed discussion on this, and instead refer the reader to [25,26,31] for more detail. The fitness function we use is a modified Santa Fe Ant Trail in 3D, where terminal symbols are: Move, Right, Left, Up, Down, and non-terminal symbols (functions) are: IfFoodAhead and ProgN2. ...
... It also indicates that there may be an issue in population diversity. In [26], we discussed the parameters p , g and !, and mentioned that they are best set to static empirically obtained values. This is as opposed to weighted values depending on the fitness values associated with the gBest, lBest, and current candidates. ...
... 6. An investigation of the Karva language in conjunction with the Geometric Particle Swarm Optimiser [121] (see Chapter 4, Section 4.6.1). ...
Thesis
Full-text available
Agent-based models offer valuable simulations of various phenomena found in disciplines such as ecology [1], microbiology [2] and social science [3]. Even the spatial distribution of crime can be simulated [4] using this method. Representing discrete entities in a system as autonomous, interactive and situated in some manner, gives rise to the notion of "bottom-up" modelling, as opposed to "top-down" approaches commonly done using partial differential equations [5]. While not considered a replacement for these, agent-based modelling (ABM) is another potent computational science tool.
Chapter
Geometric unification of Evolutionary Algorithms (EAs) has resulted in an expanding set of algorithms which are search space invariant. This is important since search spaces are not always parametric. Of particular interest are combinatorial spaces such as those of programs that are searchable by parametric optimisers, providing they have been specially adapted in this way. This typically involves redefining concepts of distance, crossover and mutation operators. We present an informally modified Geometric Firefly Algorithm for searching expression tree space, and accelerate the computation using Graphical Processing Units. We also evaluate algorithm efficiency against a geometric version of the Genetic Programming algorithm with tournament selection. We present some rendering techniques for visualising the program problem space and therefore to aid in characterising algorithm behaviour.
Conference Paper
Full-text available
Spatial data partitioning techniques are important for obtaining fast and efficient simulations of N-Body particle and spatial agent based models where they considerably reduce redundant entity interaction computation times. Highly parallel techniques based on concurrent threading can be deployed to further speed up such simulations. We study the use of GPU accelerators and highly data parallel techniques which require more complex organisation of spatial datastructures and also sorting techniques to make best use of GPU capabilities. We report on a multiple-GPU (mGPU) solution to grid-boxing for accelerating interaction-based models. Our system is able to both simulate and also graphically render in excess of 10510610^5 - 10^6 agents on desktop hardware in interactive-time.
Technical Report
Full-text available
Agent-based Modelling (ABM) is a powerful technique for studying complex and emergent phenomena in many areas of science, including engineering, sociology, and other fields where collective macroscopic properties are driven by individual microscopic behaviours. Agents are typically modelled as a short set or rules or encoded microscopic behaviours that each agent in a system will use to interact with its environment. Spatial or situated agents are given a location in a simulated space -- typically in 2 or 3 dimensions and thus they have additional ways to interact with their environment. In this article we review some common approaches to modelling spatial agents and identify two main categories of models based on continuous space or discrete-mesh space. We explore two representative models for flocking animats and for mood propagation and use these to discuss how fast and efficient spatial ABMs can be simulated in order to explore multi-scale collective phenomena. We consider data-parallel simulation technologies and approaches such as the use of graphical processing units as accelerators and other computational optimisation ideas.
Article
Full-text available
Particle Swarm Optimisation (PSO) is a powerful algorithm for space search problems such as parametric optimisation. Particles with Levy flights have a long-tailed probability of outlier jumps in the problem space that provide a good compromise between local space exploration and local minima avoidance. Generating many particles and their trajectories with Levy random deviates is computationally expensive, however. We present a data-parallel algorithmic implementation of Levy flighted particle swarm optimisation and show how it makes use of accelerators such as graphical processing units (GPUs). We discuss the computational tradeoffs, performance achievable using GPUs, and the scalability of such an approach using various uni-modal and multi-modal test functions in a range of dimensions.
Conference Paper
Full-text available
Optimisation (global minimisation or maximisation) of complex, unknown and nondifferentiable functions is a difficult problem. One solution for this class of problem is the use of meta-heuristic optimisation. This involves the systematic movement of n-vector solutions through n- dimensional parameter space, where each dimension corresponds to a parameter in the function to be optimised. These methods make very little assumptions about the problem. The most advantageous of these is that gradients are not necessary. Population-based methods such as the Particle Swarm Optimiser (PSO) are very effective at solving problems in this domain, as they employ spatial exploration and local solution exploitation in tandem with a stochastic component. Parallel PSOs on Graphical Processing Units (GPUs) allow for much greater system sizes, and a dramatic reduction in compute time. Meta-optimisation presents a further super-optimiser which is used to find appropriate algorithmic parameters for the PSO, however, this practice is often overlooked due to its immense computational expense. We present and discuss a PSO with an overlaid super-optimiser also based on the PSO itself.
Conference Paper
Full-text available
Parametric Optimisation is an important problem that can be tackled with a range of bio-inspired problem space search algorithms. We show how a simplified Particle Swarm Optimiser (PSO) can exploit advanced space exploration with Levy flights, Rayleigh flights and Cauchy flights, and we discuss hybrid variations of these. We present implementations of these methods and compare algorithmic convergence on several multi-modal and uni-modal test functions. Random flights considerably enhance the PSO and the Levy flight gives good balance between local space exploration and local minima avoidance. We discuss computational tradeoffs involved in generating such flights.
Conference Paper
Full-text available
Parametric optimisation techniques such as Particle Swarm Optimisation (PSO), Firefly algorithms (FAs), genetic algorithms (GAs) are at the centre of attention in a range of optimisation problems where local minima plague the parameter space. Variants of these algorithms deal with the problems presented by local minima in a variety of ways. A salient feature in designing algorithms such as these is the relative ease of performance testing and evaluation. In the literature, a set of well- defined functions, often with one global minimum and several local minima is available to evaluate the convergence of an algorithm. This allows for simultaneously evaluating performance as well as the quality of the solutions calculated. We report on a parallel graphical processing unit (GPU) implementation of a modified Firefly algorithm, and the associated performance and quality of this algorithm. We also discuss spatial partitioning techniques to dramatically reduce redundant entity interactions introduced by our modifications of the Firefly algorithm.
Chapter
Part I of this book has used several existing GENESIS simulations to introduce some of the theory underlying neural modeling. In Part II, we will build upon this background, using the GENESIS script language to create our own simulations. We will begin by simulating a simple neural compartment like that described in the first two sections of Chapter 2. Before proceeding with this tutorial, you may find it useful to review that material.
Article
With a method first indicated by Ornstein the mean values of all the powers of the velocity u and the displacement s of a free particle in Brownian motion are calculated. It is shown that u−u0exp(−βt) and s−u0β[1−exp(−βt)] where u0 is the initial velocity and β the friction coefficient divided by the mass of the particle, follow the normal Gaussian distribution law. For s this gives the exact frequency distribution corresponding to the exact formula for s2 of Ornstein and Fürth. Discussion is given of the connection with the Fokker-Planck partial differential equation. By the same method exact expressions are obtained for the square of the deviation of a harmonically bound particle in Brownian motion as a function of the time and the initial deviation. Here the periodic, aperiodic and overdamped cases have to be treated separately. In the last case, when β is much larger than the frequency and for values of t≫β−1, the formula takes the form of that previously given by Smoluchowski.