Content uploaded by Giovanni Squillero
Author content
All content in this area was uploaded by Giovanni Squillero on Nov 04, 2015
Content may be subject to copyright.
DRAFT: THIS IS NOT THE FINAL COPY
THE PUBLISHED PAPER HAS BEEN EDITED AND REFORMATTED
DOI: 10.1109/CEC.2003.1299947
Exploiting Co-Evolution and a Modified Island Model to
Climb the Core War Hill
F. Corno, E. Sanchez, G. Squillero
Abstract- In this paper, Core War, a very peculiar game
popular in mid 80’s, is exploited as a benchmark to improve
the μGP, an evolutionary algorithm able to generate Tour-
ing-complete, realistic assembly programs. Two techniques
were analyzed: co-evolution and a modified island model.
Experimental results showed that the former is essential in
the beginning of the evolutionary process, but may be de-
ceptive in the end. Differently, the latter enables focusing
the search on specific region of the search space and lead to
dramatic improvements. The use of both techniques to help
the μGP in its real task (test program generation for micro-
processor) is currently being evaluated.
1 Introduction
While playing has an undeniably role in children develop-
ment, it also helps adult researchers. For instance, in the
game theory the idea of competitors playing a game is used
to understand animal, human, and even gene behavior: any
competitive environment, such as economy, can be viewed
as a game and the metaphor used to explain cooperative and
uncooperative strategies. Also, in the field of evolutionary
computation, games are commonly used to study the mech-
anisms of evolution: a specific game is played many times
by players cultivated in populations to maximize their per-
formances. At the end of the experiments, the results at-
tained by the players measure, indirectly, the effectiveness
of the evolutionary methodologies, selection schemes and
operators adopted.
Similarly, in this paper a game is used to improve the evolu-
tionary approach called µGP. A distinctive game is used to
tweak the method and test new enhancements, while the
final goal is to exploit the approach against a completely
different problem.
µGP is an evolutionary method for generating Touring-
complete, realistic assembly programs. The evolved pro-
grams may take full advantage of the assembly syntax, ex-
ploit the different addressing modes and instruction set
asymmetries. The method is extremely general, but it was
originally devised to generate test programs for micropro-
cessors. Roughly speaking, a test program is an assembly
program devised to extract information from the machine
that executes it, rather than calculating some function or a
value. Test programs may be used to validate the correct-
ness of a microprocessor design, in a process similar to de-
bugging, or to check its functionality after production.
Evolving effective test programs is a challenging task. First
of all, each single evaluation of the fitness function requires
simulating the hardware model of the microprocessor when
it is executing the test program. This process may require
hours even on a fast workstation. Additionally, the problem
is complex and the fitness landscape is completely un-
known, but probably strongly rugged and deceptive.
This paper focuses on Core War, a game played by two or
more programs written in an assembly language called red-
code and run in a virtual computer called Memory Array
Redcode Simulator (MARS). The object of the game is to
cause all processes of the opposing programs to terminate,
leaving the winner in sole possession of the machine. Red-
code programs are usually called “warriors”.
Next section better details the Core War game and the pre-
vious evolutionary attempt to devise effective warriors. Sec-
tion 3 illustrates the proposed framework. Section 4 ex-
plains why Core War was chosen, describes the adopted
enhancements and reports some experimental results. Sec-
tion 5 concludes the paper
2 Core Wars
A game where two or more programs try to kill opponents
by overwriting them was devised by Victor Vyssotsky,
Robert Morris Sr., and Dennis Ritchie in the early 1960s at
Bell Labs. Its name was “Darwin”. About 20 years later, in
1984, D. G. Jones and A. K. Dewdney wrote the “Core War
Guidelines” formalizing the modern Core War. In the same
year, Dewdney’s column in Scientific American [1] popular-
ized the game drawing a huge interest from both the scien-
tific community and hobbyists.
Core War competitions are usually called hills. Classical
hills are repository of N (usually 20) warriors. When a new
program is submitted, it plays a certain number (usually
100) of one-on-one games against each of the N other pro-
grams currently on the hill. The new warrior gets 3 points
for each win and 1 point for each tie, then all scores are up-
dated to reflect battles (the existing programs do not replay
each other, but their previous battles are recalled) and pro-
grams are ranked from high to low. Finally, the least one is
pushed off the hill.
The International Core War Society (ICWS) was also estab-
lished in 1984 for the creation and maintenance of Core War
standards and for running of Core War tournaments. There
have been 6 annual tournaments and 2 standards (ICWS’86
and ICWS’88) and several different servers hosted Core
War tournaments, such as the King of the Hill (KOTH) and
the Pizza Hill. Several authors started competing in the
game, either directly, writing sharp redcode warriors; or
automatically, inducing new ones. At the same time, other
researchers focused on the metaphor of competing organ-
isms in a synthetic environment [2]. A big community sud-
denly has appeared. In 1994 the ICSW proposed the new
Core War standard named ICSW’94. However, the golden
era of Core Wars was almost ended and interest was swiftly
decreasing. Today, the official Core Wars FAQ still main-
tains that the ICSW’94 “is currently being evaluated”.
2.1 Evolutionary Core Wars
Since its appearance, Core War attracted the interest from
the evolutionary-algorithm community. In 1991, John Perry
showed how random code can evolve into successful Core
War warriors in only a few generations1. His paper targeted
the evolution of predatory behavior in computer programs.
More recently, several interesting new approaches have
been devised2. In most of the proposed methods, the first
population of warriors is randomly generated. Then, since
redcode is a very simple and completely orthogonal assem-
bly language (all addressing modes are utilizable with all
instructions, and all instructions are exactly in the same
format), individuals are evolved using very simple genetic
operators.
Major contributions include: Terry Newton’s Redmaker3, an
experimental Core Warrior evolver based on a grid-shaped
1 John Perry, “Core Wars Genetics: the evolution of preda-
tion”, available at: http://www.soberit.hut.fi/tik-76.115/
96-97/palautu-set/groups/DSM/ma/documents/
cwbasics.html
2 http://corewars.sourceforge.net/cgi-bin/control.py?
action=links
3 http://www.tl.infi.net/~wtnewton/corewar/evol/
evolution pool; Martin Ankerl’s Yace4 (Yet Another Core-
war Evolver) and Dave Hillis’s RedRace5 (Red Queen’s
Race). Both approaches start with a random population. In
Yace, warriors fight against each other and the losers are
replaced by slightly modified versions of the winners. In
RedRace, on the other hand, all warriors in the population
compete against all warriors on a target hill. RedRace also
includes sharp techniques for speeding-up the search pro-
cess, saving and restoring effective warriors (Valhalla and
Resurrection) and handling multiple populations.
In 2002, Brian Blaha and Don Wunsch presented a study on
automatic assembly program optimization using Core War
as a case study [3]. They investigate different techniques
and attain interesting results, although, in authors’ own
words, probably unable to devise really effective warriors.
3 µGP
µGP is an evolutionary approach for generating Touring-
complete programs to optimize the solution of a given prob-
lem. The environment can be used for different assembly
languages and diverse metrics. µGP is composed of three
parts: the evolutionary core, the instruction library and the
external evaluator (see Figure 1).
evolutionary
core
test
program
external
evaluator
Fitness
instruction
library
evolutionary
core
test
program
external
evaluator
Fitness
instruction
library
Figure 1:
µ
GP System Architecture
The evolutionary core cultivates a population of individuals.
It uses auto-adaptation mechanisms, dynamic operator prob-
abilities, dynamic operator strength, and variable population
size. The instruction library is used to map individuals to
valid assembly language programs. It contains a highly con-
cise description of the assembly syntax or more complex,
parametric fragments of code. Finally, the external evaluator
simulates the assembly program, providing the necessary
feedback to the evolutionary core.
4 http://students.fhs-hagenberg.ac.at/se/se00001/ yace.html
5 http://users.erols.com/dbhillis/
The instruction library defines the assembly language syn-
tax. It enumerates a set of macros, i.e., fragments of code of
arbitrary length with an arbitrary number of parameters rep-
resented as “$n”. A sharp use of the instruction library al-
lows enumerating all valid instructions in a very compact
way. Parameters may be used to encode operands, instruc-
tions and addressing modes, as in Figure 2.
.macro
.probability 6
$1.$2 $3$4+$5, $6$7+$8
.parameter constant add sub div mul mod nop
.parameter constant a b ab ba f x i
.parameter constant # $$ @ * { } < >
.parameter inner_generic_label
.parameter integer 0 799
.parameter constant # $$ @ * { } < >
.parameter inner_generic_label
.parameter integer 0 799
.endmacro
Figure 2: A Macro Describing Several Redcode Instructions
More formally, the instruction library supports different
types of parameters:
• Integer: represents a numeric value. The valid range
must be specified.
• Constant: represents a string inside a predefined set.
• Inner Forward Label: a forward reference.
• Inner Backward Label: a backward reference.
• Inner Generic Label: a generic reference.
The user may specify explicitly the probability that a ran-
dom node encodes a specified macro. This is necessary to
have equally probable assembly instructions: two compact
macros may encode a different number of elementary in-
structions, and if the macros are equally probable, the as-
sembly instructions are not.
Individuals are represented as directed graphs, where the
structure of the graph encodes the syntactic appearance of
the program and each node corresponds to a macro of the
instruction library (Figure 3).
The µGP evolutionary core exploits a generational strategy.
A population of µ individuals is stored and in each genera-
tion λ genetic operators are applied. Operators are chosen
according to their activation probabilities. Since each genet-
ic operator produces a variable offspring, the number of
individuals at the end of each generation it is not fixed. Af-
ter offspring generation the population is sorted, the best µ
individuals are selected for survival and transferred to the
next generation. The evolutionary core implements both a
crossover (recombination) operator and different mutation
operators. All activation probabilities are variable and are
automatically auto-adapted by the algorithm.
Further details can be found in [4].
4 µCoreWar
As stated before, test program generation is a hard task. The
fitness function is computationally severe and running a
massive set of experiments is practically infeasible. Thus,
evolutionary operators and strategies are tuned resorting
mainly to engineers’ experience and intuition. This situation
is far from ideal.
sub $1, $2
jz
$3
R1
12
$1:
$2:
$3:
jmp $1$2, $3$4
.parameter type ADDR_MODES
.parameter inner_label
.parameter type INC_DEC
.parameter type INT
$
12
$1:
$2:
$3: >
$3:
$4: 35
Figure 3: Individual Representation
Devising a sharp test program for exciting all behaviors in a
microprocessor and devising a strong warrior able to exploit
opponents’ flaws are somewhat similar. Both tasks require
very specific instructions, able to excite very specific corner
cases. For both domains, a small mutation is likely to trans-
form an effective program in a completely useless one.
However, the MARS is extremely fast compared to simulat-
ing a detailed hardware description of a complex micropro-
cessor executing a program. Thus, it is possible to run con-
siderable set of experiments for investigating on new evolu-
tionary strategies.
The modified prototype of the µGP exploited for generating
redcode warriors has been called µCoreWar. Enhancements
developed for climbing Core War hills will be integrated in
the µGP and tested on test program generation tasks.
Experiments were performed trying to climb the evolved
tiny hill hosted by sourceforge6. The hill contains the 20
strongest evolved warriors available on the Internet. Re-
markably, they have been all generated using Redmaker,
Yace, RedRace or customized versions of the these three
programs (except “the 99280th marsh herring”, a warrior
devised by a Jaska Tyni’s program).
4.1.1 Co-Evolution
6 http://corewars.sourceforge.net/
Co-evolution is usually defined as evolution involving suc-
cessive changes in two or more ecologically interdependent
species that affect their interactions. In the Evolutionary
Computation field, co-evolution has been advocated to be
particularly useful evolving players [6].
In the evolutionary Core War, the simplest form of co-
evolution can be achieved by co-evolving a population of
warriors and a hill. This technique has some advantages and
one main disadvantage.
First of all, co-evolving the hill is essential in the first gen-
erations of the programs. Random programs are unlike to be
effective warriors, thus their score would be invariably zero
on a regular hill, preventing any evolution.
On the other hand, playing against a co-evolved hill may
bias the fitness function, leading to programs that achieve
good results on the test hill, but poor scores on the target
one.
Another possible solution would have been to include dum-
my warriors in the target hill. Although effective, this solu-
tion targets specifically the Core War environment and can-
not be easily translated to the test program generation do-
main. Thus it was discarded, as were discarded all other
Core War-specific enhancements.
4.1.2 Modified Island Model
In the Island Model, sometimes called Migration Model or
Coarse Grained Model, the population is partitioned into
several distinct subpopulations. Then, these subpopulations
have extensive periods of isolated evolution occasionally
interspersed with migration. Several authors exploited this
mechanism, showing its advantages over canonical single-
population evolutionary algorithms [5].
Different to the standard island model, in the µCoreWar,
different population are used to independently explore dif-
ferent region of the search space. Different populations use
instruction libraries with identical set of macros, but with
different macro probabilities.
Thus, each sub population is polarized to explore a specific
region of the search space. More specifically, three popula-
tions exploit three strategies, while a forth one is not polar-
ized (Table 1). The first population uses all available direct
and indirect addressing modes, with optional pre-increment
and decrement. However, it favors the use of large integer
constants ([0, 799]). The second population is identical to
the first one, except it favors the use of small integer con-
stants ([-8, +8]). The third population, conversely, uses
more probably the MOV (move) and the SPL (split) instruc-
tions.
Population
Strategy
STR1
Favor small integer constants
STR2
Favor large integer constants
STR3
Favor MOV and SPL operations
ALL
No special polarization
Table 1: Strategies
These populations evolve independently for N = 60 genera-
tions, using as parameters µ = 20 and λ = 80. Then, the best
5 individuals of each population are duplicated in the three
others population, and the process is iterated. Parameter N is
reduced during the experiments.
0
1000
2000
3000
4000
5000
020 40 60 80 100
Genera tions
Score
ALL
Simple
STR1
STR2
STR3
Figure 4: Results Attained by Different Approaches
(generations 1 through 100)
When compared to a conventional approach, all these strat-
egies cause a reduction in performances. Figure 4 shows the
results attained by the best individual with the different in-
struction libraries in the first 100 generations. The line la-
beled Simple reports the results of the standard approach,
with no migrations and where all instructions are given the
same probability. Lines labeled STR1, STR2 and STR3
shows results of the three strategies with polarized instruc-
tion libraries. In the first 60 generations, all sub-populations
achieve poor results. Line labeled ALL reports the result of
the forth population, since it uses exactly the same parame-
ters and the same instruction library of the Simple approach,
results before the first migration are identical. However,
values dramatically rise after it.
More interestinglu, Figure 5 reports the result for the com-
plete optimization process of 250 generations. The effect of
co-evolution can be easily seen in all figures. The fitness
values of all experiments are monotonic with respect to the
co-evolved hills, but the scores against the target hill some-
times decrease.
In particular, the first effect of each migration is a reduction
of the fitness values. However, the new genetic material
inserted in the population is quickly exploited to increase
the best warrior.
3000
3400
3800
4200
4600
5000
050 100 150 200 250
Generations
Score
ALL
Simple
Figure 5: Results Attained by Different Approaches
(generations 1 through 250)
There is a direct applicability of these strategies for test
programs generation. All modern microprocessors, such as
SPARC [7], have two different modes of operations: real
(or user) and privileged (or supervisor). The former is in-
tended for executing normal user programs and prohibits
potentially dangerous operations, such as modification of
the processor status register (PSR). The latter, on the other
hand, is intended for executing the operating system code.
Usually, these two modes are physically distinct in the mi-
croprocessor and uses different resources such as different
stack registers. In order to check the correctness of a design,
the μGP must be able to generate appropriate code for both
modes.
As the first experiments showed, this result can be more
efficiently achieved using two sub-populations with polar-
ized instruction library. The first targeting the real mode,
and the second one targeting the protected mode.
On the other hand, the use of a co-evolution mechanism is
currently under study. However, the first results attained
exacerbating the fitness function during evolution are prom-
ising.
Finally, it must be noted that the evolved warriors behaved
reasonably well, getting scores between 4,800 and 5,000,
but no programs was ever able to enter on the hill. Indeed,
the best program on June 2003, “rdrc: Dementia Minni”,
was evolved by Dave Hillis with Red Race and scored 5,604
points. “Eco 1.3/1020/500”, the 20th one, scored 5,239
points.
5 Conclusions
This paper presented an evolutionary for devising Core War
warriors. Core War has been selected as a benchmark prob-
lem, while the final goal of the research is to explore new
evolutionary strategies applicable to a real problem: test
program generation for microprocessor.
Two techniques where used for the task: co-evolution and a
modified island model.
Experimental results showed that the former is essential in
the beginning of the evolutionary process, but may be de-
ceptive in the end. The latter, on the other hand, enables
exploring different region of the search space and lead to
effective improvements.
The use of both techniques during test program generation
for microprocessor is currently being evaluated.
6 Acknowledgments
Authors wish to thanks to Martin Ankerl, Brian Blaha, Ken
Paul Dolan, Dave Hillis, Terry Newton and Donald Wunsch
for their friendly advices and insightful comments.
7 Bibliography
[1] A. K. Dewdney, “Computer recreations: In the game
called Core War hostile programs engage in a battle
of bits”, Scientific American, 250(5), pp. 14-22, 1984
[2] T. S. Ray, “An evolutionary approach to synthetic
biology: Zen and the art of creating life”, Artificial
Life 1 (1/2), pp. 195-226, 1994
[3] B. Blaha, D. Wunsch, “Evolutionary programming to
optimize an assembly program”, CEC’02: Proceed-
ings of the Congress on Evolutionary Computation,
pp 1901-1903, 2002
[4] F. Corno, G. Squillero, “An Enhanced Framework for
Microprocessor Test-Program Generation”, EU-
ROGP2003: 6th European Conference on Genetic
Programming, Essex (UK), April 14-16, 2003, pp.
307-315
[5] F. Fernández, M. Tomassini, J. M. Sánchez, “Exper-
imental study of Isolated Multipopulation Genetic
Programming”, Proceedings of the 26th Annual Con-
ference of the IEEE Industrial Electronics Society,
IEEE Press, Vol. 1697, pp. 2672-2677, 2000
[6] J. E. Davis, G. Kendall, “An investigation, using co-
evolution, to evolve an Awari player”, Proceedings
of the 2002 Congress on Evolutionary Computation,
pp. 1408-1413, 2002
[7] SPARC International, The SPARC Architecture
Manual
