Applying Genetic Improvement to a Genetic Programming library in C++

Abstract

A young subfield of evolutionary computing that has gained the attention of many researchers in recent years is genetic improvement. It uses an automated search method that directly modifies the source code or binaries of a software system to find improved versions based on some given criteria. Genetic improvement has achieved notable results and the acceptance of several research communities, namely software engineering and evolutionary computation. Over the past 10 years there have been core publications on the subject; however, we have identified, to the best of our knowledge, that there is no work on applying genetic improvement to a meta-heuristic system. In this work we apply the GI framework called GISMO to the Beagle Puppy library version 0.1 in C++, a genetic programming system configured to perform symbolic regression on several benchmark and real-world problems. The objective is to improve the processing time while maintaining a similar or better test fitness of the best individual produced by the unmodified genetic programming search. Results show that GISMO can generate individuals that present an improvement on those two key aspects over some problems, while also reducing the effects of bloat, one of the main issues in genetic programming.

Full text

Noname manuscript No.
(will be inserted by the editor)
Applying Genetic
Improvement to a Genetic
Programming library in C++
V´ıctor R. L´opez
L´opez ·Leonardo
Trujillo∗
·Pierrick
Legrand
Received: date / Accepted: date
Abstract A young sub-field of Evolutionary Comput-
ing that has gained the attention of many researchers in
recent years is Genetic Improvement. It uses an auto-
mated search method that directly modifies the source
code or binaries of a software system to find improved
versions based on some given criteria. Genetic Improve-
ment has achieved notable results and the acceptance
of several research communities, namely software engi-
neering and evolutionary computation. Over the past
10 years there has been core publications on the sub-
ject, however, we have identified, to the best of our
knowledge, that there is no work on applying Genetic
Improvement to a meta-heuristic system. In this work
we apply the GI framework called GISMO to the Beagle
Puppy library version 0.1 in C++, a Genetic Program-
ming system configured to perform symbolic regression
on several benchmark and real-world problems. The ob-
jective is to improve the processing time while main-
taining a similar or better test-fitness of the best indi-
vidual produced by the unmodified Genetic Program-
ming search. Results show that GISMO can generate
individuals that present an improvement on those two
key aspects over some problems, while also reducing
the effects of bloat, one of the main issues in Genetic
Programming.
V´ıctor R. L´opez L´opez ·Leonardo Trujillo
Instituto Tecnol´ogico de Tijuana
Tijuana, BC, M´exico
E-mail: vlopez@tectijuana.edu.mx
E-mail: leonardo.trujillo@tectijuana.edu.mx
Pierrick Legrand
University of Bordeaux and the IMB - Institut de
Math´ematiques de Bordeaux, UMR CNRS 5251 and Inria
Bordeaux Sud-Ouest, France
E-mail: pierrick.legrand@u-bordeaux.fr
Keywords Genetic Improvement ·Meta-heuristic ·
Genetic Programming
1 Introduction
Evolutionary Computing (EC) is a widely used com-
putational search technique from the field of artificial
intelligence. It is inspired by a simplified model of Dar-
winian evolutionary principles and contains a broad
family of Evolutionary Algorithms (EAs) for search and
optimization. EAs iteratively process a population of
candidate solutions to find the optimum of a given prob-
lem by applying at each iteration, typically, the actions:
selection of the fittest, reproduction of selected parts or
the whole population, and survival of the fittest evolved
offspring. They have achieved strong results in many
domains (Koza, 2010), and one of its most popular algo-
rithms is Genetic Programming (GP). GP is generally
used to solve machine learning problems by evolving
computer programs, in the form of symbolic expres-
sions, from a high-level problem statement; i.e., creat-
ing working programs from scratch, an aim shared by
program synthesis (Manna and Waldinger, 1971) and
automatic program induction (Koza, 1992).
An emerging sub-field of EC is Genetic Improve-
ment (GI) that, in contrast to GP, usually starts from
a working program that normally is much larger than
those evolved in GP (Petke et al, 2017). GI uses an
automated search method that directly modifies the
source code or the software binaries to find improved
versions of existing software based on a given set of
criteria for improvement (Petke et al, 2017). The crite-
ria for software improvement can be divided into func-
tional and non-functional properties. The functional
properties refer to the logic of the program, i.e., what
the program should do, an example is software repair
(bug-fixing). The non-functional properties refer to the
physical properties that the program presents, e.g. ex-
ecution time, memory storage and energy consump-
tion, just to mention a few. The strengths of GI lie
in the abundance of source code available for reuse
and the expanded search space of software variants cre-
ated by relaxing restrictions on program correctness. GI
has won human-competitive awards, “Humies”, from
the Genetic and Evolutionary Computation Conference
(GECCO)1, attracting researchers from the software
engineering and evolutionary computation communi-
ties. An exhaustive survey can be found in (Petke et al,
2017), which presents an overview of GI work gathered
from most of the publications in this field summarized
1More information about “Humies” awards can be found
at http://www.human-competitive.org/

2 V´ıctor R. L´opez L´opez et al.
into four topics. First, preserved properties focuses on
how to capture software functionality that needs to be
retained (Orlov and Sipper, 2009). Second, use of exist-
ing software deals with the reuse of source code, where
the content of the new source code can be assembled
out of existing source code, pieces of source code gener-
ated from scratch, or by transplanting source code from
a single or multiple donors (Petke et al, 2014). Third,
criteria for improvement defines non-functional prop-
erties to improve (White et al, 2011; Langdon, 2015).
Fourth, search refers to the methods used to explore the
search space of possible software versions (Forrest et al,
2009; Ackling et al, 2011). The type of software tack-
led in GI ranges from simple programs of less than 50
lines of source code, for example a program of 13 lines
of source code that represents an image operator for
a computer vision problem (L´opez-L´opez et al, 2016),
to complex software systems such as Bowtie2 that is
a DNA sequencing system consisting of 50,000 lines
of source code in C++ (Langdon and Harman, 2015a).
On the other hand, we have identified that there is no
work on applying GI to a meta-heuristic library. There
is a field related to GI called Automated Design of Algo-
rithms (ADA) (Haraldsson and Woodward, 2014) that
has been applied to improve a meta-heuristic algorithm
for a specific problem, but, to the best of our knowledge,
it has not been used to directly operate at the level of
source code.
The aim of this paper is to apply a GI frame-
work to improve a meta-heuristic system. We apply the
GI framework called GISMO (Langdon and Harman,
2015a) to the Beagle Puppy library version 0.1 in
C++ (Gagn´e and Parizeau, 2006), that is a GP system
configured to perform symbolic regression on several
benchmark and real-world problems. The objective is
to improve the processing time (non-functional proper-
ties) while maintaining a similar or better test-fitness of
the best individual produced by the unmodified GP cy-
cle of the Beagle Puppy library (functional property).
The remainder of this paper is organized as follows:
Section 2 reviews GI, focusing on GISMO. Section 3
gives an overview of ADA and describes other related
work. Experiments are described in Section 4, results
are presented in Section 5 and a discussion is exposed
in Section 6. Finally, Section 7 contains concluding re-
marks and suggestions for future work.
2 Background
GI, as mentioned before, is used to improve a software
system by using an automated search method. The idea
behind a GI framework is to start with a human writ-
ten source code and use it as genetic material for an
automated search method to explore a search space of
possible program modifications, to find a modified ver-
sion of a program that improves some given criteria,
while preserving the functionality of the original source
code. To further illustrate GI, we will concentrate
on the GISMO framework proposed by Harman et al
(2012), which is used in the experimental work of
this paper. The GISMO framework has achieved sig-
nificant results in the optimization of open-source
software, particularly in software speed-ups. Differ-
ent applications of GISMO have been developed, such
as auto-porting functionality (Langdon and Harman,
2010), improving parallel code (Langdon and Harman,
2014; Langdon et al, 2014, 2015), specialization of a
C++ program (Langdon and Harman, 2015a), code
transplant (Petke et al, 2014, 2018) and grow and graft
(Langdon and Harman, 2015b; Harman et al, 2014).
More details can be found in (Langdon, 2015).
In particular, we are interested in the specialization
of a C++ program, specifically the work presented
by Langdon and Harman (2015a) where they applied
GISMO to improve a system of 50,000 lines of source
code, producing a modified system that is, on average,
70 times faster than the original system. This work
demonstrated the ability of GI to successfully improve a
complex system. For this GISMO is used in the present
work, using the public open-source libraries available at
http://www0.cs.ucl.ac.uk/staff/ucacbbl/gismo/
in the “Free Code” section (Langdon and Harman,
2015a).
In GISMO, the original source code is represented
by a BNF grammar, and a GP search is used to ex-
plore alternate versions of the original source code by
generating individuals that represent a list of modifica-
tions to the BNF grammar. The modifications can be
the deletion of a line of source code, insertion of a line
of source code next to another, and replacement of a
line of source code with another line. The reproduction
methods of the GP search are mutation and crossover,
where mutation consist of applying one of the modi-
fications previously mentioned, and crossover consists
of the concatenation of the list of modifications from
two individuals to create a new one. The GISMO pro-
cess is illustrated in Figure 1 and can be summarized
as follows:
–It starts with the source code of the software to
be improved, then a BNF grammar representation,
which describes line by line the original source code,
is created from it.
–A sensitivity analysis is performed, profiling the
source code to identify which parts have the great-
est effect on the non-functional property of interest
(such as execution time).

Applying Genetic Improvement to a Genetic Programming library in C++ 3
–The initial population is created of random single
mutations weighted towards the parts identified in
the previous step, corresponding to the BNF gram-
mar.
–The reproduction methods mentioned above are
used to create a population of mutated individuals.
–Each mutated individual is transformed to mutated
source code by reversing the grammar.
–The fitness is assessed by compiling and executing
the mutated source code over a set of test-cases and
measuring the non-functional properties. The test-
cases define inputs/outputs that describe the be-
havior of the original program; these serve to evalu-
ate the functional properties. There are constraints
that affect the fitness of an individual such as suc-
cessful compilation and execution, number of test-
cases passed, and thresholds for the functional and
non-functional properties.
–A selection step is carried out to determine the new
population. Based on the constraints, the eligible
individuals are sorted in ascending order to be the
parents for the next generation. At most half of the
parent population is selected to have 2 children in
the next generation, with the restriction that all
selected parents must compile. One child is a mu-
tant, the other is a crossover between a selected par-
ent and another member of the current population.
Children must be different from each other and from
the current population. If crossover cannot create a
different child, the child is created by mutating the
selected parent. If the number of selected parents is
fewer than half of the population, then two children
are created at random.
–Finally, the evolutionary cycle continues until a
given number of generations.
3 Related work
ADA is a general methodology for using search methods
to improve or discover algorithms for any given com-
putational problem (Haraldsson and Woodward, 2014).
ADA is related to GI by having a similar methodology,
but differs in certain aspects, mainly in the way it is
applied. GI is applied in-situ or directly to the source
code, while ADA works ex-situ, i.e., evolves high-level
instructions or functions to manage an algorithm. Both
approaches have been used to improve non-functional
properties, but GI is the only one that has been used
for bug repairing. The work in ADA has concentrated
in three strategies, given certain information from a
problem: by replacing components with an alternative
(Woodward and Swan, 2012); by reordering the com-
ponents within the algorithm (Caseau et al, 1999); by
parameter tuning (Smit and Eiben, 2009). From these
strategies, we consider that the work from the first two
is most relevant to GI and our work. ADA has cen-
tered on the specialization of algorithms to a certain
problem or a subset of problems (Burke et al, 2007;
Woodward and Swan, 2012, 2011). However, these only
manipulate one component of a general evolutionary
process and not the whole cycle as can be done with
GI.
Another related work is from Louren¸co et al (2012),
where they applied a Grammatical Evolution (GE)
framework to automatically design an EA to solve the
Royal Road Functions problem. They showed that their
proposed framework can produce better variants for
this problem. However, they do not consider more real-
istic benchmarks or real-world problems.
Other work from GI is the specialization of Min-
iSAT by Petke et al (2013), an open-source C++ solver
for Boolean satisfiability (SAT) problems. This works
is relevant because they applied GISMO, and could not
find a significant improvement relative to the original
system. After that, they proposed the application of
code transplant (Petke et al, 2014) that lead to suc-
cessfully improve MiniSAT by using multiple versions
of the system from different authors to create one im-
proved version.
A paper that proposes a similar idea to the one
in the present paper, of using the same meta-heuristic
to optimize itself, is presented by Harman and Petke
(2015). They propose to use GI to find an optimal fit-
ness function to measure the energy consumption of a
software for a GI search. But to the best of our knowl-
edge there are no published results that were derived
from that proposal.
4 Experimental methodology
We have chosen to use the Beagle Puppy library, a GP
system implemented in C++ that is still actively used
in research (Pozi et al, 2016; Jablkowski et al, 2017).
Moreover, it is a minimalist GP library, thus being easy
to understand and interpret for future modifications.
More important, we chose the first version of Beagle
Puppy because, of all the available versions it would be
the most likely to have room for improvements given
that it was the original release.
To apply GISMO on Beagle Puppy we used the fol-
lowing setup. First, we define the test-cases used by the
GISMO approach. For this, we use standard symbolic
benchmarks proposed by McDermott et al (2012) and
real-world problems from (Lichman, 2013). The prob-
lems are summarized in Table 1 and Table 2, respec-
tively.

4 V´ıctor R. L´opez L´opez et al.
Fig. 1 Illustration of the GISMO process.
Test-case Name Benchmark function
0 Koza-1 (Koza, 1992) x4+x3+x2+x
1 Koza-2 (Koza, 1994) x5
−2x3+x
2 Koza-3 (Koza, 1994) x6
−2x4+x2
3 Nguyen-1 (Uy et al, 2011) x3+x2+x
4 Nguyen-3 (Uy et al, 2011) x5+x4+x3+x2+x
5 Nguyen-4 (Uy et al, 2011) x6+x5+x4+x3+x2+x
6 Nguyen-5 (Uy et al, 2011) sin(x2)cos(x)−1
7 Nguyen-6 (Uy et al, 2011) sin(x) + sin(x+x2)
8 Nguyen-7 (Uy et al, 2011) ln(x+ 1) + ln(x2+ 1)
9 Korns-11 (Korns, 2011) 6.87 + 11cos(7.23x3)
Table 1 The Benchmark problems conforming part of the test-cases.
Test-case Name Total Variables
10 Concrete (Yeh, 1998) 8
11 Housing (Harrison and Rubinfeld, 1978) 13
12 Tower (Vladislavleva et al, 2009) 25
Table 2 The Real problems used as test-cases ; for each problem, there is one output variable.
Second, we generate the symbolic regression data
sets for each test-case. It is important to note that for
each test-case we generate Ndifferent data set parti-
tions, where Nis 100 for the benchmarks and 30 for
the real-world problems. For the benchmarks, the data
were randomly generated for each partition, while for
the real-world problems we used random partitions of
each complete data set.
Third, we define the training and testing sets to use
in GISMO. The training set is composed of arbitrary se-
lected test-cases, and the remaining test-cases compose
the testing set to evaluate the best individual found by
GISMO.
Fourth, we establish the reference results that will
be used to evaluate the results from the modified ver-
sions of the Beagle Puppy program (GISMO individ-

Applying Genetic Improvement to a Genetic Programming library in C++ 5
uals) for each of the test-cases. These are obtained by
executing Ntimes the original Beagle Puppy program
and calculating the median of the processing time and
test fitness measures. Finally, we establish the GISMO
and Beagle Puppy setups as follows.
4.1 GISMO setup
–Representation: List of replacements, deletions and
insertions into the BNF grammar.
–Fitness: The fitness function is defined by measures
that represent the gain (or loss) of each modified
GISMO individual relative to the reference individ-
ual (original code). These measures are obtained by
executing the modified code N times and calculating
the processing time gain P tGain and the test fitness
gain FGP –Gain, such that fitness is given by
FGI =P tGain +FGP –Gain,(1)
which is to be maximized, where
P tGain =
cn
PP tGain(c)
cn
,(2)
and
FGP –Gain =
cn
PFGP –Gain(c)
cn
,(3)
with crepresenting a test-case, and cnis the total
number of test-cases used in the experiment; both
represent the average gain over all cin the experi-
ment. The P tGain(c) and FGP –Gain(c) are the per-
centage gain of the median processing time f
P t(c)
and median test fitness e
FGP (c) relative to their ref-
erence individual f
P tref (c) and e
FGP –ref (c), given by
FGP –Gain(c) = e
FGP –ref (c)−e
FGP (c)
e
FGP –ref (c)
·100,(4)
and
P tGain(c) = f
P tref (c)−f
P t(c)
f
P tref (c)
·100.(5)
The median is calculated from the N executions.
–Reproduction: Single crossover operator and 3 types
of mutation are used. The crossover is a concate-
nation of list replacements. The mutations delete,
replace or insert new edits into the code, and have
equal probability of selection.
–Selection: An evolved version cannot be selected to
be a parent unless it passes more than half of the
test-cases by satisfying the following constraints:
–The f
P t(c) and e
FGP (c) must meet:
*f
P t(c)<200%f
P tref (c)
*e
FGP (c)>80% e
FGP –ref (c)
–Succeed in compilation and execution.
In the case of failing less than half of cn, we assign
f
P t(c) = 2×f
P tref (c) and e
FGP (c) = 0; we will refer to
this as fail fitness. The purpose of this is to be able
to compute the FGI with the failed test-cases. After
applying these conditions to each member of the
population, the same selection step from GISMO,
described in Section 2, is applied.
–Population: Panmictic, non-elitist, generational
with 20 individuals.
–Others: Initial population of random single mutants,
50% append crossover. No size limit is used for the
evolved solutions, and the stopping criterion for the
search is 200 generations.
4.2 Beagle Puppy setup
–Representation: Classical Tree-based representa-
tion.
–Fitness: FGP =1
1+RM SE where RMSE is the Root
Mean Square Error on the training set, performance
on the test set is computed in the same manner.
–Reproduction: There are 2 types of mutation and
sub-tree crossover. The standard (Koza) mutation
and point mutation, each have a probability of 5%,
and sub-tree crossover with 90% mating probability,
and a maximum tree depth of 17.
–Selection: Tournament selection with 2 participants.
–Population: Steady-state, elitist with 50 individuals.
–Others: Initialize population with ramped half-and-
half. The stopping criterion for the search is a max-
imum of 50 generations.
To have a better visualization of these setups, Figure
2 shows the whole evolutionary cycle, where the dashed
arrows indicate the evolutionary cycle of GISMO, as
presented in Section 2, and the solid arrows inside the
solid block (inner-left corner) indicate the evolution-
ary cycle of Beagle Puppy GP. The GISMO process
starts by defining the BNF grammar from the Beagle
Puppy Library source code; then, the initial population
is created, as mentioned in the GISMO setup, with 20
individuals but without the sensitive analysis. Instead,
we selected the parts of the BNF grammar randomly
with the intention to promote diversity in the creation
of the initial individuals. Next, the reproduction pro-
duces a mutated population; after this, each individual
is translated with the BNF grammar to generate a mu-
tated source code. All individuals are compiled, if com-
pilation fails then assign a fail measure as fitness (as
mentioned in the GISMO setup). If compilation was

6 V´ıctor R. L´opez L´opez et al.
Experiment test-cases (problems)
1 0,2,4,6,8
2 1,5,9,10
3 5,6,7,8,9,10,11
4 0,3,7,8,9,10,11
5 0,6,10,11
6 0,1,2
7 0,3,7,8,9,10,11
8 2,7,10,11
9 4,9,11
10 10
11 11
12 12
Table 3 Table of experiments that show different problems
used as test-cases for GISMO training, showing the problem
number of Tables 1 and Table 2.
Experiment FGI (%) P tGain (%) FGP –Gain (%)
16.8 20.4 -13.6
2 3.5 13.4 -9.9
3 5 13.7 -8.7
4 -52.7 -58.6 5.9
5 2.6 10.7 -8.1
6 5.5 15.5 -10
7 1.6 8.8 -7.2
8 3.4 12.6 -9.2
9 -5 -4.6 -0.4
10 2.3 5.6 -3.3
11 4.5 5.2 -0.7
12 -9.6 -14.6 5
Table 4 Numerical results of the best individual for each
experiment; bold values in each column, except the leftmost,
indicate the highest value obtained over all experiments; the
shaded row indicates the experiment that achieved the high-
est P tGain maintaining a negligible loss of FGP –Gain.
successful, then for each of the test-cases execute the
mutated Beagle Puppy program Ntimes, measure the
GISMO objectives (processing time and test-fitness) for
each execution and calculate f
P t(c) and e
FGP (c). If the
individual fails to execute or exceeds the permitted ex-
ecution time then assign the fail fitness. After applying
the previous conditions, compute FGI for all individu-
als. Next, apply the selection process to determine the
new mutated population. Finally, the evolutionary cy-
cle continues until the stop condition is reached.
5 Experiments and results
All the experiments were conducted on a PC with a
AMD Turion(tm) II Dual-Core Mobile M520 processor.
We use the PAPI library (London et al, 2001) to mea-
sure the processing time of the GP cycle. The experi-
ments consist of applying GISMO using different sets of
test-cases, as shown in Table 3, with the methodology
presented in the previous section.
The results of GISMO for each experiment are pre-
sented graphically in a Pareto front plot of P tGain and
FGP –Gain of the evolved solutions, shown in Figure 3
and Figure 4. Each plot includes the reference individ-
ual (red dot) located at position (0,0), and a selected
individual representing the best individual of the ex-
periment (green dot); what we desire is an individual
far from zero in both axes, but most individuals are far
from such an ideal result. In other words, we chose the
individual that exhibits approximately the best com-
promise between both objectives.
The approximated Pareto front of most of the exper-
iments shows a similar behavior, with the performance
of most individuals in the range of −40% to 30% for
P tGain and −40% to 10% for FGP –Gain. Between these
ranges resides the individual that was chosen to eval-
uate against the remaining test-cases (testing-set for
GISMO) of each experiment. We will refer to this indi-
vidual as the “best” individual of the experiment. The
best individual was selected based on two criteria, that
it is a non-dominated solution and that its performance
generalizes to unseen test cases.
The results of the best individuals are presented in
Figure 5 and Figure 6, in the form of bar graphs showing
the percentage of gain (or improvement) in the verti-
cal axis, obtained between the best individual of the
experiment against the reference individual (unmodi-
fied Beagle puppy program) for each test-case (horizon-
tal axis) from each experiment (row). Each test-case
shows three bars that are the processing time (blue),
test-fitness (orange) and mean size of the population
P op −size(c) from the best individual found (yellow).
This last measure is only included for analysis and was
not part of the GISMO fitness but it is a critical aspect
of GP evolution (Trujillo et al, 2016). Also, the mean
gain of P tGain(c) and FGP –Gain (c), both calculated
over all test-cases, are shown as dashed lines (blue and
orange, respectively). Last, each experiment shows the
test-cases used for training, indicated by red bold num-
bers. In the best case scenario, the gain should always
be higher than zero. It is not easy to determine which
experiment achieves the highest overall gain. This can
be confirmed with numerical results shown in Table 4.
The results indicate that the individual with the
highest FGI and P tGain was found in Experiment 1
with 6.8% and 20.4% respectively, it also achieved a
FGP –Gain of −13.6%. Moreover, the lowest FGI but
highest FGP –Gain was found in Experiment 4 with
−52.7% and 5.9% respectively, it also achieved a P tGain
of −58.6%. Despite these two individuals achieving the
highest in at most two of the measures, the individual
that achieves a high P tGain while maintaining a sim-
ilar or better FGP –Gain was found in Experiment 11,

Applying Genetic Improvement to a Genetic Programming library in C++ 7
Fig. 2 Illustration of the whole evolutionary cycle.
Table 5 Total and important modifications, corresponding
to the best individual found by GISMO for each experiment.
Experiment Total
modifications
Important
modifications
1 106 3
2 37 3
3 21 3
4 11 2
5 18 2
6 10 2
7 29 5
8 52 4
9 40 3
10 54 2
11 73 1
12 39 1
with 5.2% and −0.7% respectively, it also achieved a
FGI–Gain of 4.5%. With respect to P op −size(c) in all
experiments, we observe a slightly lower gain compared
to P tGain. It its important to note that P op −siz e(c)
higher than zero means reduction in the mean size of the
population. We also observe that test-cases 1, 2 and 9,
in most of the experiments, present high P op −siz e(c)
and P tGain, but maintain a FGP –Gain lower than -3%;
these results indicate the presence of bloat in the origi-
nal GP, which occurs when program trees tend to grow
unnecessarily large, without a corresponding increase in
fitness (Poli et al, 2008; Silva and Costa, 2009). This
means that GISMO found a version of GP that pro-
duces smaller individuals than the original GP with
negligible difference in fitness, hence, the original GP
containing bloat.
In addition to these numerical results, we analyze
the most important modifications found by GISMO for
each best individual. For this, we need to identify which
modifications of the individual are responsible for at
least some of the gain. Hence, we use a clean up proce-
dure that evaluates the P tGain and FGP–Gain of a ver-
sion of the analyzed individual that excludes one muta-
tion against the P tGain and FGP –Gain of the same in-
dividual without excluding the modifications. One ver-
sion of the individual is generated for each of the total
number of modifications. The excluded modification is
discarded if the P tGain and FGP –Gain are the same
or lower as the original. Table 5 shows the number of
modifications of the best individual from each experi-
ment and the resulting number of important modifica-
tions found with the clean up procedure. We observed
that most of the modifications found by GISMO have
a “neutral” effect in gain, and others present the dele-
tion of assert functions, which only marginally effect
P tGain.
Table 6 and Table 7 show the new and original
source code corresponding to the modifications respon-
sible for the results of each best individual. In general,
we can observe that the number of modifications re-
sponsible for the results range between 1 and 5, and

8 V´ıctor R. L´opez L´opez et al.
−40 −30 −20 −10 0 10
−40
−30
−20
−10
0
10
20
30 Pareto Front
FGP–Gain
PtGain
(b) Experiment 1
−40 −30 −20 −10 0 10
−40
−30
−20
−10
0
10
20
30 Pareto Front
FGP–Gain
PtGain
(c) Experiment 2
−40 −30 −20 −10 0 10
−40
−30
−20
−10
0
10
20
30 Pareto Front
FGP–Gain
PtGain
(d) Experiment 3
−40 −30 −20 −10 0 10
−40
−30
−20
−10
0
10
20
30 Pareto Front
FGP–Gain
PtGain
(e) Experiment 4
−40 −30 −20 −10 0 10
−40
−30
−20
−10
0
10
20
30 Pareto Front
FGP–Gain
PtGain
(f) Experiment 5
−40 −30 −20 −10 0 10
−40
−30
−20
−10
0
10
20
30 Pareto Front
FGP–Gain
PtGain
(g) Experiment 6
Fig. 3 Pareto front of experiments 1 to 6. The red circle is the reference individual and the green circle is the selected
individual to evaluate.

Applying Genetic Improvement to a Genetic Programming library in C++ 9
−40 −30 −20 −10 0 10
−40
−30
−20
−10
0
10
20
30 Pareto Front
FGP–Gain
PtGain
(b) Experiment 7
−40 −30 −20 −10 0 10
−40
−30
−20
−10
0
10
20
30 Pareto Front
FGP–Gain
PtGain
(c) Experiment 8
(d) Experiment 9 (e) Experiment 10
−40 −30 −20 −10 0 10
−40
−30
−20
−10
0
10
20
30 Pareto Front
FGP–Gain
PtGain
(f) Experiment 11
−40 −30 −20 −10 0 10
−40
−30
−20
−10
0
10
20
30 Pareto Front
FGP–Gain
PtGain
(g) Experiment 12
Fig. 4 Pareto front of experiments 7 to 12. The red circle is the reference individual and the green circle is the selected
individual to evaluate.

10 V´ıctor R. L´opez L´opez et al.
−70
−50
−30
−10
10
30
50
70
1 3 5 7 9 10 11 12
0 2 4 6 888
Gain [%]
Exp:1
20.4
−13.6
PtGain(c)FGP–Gain(c)Pop–size(c)
f
PtGain
e
FGP–Gain
−70
−50
−30
−10
10
30
50
70
0 2 3 4 6 7 8 11 12
1 5 9 10101010
Gain [%]
Exp:2
13.4
−9.9
−70
−50
−30
−10
10
30
50
70
0 1 2 3 4 12
5 6 7 8 9 10 11
Gain [%]
Exp:3
13.7
−8.7
−70
−50
−30
−10
10
30
50
70
1 2 4 6 8 9 12
0 3 5 7 10 1111
Gain [%]
Exp:4
−58.6
5.9
−70
−50
−30
−10
10
30
50
70
1 2 3 4 5 7 8 9 12
0 6 10 11111111
Gain [%]
Exp:5
10.7
−8.1
−70
−50
−30
−10
10
30
50
70
3 4 5 6 7 8 9 10 11 12
0 1 22222
Gain [%]
Exp:6
15.5
−10
Test−case (c)
Fig. 5 Bar graphs showing P tGain(c) (blue),FGP –Gain(c) (orange) and P op −size(c) (yellow) for all test-cases cof the best
individual for experiments 1 to 6; P tGain and FGP –Gain are calculated over all test-cases (dashed line blue and orange,
respectively). Red bold numbers (in vertical axis) indicate the test-cases used for training.

Applying Genetic Improvement to a Genetic Programming library in C++ 11
−70
−50
−30
−10
10
30
50
70
1 2 4 5 6 12
0 3 7 8 9 10 11
Gain [%]
Exp:7
8.8
−7.2
PtGain(c)FGP–Gain(c)Pop–size(c)
f
PtGain
e
FGP–Gain
−70
−50
−30
−10
10
30
50
70
0 1 3 4 5 6 8 9 12
2 7 10 11111111
Gain [%]
Exp:8
12.6
−9.2
−70
−50
−30
−10
10
30
50
70
0 1 2 3 5 6 7 8 10 12
4 9 1111111111
Gain [%]
Exp:9
−4.6
−0.4
−70
−50
−30
−10
10
30
50
70
0 1 2 3 4 5 6 7 8 9 11 12
10101010101010
Gain [%]
Exp:10
5.6
−3.3
−70
−50
−30
−10
10
30
50
70
0 1 2 3 4 5 6 7 8 9 10 12
11111111111111
Gain [%]
Exp:11
5.2
−0.7
−70
−50
−30
−10
10
30
50
70
0 1 2 3 4 5 6 7 8 9 10 11 12121212121212
Gain [%]
Exp:12
−14.6
5
Test−case (c)
Fig. 6 Bar graphs showing P tGain (c) (blue),FGP –Gain(c) (orange) and P op −size(c) (yellow) for all cof the best individual
for experiments 7 to 12; P tGain and FGP –Gain are calculated over all test-cases (dashed line blue and orange, respectively).
Red bold numbers (in vertical axis) indicate the test-cases used for training.

12 V´ıctor R. L´opez L´opez et al.
Table 6 The most important source code modifications found in the best individual for experiments 1 to 6.
Original sour code New source code
Experiment 1
<IF_Puppy_257><IF_Puppy_573>
if(ioContext.mRandom.rollUniform() >=
inInitGrowProba) if(ioPopulation.size() == 0)
<IF_Puppy_71>
if(ioContext.mRandom.rollUniform() <= inMatingProba) If(0)
<for2_Puppy_383>
for(unsigned int i=0; i<ioPopulation.size(); ++i) for(unsigned int i=0; 0; ++i)
Experiment 2
<IF_Puppy_473>
if(ioTree.size() > 1) if(0)
<for2_Puppy_69>
for(unsigned int i=0;i<ioPopulation.size();++i) for(unsigned int i=0;0;++i)
<_Puppy_503>
ioTree[lMutIndex].mPrimitive =
lFunction->giveReference(ioContext) //empty
Experiment 3
<for2_Puppy_69>
for(unsigned int i=0;i<ioPopulation.size();++i) for(unsigned int i=0;0;++i)
Experiment 4
<_Puppy_195>
lNode2IsTerminal =
(ioContext.mRandom.rollUniform() >= inDistribProba) //empty
<for1_Puppy_527><for1_Puppy_149>
for(unsigned int j=1; for(unsigned int j=0;
j<inNumberParticipants; ++j) j<inNumberParticipants; ++j)
Experiment 5
<_Puppy_226>
exchangeSubTrees(ioTree1, lChoosenNode1, lStack1,
ioTree2, lChoosenNode2, lStack2) //empty
<IF_Puppy_473>
if(ioTree.size() > 1) if(0)
Experiment 6
<IF_Puppy_330>
if(inMinDepth > 1) if(0)
<for2_Puppy_176>
for(unsigned int i=0; i<7; ++i) for(unsigned int i=0; 0; ++i)
that the majority of these important modifications are
deletions, and a few are replacements. The deletion of
source code may seem to be a trivial way to reduced
processing time, and in some cases maintains accept-
able accuracy on some training cases. However, trivial
solutions do not tend to generalize well to all test-cases,
while for the solutions chosen for analysis the loss in ac-
curacy is negligible in many test-cases, compared with
the improvement in processing time which is sometimes
above 40%. By analyzing what each modifications does
to the original code, we can group them into the follow-
ing groups:
–No-crossover: There is no crossover. This is because
the functions related to crossover are interrupted by
modifications <IF_Puppy_71>,<for2_Puppy_69>,
<IF_Puppy_220>,<_Puppy_226> and
<for2_Puppy_176> that eliminates the if ar-
gument or the for condition.
–No-mutStand: There is no standard muta-
tion. The modifications <IF_Puppy_385> and
<for2_Puppy_383> eliminate the possibility of
applying the mutation standard.
–No-mutPoint: There is no point mutation. The mod-
ifications <IF_Puppy_451> and <_Puppy_453> elim-
inate the possibility of applying the mutation swap.
–TreeGrow: The initial population is
only created by treegrow. The modifica-
tions <IF_Puppy_257><IF_Puppy_573> and
<IF_Puppy_257><IF_Puppy_335> replace the ar-
gument “random uniform number is more than
or equal to an initial grow probability” of the
condition that controls the method to initialize
the population with full trees with the argument
of “population size equal to zero” or “max depth
equal to 1”. By applying either arguments, the
initial population size will always be more than
zero and the max depth more than 1 thus never

Applying Genetic Improvement to a Genetic Programming library in C++ 13
Table 7 The most important source code modifications found in the best individual for experiments 7 to 12.
Original sour code New source code
Experiment 7
<IF_Puppy_257><IF_Puppy_335>
if(ioContext.mRandom.rollUniform() >= inInitGrowProba) if(inMaxDepth == 1)
<for2_Puppy_383>
for(unsigned int i=0; i<ioPopulation.size(); ++i) for(unsigned int i=0; 0; ++i)
<IF_Puppy_182>
if(ioTree1.size() > 1) if(0)
<WHILE_Puppy_187>
while(
(ioTree1[lChoosenNode1].mPrimitive->
getNumberArguments() == 0) != lNode1IsTerminal) While(0)
<IF_Puppy_330>
if(inMinDepth > 1) if(0)
Experiment 8
<IF_Puppy_451>
if(ioContext.mRandom.rollUniform() < inMutationProba) if(0)
<for2_Puppy_176>
for(unsigned int i=0; i<7; ++i) for(unsigned int i=0; 0; ++i)
<IF_Puppy_330>
if(inMinDepth > 1) if(0)
<for1_Puppy_527><for1_Puppy_149>
for(unsigned int j=1; for(unsigned int j=0;
j<inNumberParticipants; ++j) j<inNumberParticipants; ++j)
Experiment 9
<IF_Puppy_385>
if(ioContext.mRandom.rollUniform() < inMutationProba) if(0)
<_Puppy_195>
lNode2IsTerminal =
(ioContext.mRandom.rollUniform() >= inDistribProba) //empty
<WHILE_Puppy_198>
while((ioTree2[lChoosenNode2].mPrimitive->
getNumberArguments() == 0) != lNode2IsTerminal)
!= lNode2IsTerminal) While(0)
Experiment 10
<IF_Puppy_330>
if(inMinDepth > 1) if(0)
<IF_Puppy_473>
if(ioTree.size() > 1) if(0)
Experiment 11
<for2_Puppy_383>
for(unsigned int i=0; i<ioPopulation.size(); ++i) for(unsigned int i=0; 0; ++i)
Experiment 12
<for1_Puppy_527><for1_Puppy_86>
for(unsigned int j=1; for(unsigned int j=0;
j<inNumberParticipants; ++j) j<inNumberParticipants; ++j)
meeting the condition to initialize the population
with full trees.
–TreeGrow-1: The individuals in the grow approach
are created by selecting the root node randomly be-
tween the function and terminal sets. The modifi-
cation <IF_Puppy_330> eliminates the condition of
selecting a function for the root node, leaving only
the viable condition of selecting a root node at ran-
dom from both the function and terminal sets.
–Tournament: The number of participants in the
tournament is incremented by 1. The modifi-
cations <for1_Puppy_527><for1_Puppy_149>,
<for1_Puppy_527><for1_Puppy_149> and
<for1_Puppy_527><for1_Puppy_86> replace
the initialization “unsigned int j = 1” of the for
loop that controls the number of participants in
the tournament with “unsigned int j = 0”, thus
incrementing the number of iterations, and hence
the number of tournament participants by 1.
–Matetrees-1: Only permits selecting terminal nodes
to swap in one tree of the crossover operator. The
modification <_Puppy_195> eliminates the variable
that is used for to decide if the selected node to
swap from a tree is a function or primitive based on
a 90% probability. By doing this, the variable is by

14 V´ıctor R. L´opez L´opez et al.
Experiment
No-Crossover
No-MutStand
No-mutPoint
TreeGrow
TreeGrow-1
Tournament
MateTrees-1
MateTrees-2.1
MutPoint-1
MutPoint-2
1 x x x
2 x x x
3x
4x x
5 x x
6 x x
7 x x x
8 x x x x
9 x x
10 x x
11 x
12 x
Table 8 The most important modifications found by GISMO and how they appeared in the best individual of each experiment.
default a terminal thus leaving only the possibility
to swap in terminal nodes for one of the parents.
–Matetrees-2: Makes the nodes to swap completely
random in the crossover operator. The mod-
ifications <IF_Puppy_182>,<WHILE_Puppy_187>,
<_Puppy_195> and <WHILE_Puppy_198> eliminate
the 90% probability of selecting a function or ter-
minal node for swaping, thus leaving the selection
completely random.
–MutPoint-1: Makes the nodes to swap completely
random in the point mutation operator. The mod-
ification <IF_Puppy_473>, similar to Matetress-2,
eliminates the probability of selecting a function or
terminal node for swaping, thus leaving the selection
completely random.
–MutPoint-2: Only permits a swap in terminal nodes
in the point mutation operator. The modification
<_Puppy_503> eliminates an instruction that as-
signs the selected function node to the tree, thus
only permitting change when the selected node to
swap is a terminal.
Table 8 shows the tags corresponding to an explana-
tion of the important optimizations found by GISMO
for the best individual of each experiment. We can ob-
serve that the most frequent tag is the No-crossover,
which is present in the majority of the individuals with
high P tGain. In the opposite case, the Tournament is
present in only two experiments that achieved a high
FGP –Gain. Moreover, the individual of Experiment 11
only has No-MutStand.
6 Discussion
We observe that GISMO tends to eliminate the
crossover method from GP to improve the process-
ing time, a naive way to reduce computational cost.
This is also reflected in the mean size of the popula-
tion P op −size(c) in the GP runs, where the individ-
uals tend to be smaller in size, which also reduces the
computational cost of during fitness evaluation. A note-
worthy observation is that, as expected, the reference
GP system exhibits the effect of bloat in most of the
test-cases, since GISMO found solutions that achieve
the same performance with a reduced population size.
In future work we will consider bloat as another non-
functional property to be optimized by a GI approach.
Another result was that the optimizations found by
GISMO did not modify the standard mutation opera-
tor, other than eliminating it, whereas all the other pro-
cesses of the evolutionary cycle are modified in at least
some of the experiments (selection, crossover, popula-
tion generation).
To put our results into a broader context, let us con-
sider those reported in similar works. As already men-
tioned, the individual that achieves the highest P tGain
while maintaining the lowest FGP –Gain was found in
Experiment 11, with 5.2% and −0.7% respectively. The
important modification found by GISMO for this in-
dividual is the elimination of the standard mutation.
This result in average gain is similar to the work of
Petke et al (2013) with their first attempt to improve
MiniSAT, where they found an answer to 107 out of
500 problems from the application tracks of SAT com-
petitions (21.4% of the test problems were solved),
with 2.5 % of improvement approximately. They found
that most of the modifications are deletions of asser-

Applying Genetic Improvement to a Genetic Programming library in C++ 15
tions as operations used for producing statistical data
and that the majority of the modifications found in
each individual of GISMO are neutral. In general, they
could not find a version that is significantly more ef-
ficient. In another example, as stated before, the work
by Louren¸co et al (2012) reports similar results to ours.
They evolved EAs with their proposed GE framework
to solve five instances of the Royal Road Functions
problem. Their work used GE with different optional
methods for each process of a simple EA, e.g., one evalu-
ation method (this was not modified by GE), four selec-
tion methods, four reproduction methods, and two re-
placement methods. Reported results showed that most
of the evolved EAs are equivalent to the standard EA.
The evolved EA that they considered the most interest-
ing, by its fitness and difference in structure compared
to the standard EA, achieved a better performance than
the standard EA in only one out of three instances used
for testing.
We argue that the lack of improvement is due to
the amount of genetic material GISMO has at its dis-
posal to produce individuals from the Beagle Puppy
library, that only consists of around 600 lines of source
code, in contrast to the system of 50,000 lines that
Langdon and Harman (2015a) improved. For example,
Petke et al (2014) addressed this problem with their
code transplantation, by adding more genetic material
to GISMO by extracting it from other source code li-
braries for the same task.
However, another perspective might be interest-
ing to consider. A recent work by Sipper et al (2017)
showed the extremely robust nature of the evolution-
ary search performed by GP. Their work showed that
GP performance was highly independent of the system
parametrization. Making a link between that work and
our results, it is sensible to argue that the robustness
of GP extends beyond parameter space. This in a sense
could explain why it was difficult for GISMO to find
GP variants that truly outperformed the reference GP
system.
7 Conclusion and future work
In this paper, a GI framework, called GISMO, was ap-
plied to improve the Beagle Puppy C++ library, config-
ured to perform symbolic regression on several bench-
mark and real-world problems. The objective is to im-
prove the processing time (non-functional property),
while maintaining a similar or better test-fitness of the
best individual produced by the unmodified GP cycle
of the Beagle Puppy library (functional property). The
experiments consist of applying GISMO to a set of test-
cases, where each test case is a benchmark or real-world
problem. The GISMO evolutionary cycle evolves mu-
tated versions of the original GP cycle, where each mu-
tated version performs symbolic regression for each of
the experiments’ test-cases.
It was possible to find versions of the Beagle Puppy
library with modifications in the GP approach that im-
prove the processing time in some of the problems and
maintain a low loss of the quality of the best GP indi-
vidual. Despite not finding an individual that can gen-
eralize the improvements to all test-cases, future work
must focus on determining if such a result is possible
by any means.
Similar results and recent findings suggests that the
basic evolutionary search is in fact quite robust, and it
might be very difficult to find significant improvements
in a general sense. However, results do suggest that if
we focus on a specific type of problem instances, then
in might be possible to derive improved variants of the
canonical GP search automatically through the GI ap-
proach.
It is important to emphasize that this is the first
work that uses GI to improve a GP implementation.
We hope this work encourages future research devoted
to exploiting the fruitful ideas of GI to optimize and de-
velop new GP systems, a particularly exciting endeavor
considering that GI can be seen as a successful offshoot
of GP itself. Further studies will include the use of code
transplanting where we can use multiple implementa-
tions of GP to create an improved version from them,
following Petke et al (2014). Another example is the use
of the popular grow and graft application to create and
incorporate new functionality to a GP implementation
from other GP implementations that utilize a different
GP approach. Finally, it might be reasonable to use
multi-objective search algorithms to better explore the
space of possible source code modifications as done in
(Yuan and Banzhaf, 2017; Arcuri and Yao, 2008).
Acknowledgments
This research was funded by CONACYT (Mexico)
Fronteras de la Ciencia 2015-2 Project No. FC-2015-
2:944, TecNM project 6826.18-P, and first authors was
supported by CONACYT graduate scholarship No.
302532.
Compliance with Ethical Standards
–Funding: This research was funded by Consejo Na-
cional de Ciencia y Tecnolog´ıa (CONACYT, Mex-
ico) Fronteras de la Ciencia 2015-2 Project No. FC-
2015-2:944.

16 V´ıctor R. L´opez L´opez et al.
–Funding: This research was funded by TecNM
(Mexico) Project No. 6826.18-P.
–Funding: first authors was supported by CONACYT
graduate scholarship No. 302532.
–Conflict of Interest: The authors declare that they
have no conflict of interest.
–Ethical approval: This article does not contain any
studies with human participants or animals per-
formed by any of the authors.
References
Ackling T, Alexander B, Grunert I (2011) Evolv-
ing patches for software repair. In: Proceedings of
the 13th Annual Conference on Genetic and Evolu-
tionary Computation, ACM, New York, NY, USA,
GECCO ’11, pp 1427–1434
Arcuri A, Yao X (2008) A novel co-evolutionary ap-
proach to automatic software bug fixing. In: 2008
IEEE Congress on Evolutionary Computation (IEEE
World Congress on Computational Intelligence), pp
162–168
Burke EK, Hyde MR, Kendall G, Woodward J (2007)
Automatic heuristic generation with genetic pro-
gramming: Evolving a jack-of-all-trades or a master
of one. In: Proceedings of the 9th Annual Conference
on Genetic and Evolutionary Computation, ACM,
New York, NY, USA, GECCO ’07, pp 1559–1565
Caseau Y, Laburthe F, Silverstein G (1999) A meta-
heuristic factory for vehicle routing problems. In:
Proceedings of the 5th International Conference on
Principles and Practice of Constraint Programming,
Springer-Verlag, London, UK, UK, CP ’99, pp 144–
158
Forrest S, Nguyen T, Weimer W, Le Goues C (2009)
A genetic programming approach to automated soft-
ware repair. In: Raidl G, et al (eds) GECCO ’09: Pro-
ceedings of the 11th Annual conference on Genetic
and evolutionary computation, ACM, Montreal, pp
947–954
Gagn´e C, Parizeau M (2006) Genericity in evolution-
ary computation software tools: Principles and case
study. International Journal on Artificial Intelligence
Tools 15(2):173–194
Haraldsson SO, Woodward JR (2014) Automated de-
sign of algorithms and genetic improvement: Con-
trast and commonalities. In: Proceedings of the Com-
panion Publication of the 2014 Annual Conference on
Genetic and Evolutionary Computation, ACM, New
York, NY, USA, GECCO Comp ’14, pp 1373–1380
Harman M, Petke J (2015) Gi4gi: Improving genetic im-
provement fitness functions. In: Proceedings of the
Companion Publication of the 2015 Annual Con-
ference on Genetic and Evolutionary Computation,
ACM, New York, NY, USA, GECCO Companion ’15,
pp 793–794
Harman M, Langdon WB, Jia Y, White DR, Arcuri A,
Clark JA (2012) The GISMOE challenge: construct-
ing the pareto program surface using genetic pro-
gramming to find better programs (keynote paper).
In: IEEE/ACM International Conference on Auto-
mated Software Engineering, ASE’12, Essen, Ger-
many, September 3-7, 2012, pp 1–14
Harman M, Jia Y, Langdon WB (2014) Babel pidgin:
SBSE can grow and graft entirely new functional-
ity into a real world system. In: Le Goues C, Yoo
S (eds) Proceedings of the 6th International Sympo-
sium, on Search-Based Software Engineering, SSBSE
2014, Springer, Fortaleza, Brazil, LNCS, vol 8636, pp
247–252
Harrison D, Rubinfeld DL (1978) Hedonic prices and
the demand for clean air. J Environ Economics &
Management 5:81–102
Jablkowski B, Gabor UT, Spinczyk O (2017) Evolu-
tionary planning of virtualized cyber-physical com-
pute and control clusters. Journal of Systems Archi-
tecture 73:17 – 27, special Issue on Reliable Software
Technologies for Dependable Distributed Systems
Korns MF (2011) Accuracy in Symbolic Regression,
Springer New York, New York, NY, pp 129–151
Koza JR (1992) Genetic Programming: On the Pro-
gramming of Computers by Means of Natural Selec-
tion. MIT Press, Cambridge, MA, USA
Koza JR (1994) Genetic Programming II: Automatic
Discovery of Reusable Programs. MIT Press, Cam-
bridge, MA, USA
Koza JR (2010) Human-competitive results produced
by genetic programming. Genetic Programming and
Evolvable Machines 11(3-4):251–284
Langdon W (2015) Genetic improvement of software
for multiple objectives. In: Barros M, Labiche Y (eds)
Search-Based Software Engineering, Lecture Notes in
Computer Science, vol 9275, Springer International
Publishing, pp 12–28
Langdon W, Harman M (2015a) Optimizing existing
software with genetic programming. Evolutionary
Computation, IEEE Transactions on 19(1):118–135
Langdon WB, Harman M (2010) Evolving a cuda ker-
nel from an nvidia template. In: IEEE Congress on
Evolutionary Computation, pp 1–8
Langdon WB, Harman M (2014) Genetically improved
cuda c++ software. In: Revised Selected Papers of
the 17th European Conference on Genetic Program-
ming - Volume 8599, Springer-Verlag New York, Inc.,
New York, NY, USA, EuroGP 2014, pp 87–99

Applying Genetic Improvement to a Genetic Programming library in C++ 17
Langdon WB, Harman M (2015b) Grow and graft a
better cuda pknotsrg for rna pseudoknot free energy
calculation. In: Proceedings of the Companion Publi-
cation of the 2015 Annual Conference on Genetic and
Evolutionary Computation, ACM, New York, NY,
USA, GECCO Companion ’15, pp 805–810
Langdon WB, Modat M, Petke J, Harman M (2014)
Improving 3d medical image registration cuda soft-
ware with genetic programming. In: Proceedings of
the 2014 Annual Conference on Genetic and Evolu-
tionary Computation, ACM, New York, NY, USA,
GECCO ’14, pp 951–958
Langdon WB, Lam BYH, Petke J, Harman M (2015)
Improving cuda dna analysis software with genetic
programming. In: Proceedings of the 2015 Annual
Conference on Genetic and Evolutionary Computa-
tion, ACM, New York, NY, USA, GECCO ’15, pp
1063–1070
Lichman M (2013) UCI machine learning repository
London K, Moore S, Mucci P, Seymour K, Luczak R
(2001) The papi cross-platform interface to hard-
ware performance counters. In: Department of De-
fense Users Group Conference Proceedings, pp 18–21
L´opez-L´opez VR, Trujillo L, Legrand P, Olague G
(2016) Genetic programming: From design to im-
proved implementation. In: Proceedings of the 2016
on Genetic and Evolutionary Computation Con-
ference Companion, ACM, New York, NY, USA,
GECCO ’16 Companion, pp 1147–1154
Louren¸co N, Pereira F, Costa E (2012) Evolving evolu-
tionary algorithms. In: Proceedings of the 14th An-
nual Conference Companion on Genetic and Evolu-
tionary Computation, ACM, New York, NY, USA,
GECCO ’12, pp 51–58
Manna Z, Waldinger RJ (1971) Toward automatic pro-
gram synthesis. Commun ACM 14(3):151–165
McDermott J, White DR, Luke S, Manzoni L, Castelli
M, Vanneschi L, Jaskowski W, Krawiec K, Harper R,
De Jong K, O’Reilly UM (2012) Genetic program-
ming needs better benchmarks. In: Proceedings of
the 14th Annual Conference on Genetic and Evolu-
tionary Computation, ACM, New York, NY, USA,
GECCO ’12, pp 791–798
Orlov M, Sipper M (2009) Genetic programming in the
wild: Evolving unrestricted bytecode. In: Proceedings
of the 11th Annual Conference on Genetic and Evo-
lutionary Computation, ACM, New York, NY, USA,
GECCO ’09, pp 1043–1050
Petke J, Langdon WB, Harman M (2013) Applying
genetic improvement to minisat. In: Proceedings of
the 5th International Symposium on Search Based
Software Engineering - Volume 8084, Springer-Verlag
New York, Inc., New York, NY, USA, SSBSE 2013,
pp 257–262
Petke J, Harman M, Langdon WB, Weimer W (2014)
Using genetic improvement and code transplants to
specialise a c++ program to a problem class. In: Re-
vised Selected Papers of the 17th European Con-
ference on Genetic Programming - Volume 8599,
Springer-Verlag New York, Inc., New York, NY,
USA, EuroGP 2014, pp 137–149
Petke J, Haraldsson S, Harman M, w langdon, White
D, Woodward J (2017) Genetic improvement of soft-
ware: a comprehensive survey. IEEE Transactions on
Evolutionary Computation PP(99):1–1
Petke J, Harman M, Langdon WB, Weimer W (2018)
Specialising software for different downstream appli-
cations using genetic improvement and code trans-
plantation. IEEE Transactions on Software Engineer-
ing 44(6):574–594
Poli R, Langdon WB, McPhee NF (2008) A Field Guide
to Genetic Programming. Lulu Enterprises, UK Ltd
Pozi MS, Sulaiman MN, Mustapha N, Perumal T
(2016) Improving anomalous rare attack detection
rate for intrusion detection system using support vec-
tor machine and genetic programming. Neural Pro-
cess Lett 44(2):279–290
Silva S, Costa E (2009) Dynamic limits for bloat control
in genetic programming and a review of past and cur-
rent bloat theories. Genetic Programming and Evolv-
able Machines 10(2):141–179
Sipper M, Fu W, Ahuja K, Moore JH (2017)
From MEGATON to RASCAL: surfing the pa-
rameter space of evolutionary algorithms. CoRR
abs/1706.04119
Smit SK, Eiben AE (2009) Comparing parameter tun-
ing methods for evolutionary algorithms. In: Pro-
ceedings of the Eleventh Conference on Congress on
Evolutionary Computation, IEEE Press, Piscataway,
NJ, USA, CEC’09, pp 399–406
Trujillo L, Munoz L, Galvan-Lopez E, Silva S (2016)
neat genetic programming: Controlling bloat natu-
rally. In: Doerr B, et al (eds) GECCO 2016 Hot of
the Press, ACM, Denver, USA, p hop104
Uy NQ, Hoai NX, O’Neill M, McKay RI, Galv´an-L´opez
E (2011) Semantically-based crossover in genetic pro-
gramming: application to real-valued symbolic re-
gression. Genetic Programming and Evolvable Ma-
chines 12(2):91–119
Vladislavleva EJ, Smits GF, Den Hertog D (2009) Or-
der of nonlinearity as a complexity measure for mod-
els generated by symbolic regression via pareto ge-
netic programming. Trans Evol Comp 13(2):333–349
White D, Arcuri A, Clark JA (2011) Evolutionary im-
provement of programs. Evolutionary Computation,
IEEE Transactions on 15(4):515–538

18 V´ıctor R. L´opez L´opez et al.
Woodward JR, Swan J (2011) Automatically design-
ing selection heuristics. In: Proceedings of the 13th
Annual Conference Companion on Genetic and Evo-
lutionary Computation, ACM, New York, NY, USA,
GECCO ’11, pp 583–590
Woodward JR, Swan J (2012) The automatic gener-
ation of mutation operators for genetic algorithms.
In: Proceedings of the 14th Annual Conference Com-
panion on Genetic and Evolutionary Computation,
ACM, New York, NY, USA, GECCO ’12, pp 67–74
Yeh IC (1998) Modeling of strength of high-
performance concrete using artificial neural networks.
Cement and Concrete Research 28(12):1797 – 1808
Yuan Y, Banzhaf W (2017) Arja: Automated repair of
java programs via multi-objective genetic program-
ming. CoRR abs/1712.07804