ArticlePDF Available

Applying Genetic Improvement to a Genetic Programming library in C++

Authors:

Abstract and Figures

A young subfield of evolutionary computing that has gained the attention of many researchers in recent years is genetic improvement. It uses an automated search method that directly modifies the source code or binaries of a software system to find improved versions based on some given criteria. Genetic improvement has achieved notable results and the acceptance of several research communities, namely software engineering and evolutionary computation. Over the past 10 years there have been core publications on the subject; however, we have identified, to the best of our knowledge, that there is no work on applying genetic improvement to a meta-heuristic system. In this work we apply the GI framework called GISMO to the Beagle Puppy library version 0.1 in C++, a genetic programming system configured to perform symbolic regression on several benchmark and real-world problems. The objective is to improve the processing time while maintaining a similar or better test fitness of the best individual produced by the unmodified genetic programming search. Results show that GISMO can generate individuals that present an improvement on those two key aspects over some problems, while also reducing the effects of bloat, one of the main issues in genetic programming.
Bar graphs showing PtGain(c)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Pt}_{Gain}(c)$$\end{document} (blue), FGP-Gain(c)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${F}_{GP{-}Gain}(c)$$\end{document} (orange) and Pop-size¯(c)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\overline{Pop-size}(c)$$\end{document} (yellow) for all test-casesc of the best individual for experiments 1 to 6; Pt¯Gain\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\overline{Pt}}_{Gain}$$\end{document} and F¯GP-Gain\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\overline{F}}_{GP{-}Gain}$$\end{document} are calculated over all test-cases (dashed line blue and orange, respectively). Red bold numbers (in vertical axis) indicate the test-cases used for training (color figure online)
… 
This content is subject to copyright. Terms and conditions apply.
Noname manuscript No.
(will be inserted by the editor)
Applying Genetic
Improvement to a Genetic
Programming library in C++
V´ıctor R. L´opez
opez ·Leonardo
Trujillo
·Pierrick
Legrand
Received: date / Accepted: date
Abstract A young sub-field of Evolutionary Comput-
ing that has gained the attention of many researchers in
recent years is Genetic Improvement. It uses an auto-
mated search method that directly modifies the source
code or binaries of a software system to find improved
versions based on some given criteria. Genetic Improve-
ment has achieved notable results and the acceptance
of several research communities, namely software engi-
neering and evolutionary computation. Over the past
10 years there has been core publications on the sub-
ject, however, we have identified, to the best of our
knowledge, that there is no work on applying Genetic
Improvement to a meta-heuristic system. In this work
we apply the GI framework called GISMO to the Beagle
Puppy library version 0.1 in C++, a Genetic Program-
ming system configured to perform symbolic regression
on several benchmark and real-world problems. The ob-
jective is to improve the processing time while main-
taining a similar or better test-fitness of the best indi-
vidual produced by the unmodified Genetic Program-
ming search. Results show that GISMO can generate
individuals that present an improvement on those two
key aspects over some problems, while also reducing
the effects of bloat, one of the main issues in Genetic
Programming.
V´ıctor R. L´opez L´opez ·Leonardo Trujillo
Instituto Tecnol´ogico de Tijuana
Tijuana, BC, M´exico
E-mail: vlopez@tectijuana.edu.mx
E-mail: leonardo.trujillo@tectijuana.edu.mx
Pierrick Legrand
University of Bordeaux and the IMB - Institut de
Math´ematiques de Bordeaux, UMR CNRS 5251 and Inria
Bordeaux Sud-Ouest, France
E-mail: pierrick.legrand@u-bordeaux.fr
Keywords Genetic Improvement ·Meta-heuristic ·
Genetic Programming
1 Introduction
Evolutionary Computing (EC) is a widely used com-
putational search technique from the field of artificial
intelligence. It is inspired by a simplified model of Dar-
winian evolutionary principles and contains a broad
family of Evolutionary Algorithms (EAs) for search and
optimization. EAs iteratively process a population of
candidate solutions to find the optimum of a given prob-
lem by applying at each iteration, typically, the actions:
selection of the fittest, reproduction of selected parts or
the whole population, and survival of the fittest evolved
offspring. They have achieved strong results in many
domains (Koza, 2010), and one of its most popular algo-
rithms is Genetic Programming (GP). GP is generally
used to solve machine learning problems by evolving
computer programs, in the form of symbolic expres-
sions, from a high-level problem statement; i.e., creat-
ing working programs from scratch, an aim shared by
program synthesis (Manna and Waldinger, 1971) and
automatic program induction (Koza, 1992).
An emerging sub-field of EC is Genetic Improve-
ment (GI) that, in contrast to GP, usually starts from
a working program that normally is much larger than
those evolved in GP (Petke et al, 2017). GI uses an
automated search method that directly modifies the
source code or the software binaries to find improved
versions of existing software based on a given set of
criteria for improvement (Petke et al, 2017). The crite-
ria for software improvement can be divided into func-
tional and non-functional properties. The functional
properties refer to the logic of the program, i.e., what
the program should do, an example is software repair
(bug-fixing). The non-functional properties refer to the
physical properties that the program presents, e.g. ex-
ecution time, memory storage and energy consump-
tion, just to mention a few. The strengths of GI lie
in the abundance of source code available for reuse
and the expanded search space of software variants cre-
ated by relaxing restrictions on program correctness. GI
has won human-competitive awards, “Humies”, from
the Genetic and Evolutionary Computation Conference
(GECCO)1, attracting researchers from the software
engineering and evolutionary computation communi-
ties. An exhaustive survey can be found in (Petke et al,
2017), which presents an overview of GI work gathered
from most of the publications in this field summarized
1More information about “Humies” awards can be found
at http://www.human-competitive.org/
2 V´ıctor R. L´opez L´opez et al.
into four topics. First, preserved properties focuses on
how to capture software functionality that needs to be
retained (Orlov and Sipper, 2009). Second, use of exist-
ing software deals with the reuse of source code, where
the content of the new source code can be assembled
out of existing source code, pieces of source code gener-
ated from scratch, or by transplanting source code from
a single or multiple donors (Petke et al, 2014). Third,
criteria for improvement defines non-functional prop-
erties to improve (White et al, 2011; Langdon, 2015).
Fourth, search refers to the methods used to explore the
search space of possible software versions (Forrest et al,
2009; Ackling et al, 2011). The type of software tack-
led in GI ranges from simple programs of less than 50
lines of source code, for example a program of 13 lines
of source code that represents an image operator for
a computer vision problem (L´opez-L´opez et al, 2016),
to complex software systems such as Bowtie2 that is
a DNA sequencing system consisting of 50,000 lines
of source code in C++ (Langdon and Harman, 2015a).
On the other hand, we have identified that there is no
work on applying GI to a meta-heuristic library. There
is a field related to GI called Automated Design of Algo-
rithms (ADA) (Haraldsson and Woodward, 2014) that
has been applied to improve a meta-heuristic algorithm
for a specific problem, but, to the best of our knowledge,
it has not been used to directly operate at the level of
source code.
The aim of this paper is to apply a GI frame-
work to improve a meta-heuristic system. We apply the
GI framework called GISMO (Langdon and Harman,
2015a) to the Beagle Puppy library version 0.1 in
C++ (Gagn´e and Parizeau, 2006), that is a GP system
configured to perform symbolic regression on several
benchmark and real-world problems. The objective is
to improve the processing time (non-functional proper-
ties) while maintaining a similar or better test-fitness of
the best individual produced by the unmodified GP cy-
cle of the Beagle Puppy library (functional property).
The remainder of this paper is organized as follows:
Section 2 reviews GI, focusing on GISMO. Section 3
gives an overview of ADA and describes other related
work. Experiments are described in Section 4, results
are presented in Section 5 and a discussion is exposed
in Section 6. Finally, Section 7 contains concluding re-
marks and suggestions for future work.
2 Background
GI, as mentioned before, is used to improve a software
system by using an automated search method. The idea
behind a GI framework is to start with a human writ-
ten source code and use it as genetic material for an
automated search method to explore a search space of
possible program modifications, to find a modified ver-
sion of a program that improves some given criteria,
while preserving the functionality of the original source
code. To further illustrate GI, we will concentrate
on the GISMO framework proposed by Harman et al
(2012), which is used in the experimental work of
this paper. The GISMO framework has achieved sig-
nificant results in the optimization of open-source
software, particularly in software speed-ups. Differ-
ent applications of GISMO have been developed, such
as auto-porting functionality (Langdon and Harman,
2010), improving parallel code (Langdon and Harman,
2014; Langdon et al, 2014, 2015), specialization of a
C++ program (Langdon and Harman, 2015a), code
transplant (Petke et al, 2014, 2018) and grow and graft
(Langdon and Harman, 2015b; Harman et al, 2014).
More details can be found in (Langdon, 2015).
In particular, we are interested in the specialization
of a C++ program, specifically the work presented
by Langdon and Harman (2015a) where they applied
GISMO to improve a system of 50,000 lines of source
code, producing a modified system that is, on average,
70 times faster than the original system. This work
demonstrated the ability of GI to successfully improve a
complex system. For this GISMO is used in the present
work, using the public open-source libraries available at
http://www0.cs.ucl.ac.uk/staff/ucacbbl/gismo/
in the “Free Code” section (Langdon and Harman,
2015a).
In GISMO, the original source code is represented
by a BNF grammar, and a GP search is used to ex-
plore alternate versions of the original source code by
generating individuals that represent a list of modifica-
tions to the BNF grammar. The modifications can be
the deletion of a line of source code, insertion of a line
of source code next to another, and replacement of a
line of source code with another line. The reproduction
methods of the GP search are mutation and crossover,
where mutation consist of applying one of the modi-
fications previously mentioned, and crossover consists
of the concatenation of the list of modifications from
two individuals to create a new one. The GISMO pro-
cess is illustrated in Figure 1 and can be summarized
as follows:
It starts with the source code of the software to
be improved, then a BNF grammar representation,
which describes line by line the original source code,
is created from it.
A sensitivity analysis is performed, profiling the
source code to identify which parts have the great-
est effect on the non-functional property of interest
(such as execution time).
Applying Genetic Improvement to a Genetic Programming library in C++ 3
The initial population is created of random single
mutations weighted towards the parts identified in
the previous step, corresponding to the BNF gram-
mar.
The reproduction methods mentioned above are
used to create a population of mutated individuals.
Each mutated individual is transformed to mutated
source code by reversing the grammar.
The fitness is assessed by compiling and executing
the mutated source code over a set of test-cases and
measuring the non-functional properties. The test-
cases define inputs/outputs that describe the be-
havior of the original program; these serve to evalu-
ate the functional properties. There are constraints
that affect the fitness of an individual such as suc-
cessful compilation and execution, number of test-
cases passed, and thresholds for the functional and
non-functional properties.
A selection step is carried out to determine the new
population. Based on the constraints, the eligible
individuals are sorted in ascending order to be the
parents for the next generation. At most half of the
parent population is selected to have 2 children in
the next generation, with the restriction that all
selected parents must compile. One child is a mu-
tant, the other is a crossover between a selected par-
ent and another member of the current population.
Children must be different from each other and from
the current population. If crossover cannot create a
different child, the child is created by mutating the
selected parent. If the number of selected parents is
fewer than half of the population, then two children
are created at random.
Finally, the evolutionary cycle continues until a
given number of generations.
3 Related work
ADA is a general methodology for using search methods
to improve or discover algorithms for any given com-
putational problem (Haraldsson and Woodward, 2014).
ADA is related to GI by having a similar methodology,
but differs in certain aspects, mainly in the way it is
applied. GI is applied in-situ or directly to the source
code, while ADA works ex-situ, i.e., evolves high-level
instructions or functions to manage an algorithm. Both
approaches have been used to improve non-functional
properties, but GI is the only one that has been used
for bug repairing. The work in ADA has concentrated
in three strategies, given certain information from a
problem: by replacing components with an alternative
(Woodward and Swan, 2012); by reordering the com-
ponents within the algorithm (Caseau et al, 1999); by
parameter tuning (Smit and Eiben, 2009). From these
strategies, we consider that the work from the first two
is most relevant to GI and our work. ADA has cen-
tered on the specialization of algorithms to a certain
problem or a subset of problems (Burke et al, 2007;
Woodward and Swan, 2012, 2011). However, these only
manipulate one component of a general evolutionary
process and not the whole cycle as can be done with
GI.
Another related work is from Louren¸co et al (2012),
where they applied a Grammatical Evolution (GE)
framework to automatically design an EA to solve the
Royal Road Functions problem. They showed that their
proposed framework can produce better variants for
this problem. However, they do not consider more real-
istic benchmarks or real-world problems.
Other work from GI is the specialization of Min-
iSAT by Petke et al (2013), an open-source C++ solver
for Boolean satisfiability (SAT) problems. This works
is relevant because they applied GISMO, and could not
find a significant improvement relative to the original
system. After that, they proposed the application of
code transplant (Petke et al, 2014) that lead to suc-
cessfully improve MiniSAT by using multiple versions
of the system from different authors to create one im-
proved version.
A paper that proposes a similar idea to the one
in the present paper, of using the same meta-heuristic
to optimize itself, is presented by Harman and Petke
(2015). They propose to use GI to find an optimal fit-
ness function to measure the energy consumption of a
software for a GI search. But to the best of our knowl-
edge there are no published results that were derived
from that proposal.
4 Experimental methodology
We have chosen to use the Beagle Puppy library, a GP
system implemented in C++ that is still actively used
in research (Pozi et al, 2016; Jablkowski et al, 2017).
Moreover, it is a minimalist GP library, thus being easy
to understand and interpret for future modifications.
More important, we chose the first version of Beagle
Puppy because, of all the available versions it would be
the most likely to have room for improvements given
that it was the original release.
To apply GISMO on Beagle Puppy we used the fol-
lowing setup. First, we define the test-cases used by the
GISMO approach. For this, we use standard symbolic
benchmarks proposed by McDermott et al (2012) and
real-world problems from (Lichman, 2013). The prob-
lems are summarized in Table 1 and Table 2, respec-
tively.
4 V´ıctor R. L´opez L´opez et al.
Fig. 1 Illustration of the GISMO process.
Test-case Name Benchmark function
0 Koza-1 (Koza, 1992) x4+x3+x2+x
1 Koza-2 (Koza, 1994) x5
2x3+x
2 Koza-3 (Koza, 1994) x6
2x4+x2
3 Nguyen-1 (Uy et al, 2011) x3+x2+x
4 Nguyen-3 (Uy et al, 2011) x5+x4+x3+x2+x
5 Nguyen-4 (Uy et al, 2011) x6+x5+x4+x3+x2+x
6 Nguyen-5 (Uy et al, 2011) sin(x2)cos(x)1
7 Nguyen-6 (Uy et al, 2011) sin(x) + sin(x+x2)
8 Nguyen-7 (Uy et al, 2011) ln(x+ 1) + ln(x2+ 1)
9 Korns-11 (Korns, 2011) 6.87 + 11cos(7.23x3)
Table 1 The Benchmark problems conforming part of the test-cases.
Test-case Name Total Variables
10 Concrete (Yeh, 1998) 8
11 Housing (Harrison and Rubinfeld, 1978) 13
12 Tower (Vladislavleva et al, 2009) 25
Table 2 The Real problems used as test-cases ; for each problem, there is one output variable.
Second, we generate the symbolic regression data
sets for each test-case. It is important to note that for
each test-case we generate Ndifferent data set parti-
tions, where Nis 100 for the benchmarks and 30 for
the real-world problems. For the benchmarks, the data
were randomly generated for each partition, while for
the real-world problems we used random partitions of
each complete data set.
Third, we define the training and testing sets to use
in GISMO. The training set is composed of arbitrary se-
lected test-cases, and the remaining test-cases compose
the testing set to evaluate the best individual found by
GISMO.
Fourth, we establish the reference results that will
be used to evaluate the results from the modified ver-
sions of the Beagle Puppy program (GISMO individ-
Applying Genetic Improvement to a Genetic Programming library in C++ 5
uals) for each of the test-cases. These are obtained by
executing Ntimes the original Beagle Puppy program
and calculating the median of the processing time and
test fitness measures. Finally, we establish the GISMO
and Beagle Puppy setups as follows.
4.1 GISMO setup
Representation: List of replacements, deletions and
insertions into the BNF grammar.
Fitness: The fitness function is defined by measures
that represent the gain (or loss) of each modified
GISMO individual relative to the reference individ-
ual (original code). These measures are obtained by
executing the modified code N times and calculating
the processing time gain P tGain and the test fitness
gain FGP Gain, such that fitness is given by
FGI =P tGain +FGP Gain,(1)
which is to be maximized, where
P tGain =
cn
PP tGain(c)
cn
,(2)
and
FGP Gain =
cn
PFGP Gain(c)
cn
,(3)
with crepresenting a test-case, and cnis the total
number of test-cases used in the experiment; both
represent the average gain over all cin the experi-
ment. The P tGain(c) and FGP Gain(c) are the per-
centage gain of the median processing time f
P t(c)
and median test fitness e
FGP (c) relative to their ref-
erence individual f
P tref (c) and e
FGP ref (c), given by
FGP Gain(c) = e
FGP ref (c)e
FGP (c)
e
FGP ref (c)
·100,(4)
and
P tGain(c) = f
P tref (c)f
P t(c)
f
P tref (c)
·100.(5)
The median is calculated from the N executions.
Reproduction: Single crossover operator and 3 types
of mutation are used. The crossover is a concate-
nation of list replacements. The mutations delete,
replace or insert new edits into the code, and have
equal probability of selection.
Selection: An evolved version cannot be selected to
be a parent unless it passes more than half of the
test-cases by satisfying the following constraints:
The f
P t(c) and e
FGP (c) must meet:
*f
P t(c)<200%f
P tref (c)
*e
FGP (c)>80% e
FGP ref (c)
Succeed in compilation and execution.
In the case of failing less than half of cn, we assign
f
P t(c) = 2×f
P tref (c) and e
FGP (c) = 0; we will refer to
this as fail fitness. The purpose of this is to be able
to compute the FGI with the failed test-cases. After
applying these conditions to each member of the
population, the same selection step from GISMO,
described in Section 2, is applied.
Population: Panmictic, non-elitist, generational
with 20 individuals.
Others: Initial population of random single mutants,
50% append crossover. No size limit is used for the
evolved solutions, and the stopping criterion for the
search is 200 generations.
4.2 Beagle Puppy setup
Representation: Classical Tree-based representa-
tion.
Fitness: FGP =1
1+RM SE where RMSE is the Root
Mean Square Error on the training set, performance
on the test set is computed in the same manner.
Reproduction: There are 2 types of mutation and
sub-tree crossover. The standard (Koza) mutation
and point mutation, each have a probability of 5%,
and sub-tree crossover with 90% mating probability,
and a maximum tree depth of 17.
Selection: Tournament selection with 2 participants.
Population: Steady-state, elitist with 50 individuals.
Others: Initialize population with ramped half-and-
half. The stopping criterion for the search is a max-
imum of 50 generations.
To have a better visualization of these setups, Figure
2 shows the whole evolutionary cycle, where the dashed
arrows indicate the evolutionary cycle of GISMO, as
presented in Section 2, and the solid arrows inside the
solid block (inner-left corner) indicate the evolution-
ary cycle of Beagle Puppy GP. The GISMO process
starts by defining the BNF grammar from the Beagle
Puppy Library source code; then, the initial population
is created, as mentioned in the GISMO setup, with 20
individuals but without the sensitive analysis. Instead,
we selected the parts of the BNF grammar randomly
with the intention to promote diversity in the creation
of the initial individuals. Next, the reproduction pro-
duces a mutated population; after this, each individual
is translated with the BNF grammar to generate a mu-
tated source code. All individuals are compiled, if com-
pilation fails then assign a fail measure as fitness (as
mentioned in the GISMO setup). If compilation was
6 V´ıctor R. L´opez L´opez et al.
Experiment test-cases (problems)
1 0,2,4,6,8
2 1,5,9,10
3 5,6,7,8,9,10,11
4 0,3,7,8,9,10,11
5 0,6,10,11
6 0,1,2
7 0,3,7,8,9,10,11
8 2,7,10,11
9 4,9,11
10 10
11 11
12 12
Table 3 Table of experiments that show different problems
used as test-cases for GISMO training, showing the problem
number of Tables 1 and Table 2.
Experiment FGI (%) P tGain (%) FGP Gain (%)
16.8 20.4 -13.6
2 3.5 13.4 -9.9
3 5 13.7 -8.7
4 -52.7 -58.6 5.9
5 2.6 10.7 -8.1
6 5.5 15.5 -10
7 1.6 8.8 -7.2
8 3.4 12.6 -9.2
9 -5 -4.6 -0.4
10 2.3 5.6 -3.3
11 4.5 5.2 -0.7
12 -9.6 -14.6 5
Table 4 Numerical results of the best individual for each
experiment; bold values in each column, except the leftmost,
indicate the highest value obtained over all experiments; the
shaded row indicates the experiment that achieved the high-
est P tGain maintaining a negligible loss of FGP Gain.
successful, then for each of the test-cases execute the
mutated Beagle Puppy program Ntimes, measure the
GISMO objectives (processing time and test-fitness) for
each execution and calculate f
P t(c) and e
FGP (c). If the
individual fails to execute or exceeds the permitted ex-
ecution time then assign the fail fitness. After applying
the previous conditions, compute FGI for all individu-
als. Next, apply the selection process to determine the
new mutated population. Finally, the evolutionary cy-
cle continues until the stop condition is reached.
5 Experiments and results
All the experiments were conducted on a PC with a
AMD Turion(tm) II Dual-Core Mobile M520 processor.
We use the PAPI library (London et al, 2001) to mea-
sure the processing time of the GP cycle. The experi-
ments consist of applying GISMO using different sets of
test-cases, as shown in Table 3, with the methodology
presented in the previous section.
The results of GISMO for each experiment are pre-
sented graphically in a Pareto front plot of P tGain and
FGP Gain of the evolved solutions, shown in Figure 3
and Figure 4. Each plot includes the reference individ-
ual (red dot) located at position (0,0), and a selected
individual representing the best individual of the ex-
periment (green dot); what we desire is an individual
far from zero in both axes, but most individuals are far
from such an ideal result. In other words, we chose the
individual that exhibits approximately the best com-
promise between both objectives.
The approximated Pareto front of most of the exper-
iments shows a similar behavior, with the performance
of most individuals in the range of 40% to 30% for
P tGain and 40% to 10% for FGP Gain. Between these
ranges resides the individual that was chosen to eval-
uate against the remaining test-cases (testing-set for
GISMO) of each experiment. We will refer to this indi-
vidual as the “best” individual of the experiment. The
best individual was selected based on two criteria, that
it is a non-dominated solution and that its performance
generalizes to unseen test cases.
The results of the best individuals are presented in
Figure 5 and Figure 6, in the form of bar graphs showing
the percentage of gain (or improvement) in the verti-
cal axis, obtained between the best individual of the
experiment against the reference individual (unmodi-
fied Beagle puppy program) for each test-case (horizon-
tal axis) from each experiment (row). Each test-case
shows three bars that are the processing time (blue),
test-fitness (orange) and mean size of the population
P op size(c) from the best individual found (yellow).
This last measure is only included for analysis and was
not part of the GISMO fitness but it is a critical aspect
of GP evolution (Trujillo et al, 2016). Also, the mean
gain of P tGain(c) and FGP Gain (c), both calculated
over all test-cases, are shown as dashed lines (blue and
orange, respectively). Last, each experiment shows the
test-cases used for training, indicated by red bold num-
bers. In the best case scenario, the gain should always
be higher than zero. It is not easy to determine which
experiment achieves the highest overall gain. This can
be confirmed with numerical results shown in Table 4.
The results indicate that the individual with the
highest FGI and P tGain was found in Experiment 1
with 6.8% and 20.4% respectively, it also achieved a
FGP Gain of 13.6%. Moreover, the lowest FGI but
highest FGP Gain was found in Experiment 4 with
52.7% and 5.9% respectively, it also achieved a P tGain
of 58.6%. Despite these two individuals achieving the
highest in at most two of the measures, the individual
that achieves a high P tGain while maintaining a sim-
ilar or better FGP Gain was found in Experiment 11,
Applying Genetic Improvement to a Genetic Programming library in C++ 7
Fig. 2 Illustration of the whole evolutionary cycle.
Table 5 Total and important modifications, corresponding
to the best individual found by GISMO for each experiment.
Experiment Total
modifications
Important
modifications
1 106 3
2 37 3
3 21 3
4 11 2
5 18 2
6 10 2
7 29 5
8 52 4
9 40 3
10 54 2
11 73 1
12 39 1
with 5.2% and 0.7% respectively, it also achieved a
FGIGain of 4.5%. With respect to P op size(c) in all
experiments, we observe a slightly lower gain compared
to P tGain. It its important to note that P op siz e(c)
higher than zero means reduction in the mean size of the
population. We also observe that test-cases 1, 2 and 9,
in most of the experiments, present high P op siz e(c)
and P tGain, but maintain a FGP Gain lower than -3%;
these results indicate the presence of bloat in the origi-
nal GP, which occurs when program trees tend to grow
unnecessarily large, without a corresponding increase in
fitness (Poli et al, 2008; Silva and Costa, 2009). This
means that GISMO found a version of GP that pro-
duces smaller individuals than the original GP with
negligible difference in fitness, hence, the original GP
containing bloat.
In addition to these numerical results, we analyze
the most important modifications found by GISMO for
each best individual. For this, we need to identify which
modifications of the individual are responsible for at
least some of the gain. Hence, we use a clean up proce-
dure that evaluates the P tGain and FGPGain of a ver-
sion of the analyzed individual that excludes one muta-
tion against the P tGain and FGP Gain of the same in-
dividual without excluding the modifications. One ver-
sion of the individual is generated for each of the total
number of modifications. The excluded modification is
discarded if the P tGain and FGP Gain are the same
or lower as the original. Table 5 shows the number of
modifications of the best individual from each experi-
ment and the resulting number of important modifica-
tions found with the clean up procedure. We observed
that most of the modifications found by GISMO have
a “neutral” effect in gain, and others present the dele-
tion of assert functions, which only marginally effect
P tGain.
Table 6 and Table 7 show the new and original
source code corresponding to the modifications respon-
sible for the results of each best individual. In general,
we can observe that the number of modifications re-
sponsible for the results range between 1 and 5, and
8 V´ıctor R. L´opez L´opez et al.
−40 −30 −20 −10 0 10
−40
−30
−20
−10
0
10
20
30 Pareto Front
FGPGain
PtGain
(b) Experiment 1
−40 −30 −20 −10 0 10
−40
−30
−20
−10
0
10
20
30 Pareto Front
FGPGain
PtGain
(c) Experiment 2
−40 −30 −20 −10 0 10
−40
−30
−20
−10
0
10
20
30 Pareto Front
FGPGain
PtGain
(d) Experiment 3
−40 −30 −20 −10 0 10
−40
−30
−20
−10
0
10
20
30 Pareto Front
FGPGain
PtGain
(e) Experiment 4
−40 −30 −20 −10 0 10
−40
−30
−20
−10
0
10
20
30 Pareto Front
FGPGain
PtGain
(f) Experiment 5
−40 −30 −20 −10 0 10
−40
−30
−20
−10
0
10
20
30 Pareto Front
FGPGain
PtGain
(g) Experiment 6
Fig. 3 Pareto front of experiments 1 to 6. The red circle is the reference individual and the green circle is the selected
individual to evaluate.
Applying Genetic Improvement to a Genetic Programming library in C++ 9
−40 −30 −20 −10 0 10
−40
−30
−20
−10
0
10
20
30 Pareto Front
FGPGain
PtGain
(b) Experiment 7
−40 −30 −20 −10 0 10
−40
−30
−20
−10
0
10
20
30 Pareto Front
FGPGain
PtGain
(c) Experiment 8
(d) Experiment 9 (e) Experiment 10
−40 −30 −20 −10 0 10
−40
−30
−20
−10
0
10
20
30 Pareto Front
FGPGain
PtGain
(f) Experiment 11
−40 −30 −20 −10 0 10
−40
−30
−20
−10
0
10
20
30 Pareto Front
FGPGain
PtGain
(g) Experiment 12
Fig. 4 Pareto front of experiments 7 to 12. The red circle is the reference individual and the green circle is the selected
individual to evaluate.
10 V´ıctor R. L´opez L´opez et al.
−70
−50
−30
−10
10
30
50
70
1 3 5 7 9 10 11 12
0 2 4 6 888
Gain [%]
Exp:1
20.4
−13.6
PtGain(c)FGPGain(c)Popsize(c)
f
PtGain
e
FGPGain
−70
−50
−30
−10
10
30
50
70
0 2 3 4 6 7 8 11 12
1 5 9 10101010
Gain [%]
Exp:2
13.4
−9.9
−70
−50
−30
−10
10
30
50
70
0 1 2 3 4 12
5 6 7 8 9 10 11
Gain [%]
Exp:3
13.7
−8.7
−70
−50
−30
−10
10
30
50
70
1 2 4 6 8 9 12
0 3 5 7 10 1111
Gain [%]
Exp:4
−58.6
5.9
−70
−50
−30
−10
10
30
50
70
1 2 3 4 5 7 8 9 12
0 6 10 11111111
Gain [%]
Exp:5
10.7
−8.1
−70
−50
−30
−10
10
30
50
70
3 4 5 6 7 8 9 10 11 12
0 1 22222
Gain [%]
Exp:6
15.5
−10
Test−case (c)
Fig. 5 Bar graphs showing P tGain(c) (blue),FGP Gain(c) (orange) and P op size(c) (yellow) for all test-cases cof the best
individual for experiments 1 to 6; P tGain and FGP Gain are calculated over all test-cases (dashed line blue and orange,
respectively). Red bold numbers (in vertical axis) indicate the test-cases used for training.
Applying Genetic Improvement to a Genetic Programming library in C++ 11
−70
−50
−30
−10
10
30
50
70
1 2 4 5 6 12
0 3 7 8 9 10 11
Gain [%]
Exp:7
8.8
−7.2
PtGain(c)FGPGain(c)Popsize(c)
f
PtGain
e
FGPGain
−70
−50
−30
−10
10
30
50
70
0 1 3 4 5 6 8 9 12
2 7 10 11111111
Gain [%]
Exp:8
12.6
−9.2
−70
−50
−30
−10
10
30
50
70
0 1 2 3 5 6 7 8 10 12
4 9 1111111111
Gain [%]
Exp:9
−4.6
−0.4
−70
−50
−30
−10
10
30
50
70
0 1 2 3 4 5 6 7 8 9 11 12
10101010101010
Gain [%]
Exp:10
5.6
−3.3
−70
−50
−30
−10
10
30
50
70
0 1 2 3 4 5 6 7 8 9 10 12
11111111111111
Gain [%]
Exp:11
5.2
−0.7
−70
−50
−30
−10
10
30
50
70
0 1 2 3 4 5 6 7 8 9 10 11 12121212121212
Gain [%]
Exp:12
−14.6
5
Test−case (c)
Fig. 6 Bar graphs showing P tGain (c) (blue),FGP Gain(c) (orange) and P op size(c) (yellow) for all cof the best individual
for experiments 7 to 12; P tGain and FGP Gain are calculated over all test-cases (dashed line blue and orange, respectively).
Red bold numbers (in vertical axis) indicate the test-cases used for training.
12 V´ıctor R. L´opez L´opez et al.
Table 6 The most important source code modifications found in the best individual for experiments 1 to 6.
Original sour code New source code
Experiment 1
<IF_Puppy_257><IF_Puppy_573>
if(ioContext.mRandom.rollUniform() >=
inInitGrowProba) if(ioPopulation.size() == 0)
<IF_Puppy_71>
if(ioContext.mRandom.rollUniform() <= inMatingProba) If(0)
<for2_Puppy_383>
for(unsigned int i=0; i<ioPopulation.size(); ++i) for(unsigned int i=0; 0; ++i)
Experiment 2
<IF_Puppy_473>
if(ioTree.size() > 1) if(0)
<for2_Puppy_69>
for(unsigned int i=0;i<ioPopulation.size();++i) for(unsigned int i=0;0;++i)
<_Puppy_503>
ioTree[lMutIndex].mPrimitive =
lFunction->giveReference(ioContext) //empty
Experiment 3
<for2_Puppy_69>
for(unsigned int i=0;i<ioPopulation.size();++i) for(unsigned int i=0;0;++i)
Experiment 4
<_Puppy_195>
lNode2IsTerminal =
(ioContext.mRandom.rollUniform() >= inDistribProba) //empty
<for1_Puppy_527><for1_Puppy_149>
for(unsigned int j=1; for(unsigned int j=0;
j<inNumberParticipants; ++j) j<inNumberParticipants; ++j)
Experiment 5
<_Puppy_226>
exchangeSubTrees(ioTree1, lChoosenNode1, lStack1,
ioTree2, lChoosenNode2, lStack2) //empty
<IF_Puppy_473>
if(ioTree.size() > 1) if(0)
Experiment 6
<IF_Puppy_330>
if(inMinDepth > 1) if(0)
<for2_Puppy_176>
for(unsigned int i=0; i<7; ++i) for(unsigned int i=0; 0; ++i)
that the majority of these important modifications are
deletions, and a few are replacements. The deletion of
source code may seem to be a trivial way to reduced
processing time, and in some cases maintains accept-
able accuracy on some training cases. However, trivial
solutions do not tend to generalize well to all test-cases,
while for the solutions chosen for analysis the loss in ac-
curacy is negligible in many test-cases, compared with
the improvement in processing time which is sometimes
above 40%. By analyzing what each modifications does
to the original code, we can group them into the follow-
ing groups:
No-crossover: There is no crossover. This is because
the functions related to crossover are interrupted by
modifications <IF_Puppy_71>,<for2_Puppy_69>,
<IF_Puppy_220>,<_Puppy_226> and
<for2_Puppy_176> that eliminates the if ar-
gument or the for condition.
No-mutStand: There is no standard muta-
tion. The modifications <IF_Puppy_385> and
<for2_Puppy_383> eliminate the possibility of
applying the mutation standard.
No-mutPoint: There is no point mutation. The mod-
ifications <IF_Puppy_451> and <_Puppy_453> elim-
inate the possibility of applying the mutation swap.
TreeGrow: The initial population is
only created by treegrow. The modifica-
tions <IF_Puppy_257><IF_Puppy_573> and
<IF_Puppy_257><IF_Puppy_335> replace the ar-
gument “random uniform number is more than
or equal to an initial grow probability” of the
condition that controls the method to initialize
the population with full trees with the argument
of “population size equal to zero” or “max depth
equal to 1”. By applying either arguments, the
initial population size will always be more than
zero and the max depth more than 1 thus never
Applying Genetic Improvement to a Genetic Programming library in C++ 13
Table 7 The most important source code modifications found in the best individual for experiments 7 to 12.
Original sour code New source code
Experiment 7
<IF_Puppy_257><IF_Puppy_335>
if(ioContext.mRandom.rollUniform() >= inInitGrowProba) if(inMaxDepth == 1)
<for2_Puppy_383>
for(unsigned int i=0; i<ioPopulation.size(); ++i) for(unsigned int i=0; 0; ++i)
<IF_Puppy_182>
if(ioTree1.size() > 1) if(0)
<WHILE_Puppy_187>
while(
(ioTree1[lChoosenNode1].mPrimitive->
getNumberArguments() == 0) != lNode1IsTerminal) While(0)
<IF_Puppy_330>
if(inMinDepth > 1) if(0)
Experiment 8
<IF_Puppy_451>
if(ioContext.mRandom.rollUniform() < inMutationProba) if(0)
<for2_Puppy_176>
for(unsigned int i=0; i<7; ++i) for(unsigned int i=0; 0; ++i)
<IF_Puppy_330>
if(inMinDepth > 1) if(0)
<for1_Puppy_527><for1_Puppy_149>
for(unsigned int j=1; for(unsigned int j=0;
j<inNumberParticipants; ++j) j<inNumberParticipants; ++j)
Experiment 9
<IF_Puppy_385>
if(ioContext.mRandom.rollUniform() < inMutationProba) if(0)
<_Puppy_195>
lNode2IsTerminal =
(ioContext.mRandom.rollUniform() >= inDistribProba) //empty
<WHILE_Puppy_198>
while((ioTree2[lChoosenNode2].mPrimitive->
getNumberArguments() == 0) != lNode2IsTerminal)
!= lNode2IsTerminal) While(0)
Experiment 10
<IF_Puppy_330>
if(inMinDepth > 1) if(0)
<IF_Puppy_473>
if(ioTree.size() > 1) if(0)
Experiment 11
<for2_Puppy_383>
for(unsigned int i=0; i<ioPopulation.size(); ++i) for(unsigned int i=0; 0; ++i)
Experiment 12
<for1_Puppy_527><for1_Puppy_86>
for(unsigned int j=1; for(unsigned int j=0;
j<inNumberParticipants; ++j) j<inNumberParticipants; ++j)
meeting the condition to initialize the population
with full trees.
TreeGrow-1: The individuals in the grow approach
are created by selecting the root node randomly be-
tween the function and terminal sets. The modifi-
cation <IF_Puppy_330> eliminates the condition of
selecting a function for the root node, leaving only
the viable condition of selecting a root node at ran-
dom from both the function and terminal sets.
Tournament: The number of participants in the
tournament is incremented by 1. The modifi-
cations <for1_Puppy_527><for1_Puppy_149>,
<for1_Puppy_527><for1_Puppy_149> and
<for1_Puppy_527><for1_Puppy_86> replace
the initialization “unsigned int j = 1” of the for
loop that controls the number of participants in
the tournament with “unsigned int j = 0”, thus
incrementing the number of iterations, and hence
the number of tournament participants by 1.
Matetrees-1: Only permits selecting terminal nodes
to swap in one tree of the crossover operator. The
modification <_Puppy_195> eliminates the variable
that is used for to decide if the selected node to
swap from a tree is a function or primitive based on
a 90% probability. By doing this, the variable is by
14 V´ıctor R. L´opez L´opez et al.
Experiment
No-Crossover
No-MutStand
No-mutPoint
TreeGrow
TreeGrow-1
Tournament
MateTrees-1
MateTrees-2.1
MutPoint-1
MutPoint-2
1 x x x
2 x x x
3x
4x x
5 x x
6 x x
7 x x x
8 x x x x
9 x x
10 x x
11 x
12 x
Table 8 The most important modifications found by GISMO and how they appeared in the best individual of each experiment.
default a terminal thus leaving only the possibility
to swap in terminal nodes for one of the parents.
Matetrees-2: Makes the nodes to swap completely
random in the crossover operator. The mod-
ifications <IF_Puppy_182>,<WHILE_Puppy_187>,
<_Puppy_195> and <WHILE_Puppy_198> eliminate
the 90% probability of selecting a function or ter-
minal node for swaping, thus leaving the selection
completely random.
MutPoint-1: Makes the nodes to swap completely
random in the point mutation operator. The mod-
ification <IF_Puppy_473>, similar to Matetress-2,
eliminates the probability of selecting a function or
terminal node for swaping, thus leaving the selection
completely random.
MutPoint-2: Only permits a swap in terminal nodes
in the point mutation operator. The modification
<_Puppy_503> eliminates an instruction that as-
signs the selected function node to the tree, thus
only permitting change when the selected node to
swap is a terminal.
Table 8 shows the tags corresponding to an explana-
tion of the important optimizations found by GISMO
for the best individual of each experiment. We can ob-
serve that the most frequent tag is the No-crossover,
which is present in the majority of the individuals with
high P tGain. In the opposite case, the Tournament is
present in only two experiments that achieved a high
FGP Gain. Moreover, the individual of Experiment 11
only has No-MutStand.
6 Discussion
We observe that GISMO tends to eliminate the
crossover method from GP to improve the process-
ing time, a naive way to reduce computational cost.
This is also reflected in the mean size of the popula-
tion P op size(c) in the GP runs, where the individ-
uals tend to be smaller in size, which also reduces the
computational cost of during fitness evaluation. A note-
worthy observation is that, as expected, the reference
GP system exhibits the effect of bloat in most of the
test-cases, since GISMO found solutions that achieve
the same performance with a reduced population size.
In future work we will consider bloat as another non-
functional property to be optimized by a GI approach.
Another result was that the optimizations found by
GISMO did not modify the standard mutation opera-
tor, other than eliminating it, whereas all the other pro-
cesses of the evolutionary cycle are modified in at least
some of the experiments (selection, crossover, popula-
tion generation).
To put our results into a broader context, let us con-
sider those reported in similar works. As already men-
tioned, the individual that achieves the highest P tGain
while maintaining the lowest FGP Gain was found in
Experiment 11, with 5.2% and 0.7% respectively. The
important modification found by GISMO for this in-
dividual is the elimination of the standard mutation.
This result in average gain is similar to the work of
Petke et al (2013) with their first attempt to improve
MiniSAT, where they found an answer to 107 out of
500 problems from the application tracks of SAT com-
petitions (21.4% of the test problems were solved),
with 2.5 % of improvement approximately. They found
that most of the modifications are deletions of asser-
Applying Genetic Improvement to a Genetic Programming library in C++ 15
tions as operations used for producing statistical data
and that the majority of the modifications found in
each individual of GISMO are neutral. In general, they
could not find a version that is significantly more ef-
ficient. In another example, as stated before, the work
by Louren¸co et al (2012) reports similar results to ours.
They evolved EAs with their proposed GE framework
to solve five instances of the Royal Road Functions
problem. Their work used GE with different optional
methods for each process of a simple EA, e.g., one evalu-
ation method (this was not modified by GE), four selec-
tion methods, four reproduction methods, and two re-
placement methods. Reported results showed that most
of the evolved EAs are equivalent to the standard EA.
The evolved EA that they considered the most interest-
ing, by its fitness and difference in structure compared
to the standard EA, achieved a better performance than
the standard EA in only one out of three instances used
for testing.
We argue that the lack of improvement is due to
the amount of genetic material GISMO has at its dis-
posal to produce individuals from the Beagle Puppy
library, that only consists of around 600 lines of source
code, in contrast to the system of 50,000 lines that
Langdon and Harman (2015a) improved. For example,
Petke et al (2014) addressed this problem with their
code transplantation, by adding more genetic material
to GISMO by extracting it from other source code li-
braries for the same task.
However, another perspective might be interest-
ing to consider. A recent work by Sipper et al (2017)
showed the extremely robust nature of the evolution-
ary search performed by GP. Their work showed that
GP performance was highly independent of the system
parametrization. Making a link between that work and
our results, it is sensible to argue that the robustness
of GP extends beyond parameter space. This in a sense
could explain why it was difficult for GISMO to find
GP variants that truly outperformed the reference GP
system.
7 Conclusion and future work
In this paper, a GI framework, called GISMO, was ap-
plied to improve the Beagle Puppy C++ library, config-
ured to perform symbolic regression on several bench-
mark and real-world problems. The objective is to im-
prove the processing time (non-functional property),
while maintaining a similar or better test-fitness of the
best individual produced by the unmodified GP cycle
of the Beagle Puppy library (functional property). The
experiments consist of applying GISMO to a set of test-
cases, where each test case is a benchmark or real-world
problem. The GISMO evolutionary cycle evolves mu-
tated versions of the original GP cycle, where each mu-
tated version performs symbolic regression for each of
the experiments’ test-cases.
It was possible to find versions of the Beagle Puppy
library with modifications in the GP approach that im-
prove the processing time in some of the problems and
maintain a low loss of the quality of the best GP indi-
vidual. Despite not finding an individual that can gen-
eralize the improvements to all test-cases, future work
must focus on determining if such a result is possible
by any means.
Similar results and recent findings suggests that the
basic evolutionary search is in fact quite robust, and it
might be very difficult to find significant improvements
in a general sense. However, results do suggest that if
we focus on a specific type of problem instances, then
in might be possible to derive improved variants of the
canonical GP search automatically through the GI ap-
proach.
It is important to emphasize that this is the first
work that uses GI to improve a GP implementation.
We hope this work encourages future research devoted
to exploiting the fruitful ideas of GI to optimize and de-
velop new GP systems, a particularly exciting endeavor
considering that GI can be seen as a successful offshoot
of GP itself. Further studies will include the use of code
transplanting where we can use multiple implementa-
tions of GP to create an improved version from them,
following Petke et al (2014). Another example is the use
of the popular grow and graft application to create and
incorporate new functionality to a GP implementation
from other GP implementations that utilize a different
GP approach. Finally, it might be reasonable to use
multi-objective search algorithms to better explore the
space of possible source code modifications as done in
(Yuan and Banzhaf, 2017; Arcuri and Yao, 2008).
Acknowledgments
This research was funded by CONACYT (Mexico)
Fronteras de la Ciencia 2015-2 Project No. FC-2015-
2:944, TecNM project 6826.18-P, and first authors was
supported by CONACYT graduate scholarship No.
302532.
Compliance with Ethical Standards
Funding: This research was funded by Consejo Na-
cional de Ciencia y Tecnolog´ıa (CONACYT, Mex-
ico) Fronteras de la Ciencia 2015-2 Project No. FC-
2015-2:944.
16 V´ıctor R. L´opez L´opez et al.
Funding: This research was funded by TecNM
(Mexico) Project No. 6826.18-P.
Funding: first authors was supported by CONACYT
graduate scholarship No. 302532.
Conflict of Interest: The authors declare that they
have no conflict of interest.
Ethical approval: This article does not contain any
studies with human participants or animals per-
formed by any of the authors.
References
Ackling T, Alexander B, Grunert I (2011) Evolv-
ing patches for software repair. In: Proceedings of
the 13th Annual Conference on Genetic and Evolu-
tionary Computation, ACM, New York, NY, USA,
GECCO ’11, pp 1427–1434
Arcuri A, Yao X (2008) A novel co-evolutionary ap-
proach to automatic software bug fixing. In: 2008
IEEE Congress on Evolutionary Computation (IEEE
World Congress on Computational Intelligence), pp
162–168
Burke EK, Hyde MR, Kendall G, Woodward J (2007)
Automatic heuristic generation with genetic pro-
gramming: Evolving a jack-of-all-trades or a master
of one. In: Proceedings of the 9th Annual Conference
on Genetic and Evolutionary Computation, ACM,
New York, NY, USA, GECCO ’07, pp 1559–1565
Caseau Y, Laburthe F, Silverstein G (1999) A meta-
heuristic factory for vehicle routing problems. In:
Proceedings of the 5th International Conference on
Principles and Practice of Constraint Programming,
Springer-Verlag, London, UK, UK, CP ’99, pp 144–
158
Forrest S, Nguyen T, Weimer W, Le Goues C (2009)
A genetic programming approach to automated soft-
ware repair. In: Raidl G, et al (eds) GECCO ’09: Pro-
ceedings of the 11th Annual conference on Genetic
and evolutionary computation, ACM, Montreal, pp
947–954
Gagn´e C, Parizeau M (2006) Genericity in evolution-
ary computation software tools: Principles and case
study. International Journal on Artificial Intelligence
Tools 15(2):173–194
Haraldsson SO, Woodward JR (2014) Automated de-
sign of algorithms and genetic improvement: Con-
trast and commonalities. In: Proceedings of the Com-
panion Publication of the 2014 Annual Conference on
Genetic and Evolutionary Computation, ACM, New
York, NY, USA, GECCO Comp ’14, pp 1373–1380
Harman M, Petke J (2015) Gi4gi: Improving genetic im-
provement fitness functions. In: Proceedings of the
Companion Publication of the 2015 Annual Con-
ference on Genetic and Evolutionary Computation,
ACM, New York, NY, USA, GECCO Companion ’15,
pp 793–794
Harman M, Langdon WB, Jia Y, White DR, Arcuri A,
Clark JA (2012) The GISMOE challenge: construct-
ing the pareto program surface using genetic pro-
gramming to find better programs (keynote paper).
In: IEEE/ACM International Conference on Auto-
mated Software Engineering, ASE’12, Essen, Ger-
many, September 3-7, 2012, pp 1–14
Harman M, Jia Y, Langdon WB (2014) Babel pidgin:
SBSE can grow and graft entirely new functional-
ity into a real world system. In: Le Goues C, Yoo
S (eds) Proceedings of the 6th International Sympo-
sium, on Search-Based Software Engineering, SSBSE
2014, Springer, Fortaleza, Brazil, LNCS, vol 8636, pp
247–252
Harrison D, Rubinfeld DL (1978) Hedonic prices and
the demand for clean air. J Environ Economics &
Management 5:81–102
Jablkowski B, Gabor UT, Spinczyk O (2017) Evolu-
tionary planning of virtualized cyber-physical com-
pute and control clusters. Journal of Systems Archi-
tecture 73:17 – 27, special Issue on Reliable Software
Technologies for Dependable Distributed Systems
Korns MF (2011) Accuracy in Symbolic Regression,
Springer New York, New York, NY, pp 129–151
Koza JR (1992) Genetic Programming: On the Pro-
gramming of Computers by Means of Natural Selec-
tion. MIT Press, Cambridge, MA, USA
Koza JR (1994) Genetic Programming II: Automatic
Discovery of Reusable Programs. MIT Press, Cam-
bridge, MA, USA
Koza JR (2010) Human-competitive results produced
by genetic programming. Genetic Programming and
Evolvable Machines 11(3-4):251–284
Langdon W (2015) Genetic improvement of software
for multiple objectives. In: Barros M, Labiche Y (eds)
Search-Based Software Engineering, Lecture Notes in
Computer Science, vol 9275, Springer International
Publishing, pp 12–28
Langdon W, Harman M (2015a) Optimizing existing
software with genetic programming. Evolutionary
Computation, IEEE Transactions on 19(1):118–135
Langdon WB, Harman M (2010) Evolving a cuda ker-
nel from an nvidia template. In: IEEE Congress on
Evolutionary Computation, pp 1–8
Langdon WB, Harman M (2014) Genetically improved
cuda c++ software. In: Revised Selected Papers of
the 17th European Conference on Genetic Program-
ming - Volume 8599, Springer-Verlag New York, Inc.,
New York, NY, USA, EuroGP 2014, pp 87–99
Applying Genetic Improvement to a Genetic Programming library in C++ 17
Langdon WB, Harman M (2015b) Grow and graft a
better cuda pknotsrg for rna pseudoknot free energy
calculation. In: Proceedings of the Companion Publi-
cation of the 2015 Annual Conference on Genetic and
Evolutionary Computation, ACM, New York, NY,
USA, GECCO Companion ’15, pp 805–810
Langdon WB, Modat M, Petke J, Harman M (2014)
Improving 3d medical image registration cuda soft-
ware with genetic programming. In: Proceedings of
the 2014 Annual Conference on Genetic and Evolu-
tionary Computation, ACM, New York, NY, USA,
GECCO ’14, pp 951–958
Langdon WB, Lam BYH, Petke J, Harman M (2015)
Improving cuda dna analysis software with genetic
programming. In: Proceedings of the 2015 Annual
Conference on Genetic and Evolutionary Computa-
tion, ACM, New York, NY, USA, GECCO ’15, pp
1063–1070
Lichman M (2013) UCI machine learning repository
London K, Moore S, Mucci P, Seymour K, Luczak R
(2001) The papi cross-platform interface to hard-
ware performance counters. In: Department of De-
fense Users Group Conference Proceedings, pp 18–21
opez-L´opez VR, Trujillo L, Legrand P, Olague G
(2016) Genetic programming: From design to im-
proved implementation. In: Proceedings of the 2016
on Genetic and Evolutionary Computation Con-
ference Companion, ACM, New York, NY, USA,
GECCO ’16 Companion, pp 1147–1154
Louren¸co N, Pereira F, Costa E (2012) Evolving evolu-
tionary algorithms. In: Proceedings of the 14th An-
nual Conference Companion on Genetic and Evolu-
tionary Computation, ACM, New York, NY, USA,
GECCO ’12, pp 51–58
Manna Z, Waldinger RJ (1971) Toward automatic pro-
gram synthesis. Commun ACM 14(3):151–165
McDermott J, White DR, Luke S, Manzoni L, Castelli
M, Vanneschi L, Jaskowski W, Krawiec K, Harper R,
De Jong K, O’Reilly UM (2012) Genetic program-
ming needs better benchmarks. In: Proceedings of
the 14th Annual Conference on Genetic and Evolu-
tionary Computation, ACM, New York, NY, USA,
GECCO ’12, pp 791–798
Orlov M, Sipper M (2009) Genetic programming in the
wild: Evolving unrestricted bytecode. In: Proceedings
of the 11th Annual Conference on Genetic and Evo-
lutionary Computation, ACM, New York, NY, USA,
GECCO ’09, pp 1043–1050
Petke J, Langdon WB, Harman M (2013) Applying
genetic improvement to minisat. In: Proceedings of
the 5th International Symposium on Search Based
Software Engineering - Volume 8084, Springer-Verlag
New York, Inc., New York, NY, USA, SSBSE 2013,
pp 257–262
Petke J, Harman M, Langdon WB, Weimer W (2014)
Using genetic improvement and code transplants to
specialise a c++ program to a problem class. In: Re-
vised Selected Papers of the 17th European Con-
ference on Genetic Programming - Volume 8599,
Springer-Verlag New York, Inc., New York, NY,
USA, EuroGP 2014, pp 137–149
Petke J, Haraldsson S, Harman M, w langdon, White
D, Woodward J (2017) Genetic improvement of soft-
ware: a comprehensive survey. IEEE Transactions on
Evolutionary Computation PP(99):1–1
Petke J, Harman M, Langdon WB, Weimer W (2018)
Specialising software for different downstream appli-
cations using genetic improvement and code trans-
plantation. IEEE Transactions on Software Engineer-
ing 44(6):574–594
Poli R, Langdon WB, McPhee NF (2008) A Field Guide
to Genetic Programming. Lulu Enterprises, UK Ltd
Pozi MS, Sulaiman MN, Mustapha N, Perumal T
(2016) Improving anomalous rare attack detection
rate for intrusion detection system using support vec-
tor machine and genetic programming. Neural Pro-
cess Lett 44(2):279–290
Silva S, Costa E (2009) Dynamic limits for bloat control
in genetic programming and a review of past and cur-
rent bloat theories. Genetic Programming and Evolv-
able Machines 10(2):141–179
Sipper M, Fu W, Ahuja K, Moore JH (2017)
From MEGATON to RASCAL: surfing the pa-
rameter space of evolutionary algorithms. CoRR
abs/1706.04119
Smit SK, Eiben AE (2009) Comparing parameter tun-
ing methods for evolutionary algorithms. In: Pro-
ceedings of the Eleventh Conference on Congress on
Evolutionary Computation, IEEE Press, Piscataway,
NJ, USA, CEC’09, pp 399–406
Trujillo L, Munoz L, Galvan-Lopez E, Silva S (2016)
neat genetic programming: Controlling bloat natu-
rally. In: Doerr B, et al (eds) GECCO 2016 Hot of
the Press, ACM, Denver, USA, p hop104
Uy NQ, Hoai NX, O’Neill M, McKay RI, Galv´an-L´opez
E (2011) Semantically-based crossover in genetic pro-
gramming: application to real-valued symbolic re-
gression. Genetic Programming and Evolvable Ma-
chines 12(2):91–119
Vladislavleva EJ, Smits GF, Den Hertog D (2009) Or-
der of nonlinearity as a complexity measure for mod-
els generated by symbolic regression via pareto ge-
netic programming. Trans Evol Comp 13(2):333–349
White D, Arcuri A, Clark JA (2011) Evolutionary im-
provement of programs. Evolutionary Computation,
IEEE Transactions on 15(4):515–538
18 V´ıctor R. L´opez L´opez et al.
Woodward JR, Swan J (2011) Automatically design-
ing selection heuristics. In: Proceedings of the 13th
Annual Conference Companion on Genetic and Evo-
lutionary Computation, ACM, New York, NY, USA,
GECCO ’11, pp 583–590
Woodward JR, Swan J (2012) The automatic gener-
ation of mutation operators for genetic algorithms.
In: Proceedings of the 14th Annual Conference Com-
panion on Genetic and Evolutionary Computation,
ACM, New York, NY, USA, GECCO ’12, pp 67–74
Yeh IC (1998) Modeling of strength of high-
performance concrete using artificial neural networks.
Cement and Concrete Research 28(12):1797 – 1808
Yuan Y, Banzhaf W (2017) Arja: Automated repair of
java programs via multi-objective genetic program-
ming. CoRR abs/1712.07804
... Time is the concern addressed in the vast majority of papers, with 34 papers considering execution time [1, 2, 7, 8, 10, 14, 15, 17, 24, 32, 35, 39, 41-44, 47-50, 55, 58-63, 68, 70-72, 75, 77, 87, 88], number of CPU or bytecode instructions [4,11,12,21,22,85], or also loading time [23]. Other NFPs include code size [25,38,90,91], energy consumption [13,18,19,27], memory usage [7,8,88], accuracy of the underlying algorithm [30,31,59,60,62,81], readability [73], or other application-specific NFPs [37, 40, 45, 46, 51-53, 64, 65]. A summary is presented in Figure 1. ...
... A summary is presented in Figure 1. Furthermore, a few pieces of work considered multiple NFPs [7,8,27,59,60,62,88]. ...
Conference Paper
Genetic improvement (GI) improves both functional properties of software, such as bug repair, and non-functional properties, such as execution time, energy consumption, or source code size. There are studies summarising and comparing GI tools for improving functional properties of software; however there is no such study for improvement of its non-functional properties using GI. Therefore, this research aims to survey and report on the existing GI tools for improvement of non-functional properties of software. We conducted a literature review of available GI tools, and ran multiple experiments on the found open-source tools to examine their usability. We applied a cross-testing strategy to check whether the available tools can work on different programs. Overall, we found 63 GI papers that use a GI tool to improve nonfunctional properties of software, within which 31 are accompanied with open-source code. We were able to successfully run eight GI tools, and found that ultimately only two ---Gin and PyGGI--- can be readily applied to new general software.
... Lately, recent work has begun to explore in-situ algorithmic improvements of existing heuristic tools, with a technique called Genetic Improvement that operates directly at the level of source code [45]. One study has applied Genetic Improvement on a GP library directly [46]. Through this revision, we observe that GP can improve different kinds of heuristic mechanisms, which is the main path that this study will follow to generate fast approximations to the HV indicator for MO problems. ...
Article
Hypervolume (HV) has become one of the most popular indicators to assess the quality of Pareto front approximations. However, the best algorithm for computing these values has a computational complexity of O(Nk/3polylog(N)) for N candidate solutions and k objectives. In this study, we propose a regression-based approach to learn new mathematical expressions to approximate the HV value and improve at the same time their computational efficiency. In particular, Genetic Programming is used as the modeling technique, because it can produce compact and efficient symbolic models. To evaluate this approach, we exhaustively measure the deviation of the new models against the real HV values using the DTLZ and WFG benchmark suites. We also test the new models using them as a guiding mechanism within the indicator-based algorithm SMS-EMOA. The results are very consistent and promising since the new models report very low errors and a high correlation for problems with 3, 4, and 5 objectives. What is more striking is the execution time achieved by these models, which in a direct comparison against standard HV calculation achieved extremely high speedups of close to 100X for a single front and over 1000X for all the HV contributions in a population, speedups reach over 10X in full runs of SMS-EMOA compared with the standard Monte Carlo approximations of the HV, particularly for large population sizes. Finally, the evolved models generalize across multiple complex problems, using only two problems to train the problems from the DTLZ benchmark and performing efficiently and effectively on all remaining DTLZ and WFG benchmark problems.
... Some of these works have focused on speed improvement, e.g. in the case of CUDA code [Langdon, William B. and Harman, Mark 2014a;Langdon, William B. and Modat, Marc and Petke, Justyna and Harman, Mark 2014] and CUDA-based sequencing tools such as BarraCUDA [Langdon, William B. and Lam, Brian Yee Hong 2017], complex C/C++ applications such as Bowtie2 [Langdon, William B. and Harman, Mark 2014b], the OpenCV library [Langdon, William B. and White, David R. and Harman, Mark and Jia, Yue and Petke, Justyna 2016], or the MiniSAT solver [Petke, Justyna and Harman, Mark and Langdon, William B. and Weimer, Westley 2014]. A particular case of GI aimed at speed improvement is discussed in [López-López, Víctor R. and Trujillo, Leonardo and Legrand, Pierrick 2019], where the authors applied GI to a C++ GP library and noted a speedup due to the deletion of some operators (crossover, point mutation and others). Another area of GI research is the accuracy improvement of low-level implementations of various mathematical functions, such as [Langdon, William B 2019], 2 [Langdon, WB 2018;Langdon, William B. and Petke, Justyna 2019] and other functions based on lookup tables [Krauss, Oliver and Langdon, William B 2020]. ...
... Some of these works have focused on speed improvement, e.g. in the case of CUDA code [Langdon, William B. and Harman, Mark 2014a;Langdon, William B. and Modat, Marc and Petke, Justyna and Harman, Mark 2014] and CUDA-based sequencing tools such as BarraCUDA [Langdon, William B. and Lam, Brian Yee Hong 2017], complex C/C++ applications such as Bowtie2 [Langdon, William B. and Harman, Mark 2014b], the OpenCV library [Langdon, William B. and White, David R. and Harman, Mark and Jia, Yue and Petke, Justyna 2016], or the MiniSAT solver [Petke, Justyna and Harman, Mark and Langdon, William B. and Weimer, Westley 2014]. A particular case of GI aimed at speed improvement is discussed in [López-López, Víctor R. and Trujillo, Leonardo and Legrand, Pierrick 2019], where the authors applied GI to a C++ GP library and noted a speedup due to the deletion of some operators (crossover, point mutation and others). Another area of GI research is the accuracy improvement of low-level implementations of various mathematical functions, such as [Langdon, William B 2019], 2 [Langdon, WB 2018;Langdon, William B. and Petke, Justyna 2019] and other functions based on lookup tables [Krauss, Oliver and Langdon, William B 2020]. ...
Article
Routing plays a fundamental role in network applications, but it is especially challenging in Delay Tolerant Networks (DTNs). These are a kind of mobile ad hoc networks made of, e.g., (possibly, unmanned) vehicles and humans where, despite a lack of continuous connectivity, data must be transmitted while the network conditions change due to the nodes’ mobility. In these contexts, routing is NP-hard and is usually solved by heuristic “store and forward” replication-based approaches, where multiple copies of the same message are moved and stored across nodes in the hope that at least one will reach its destination. Still, the existing routing protocols produce relatively low delivery probabilities. Here, we genetically improve two routing protocols widely adopted in DTNs, namely, Epidemic and PRoPHET, in the attempt to optimize their delivery probability. First, we dissect them into their fundamental components, i.e., functionalities such as checking if a node can transfer data, or sending messages to all connections. Then, we apply Genetic Improvement (GI) to manipulate these components as terminal nodes of evolving trees. We apply this methodology, in silico, to six test cases of urban networks made of hundreds of nodes and find that GI produces consistent gains in delivery probability in four cases. We then verify if this improvement entails a worsening of other relevant network metrics, such as latency and buffer time. Finally, we compare the logics of the best evolved protocols with those of the baseline protocols, and we discuss the generalizability of the results across test cases.
... Some of these works have focused on speed improvement, e.g. in the case of CUDA code [Langdon, William B. and Harman, Mark 2014a;Langdon, William B. and Modat, Marc and Petke, Justyna and Harman, Mark 2014] and CUDA-based sequencing tools such as BarraCUDA [Langdon, William B. and Lam, Brian Yee Hong 2017], complex C/C++ applications such as Bowtie2 [Langdon, William B. and Harman, Mark 2014b], the OpenCV library [Langdon, William B. and White, David R. and Harman, Mark and Jia, Yue and Petke, Justyna 2016], or the MiniSAT solver [Petke, Justyna and Harman, Mark and Langdon, William B. and Weimer, Westley 2014]. A particular case of GI aimed at speed improvement is discussed in [López-López, Víctor R. and Trujillo, Leonardo and Legrand, Pierrick 2019], where the authors applied GI to a C++ GP library and noted a speedup due to the deletion of some operators (crossover, point mutation and others). Another area of GI research is the accuracy improvement of low-level implementations of various mathematical functions, such as [Langdon, William B 2019], 2 [Langdon, WB 2018;Langdon, William B. and Petke, Justyna 2019] and other functions based on lookup tables [Krauss, Oliver and Langdon, William B 2020]. ...
Preprint
Full-text available
Routing plays a fundamental role in network applications, but it is especially challenging in Delay Tolerant Networks (DTNs). These are a kind of mobile ad hoc networks made of e.g. (possibly, unmanned) vehicles and humans where, despite a lack of continuous connectivity, data must be transmitted while the network conditions change due to the nodes' mobility. In these contexts, routing is NP-hard and is usually solved by heuristic "store and forward" replication-based approaches, where multiple copies of the same message are moved and stored across nodes in the hope that at least one will reach its destination. Still, the existing routing protocols produce relatively low delivery probabilities. Here, we genetically improve two routing protocols widely adopted in DTNs, namely Epidemic and PRoPHET, in the attempt to optimize their delivery probability. First, we dissect them into their fundamental components, i.e., functionalities such as checking if a node can transfer data, or sending messages to all connections. Then, we apply Genetic Improvement (GI) to manipulate these components as terminal nodes of evolving trees. We apply this methodology, in silico, to six test cases of urban networks made of hundreds of nodes, and find that GI produces consistent gains in delivery probability in four cases. We then verify if this improvement entails a worsening of other relevant network metrics, such as latency and buffer time. Finally, we compare the logics of the best evolved protocols with those of the baseline protocols, and we discuss the generalizability of the results across test cases.
... This topic has been studied extensively for EAs, but has received less attention in genetic programming (GP) (Koza 1992;Langdon and Poli 2010). In GP, an EA is used to search for models, functions or small programs that perform a particular computational process, usually following a supervised learning problem formulation, such as regression or automatic improvement of code (López-López et al. 2018). Probably, the most extensive experimental study regarding hyper-parameter optimization for GP was recently published by Sipper et al. (2018). ...
Article
Full-text available
Evolutionary algorithms (EAs) have been with us for several decades and are highly popular given that they have proved competitive in the face of challenging problems’ features such as deceptiveness, multiple local optima, among other characteristics. However, it is necessary to define multiple hyper-parameter values to have a working EA, which is a drawback for many practitioners. In the case of genetic programming (GP), an EA for the evolution of models and programs, hyper-parameter optimization has been extensively studied only recently. This work builds on recent findings and explores the hyper-parameter space of a specific GP system called neat-GP that controls model size. This is conducted using two large sets of symbolic regression benchmark problems to evaluate system performance, while hyper-parameter optimization is carried out using three variants of the iterated F-Race algorithm, for the first time applied to GP. From all the automatic parametrizations produced by optimization process, several findings are drawn. Automatic parametrizations do not outperform the manual configuration in many cases, and overall, the differences are not substantial in terms of testing error. Moreover, finding parametrizations that produce highly accurate models that are also compact is not trivially done, at least if the hyper-parameter optimization process (F-Race) is only guided by predictive error. This work is intended to foster more research and scrutiny of hyper-parameters in EAs, in general, and GP, in particular.
Preprint
Full-text available
Performance is a key quality of modern software. Although recent years have seen a spike in research on automated improvement of software's execution time, energy, memory consumption, etc., there is a noticeable lack of standard benchmarks for such work. It is also unclear how such benchmarks are representative of current software. Furthermore, frequently non-functional properties of software are targeted for improvement one-at-a-time, neglecting potential negative impact on other properties. In order to facilitate more research on automated improvement of non-functional properties of software, we conducted a survey gathering benchmarks used in previous work. We considered 5 major online repositories of software engineering work: ACM Digital Library, IEEE Xplore, Scopus, Google Scholar, and ArXiV. We gathered 5000 publications (3749 unique), which were systematically reviewed to identify work that empirically improves non-functional properties of software. We identified 386 relevant papers. We find that execution time is the most frequently targeted property for improvement (in 62% of relevant papers), while multi-objective improvement is rarely considered (5%). Static approaches are prevalent (in 53% of papers), with exploratory approaches (evolutionary in 18% and non-evolutionary in 14% of papers) increasingly popular in the last 10 years. Only 40% of 386 papers describe work that uses benchmark suites, rather than single software, of those SPEC is most popular (covered in 33 papers). We also provide recommendations for choice of benchmarks in future work, noting, e.g., lack of work that covers Python or JavaScript. We provide all programs found in the 386 papers on our dedicated webpage at https://bloa.github.io/nfunc_survey/ We hope that this effort will facilitate more research on the topic of automated improvement of software's non-functional properties.
Article
Full-text available
Stock markets are a popular kind of financial markets because of the possibility of bringing high revenues to their investors. To reduce risk factors for investors, intelligent and automated stock market forecast tools are developed by using computational intelligence techniques. This study presents a hyperparameter optimal genetic programming-based forecast model generation algorithm for a-day-ahead prediction of stock market index trends. To obtain an optimal forecast model from the modeling dataset, a differential evolution (DE) algorithm is employed to optimize hyperparameters of the genetic programming orthogonal least square (GpOls) algorithm. Thus, evolution of GpOls agents within the hyperparameter search space enables adaptation of the GpOls algorithm for the modeling dataset. This evolutionary hyperparameter optimization technique can enhance the data-driven modeling performance of the GpOls algorithm and allow the optimal autotuning of user-defined parameters. In the current study, the proposed DE-based hyper-GpOls (DEHypGpOls) algorithm is used to generate forecaster models for prediction of a-day-ahead trend prediction for the Istanbul Stock Exchange 100 (ISE100) and the Borsa Istanbul 100 (BIST100) indexes. In this experimental study, daily trend data from ISE100 and BIST100 and seven other international stock markets are used to generate a-day-ahead trend forecaster models. Experimental studies on 4 different time slots of stock market index datasets demonstrated that the forecast models of the DEHypGpOls algorithm could provide 57.87% average accuracy in buy–sell recommendations. The market investment simulations with these datasets showed that daily investments to the ISE100 and BIST100 indexes according to buy or sell signals of the forecast model of DEHypGpOls could provide 4.8% more average income compared to the average income of a long-term investment strategy.
Preprint
Full-text available
Stock markets are one of the most popular financial markets since they can bring high revenues to their investors. In order to reduce the risk factor for investors, intelligent and automated stock market trend prediction tools are developed by using computational intelligence methods. This study presents a hyper-parameter optimal Genetic Programming (GP) model generation framework for a day-ahead prediction of stock market index trends. In order to obtain the best trend prediction model from the stock market data, a Differential Evolution (DE) algorithm is implemented to optimize hyper-parameters of a GP algorithm with Orthogonal Least Square (GpOls). Evolutions of GpOls agents within the hyper-parameter search space allow adaptation of GpOls algorithm for a given training dataset by enabling optimal autotuning of user-defined parameters. Authors demonstrated that this structure can significantly improve modeling performances of GpOls algorithms for data-driven modeling applications. In the study, the proposed DE based hyper GpOls (DEHypGpOls) algorithm is implemented for a day-ahead forecast problem of the Istanbul Stock Exchange 100 Index (ISE100) trends. In the experimental study, real trend data from ISE100 and seven other international stock markets are used to extract a day-ahead forecast model of daily ISE100 trends. The forecast model of the DEHypGpOls algorithm can provide 68% prediction accuracy for the test dataset. In ISE100 index market simulations, daily exchanges of ISE100 index according to buy or sell signal of the forecast model of DEHypGpOls provided about 273% more income compared to the income of a long-term investment strategy.
Article
Full-text available
Evolutionary computation (EC) has been widely applied to biological and biomedical data. The practice of EC involves the tuning of many parameters, such as population size, generation count, selection size, and crossover and mutation rates. Through an extensive series of experiments over multiple evolutionary algorithm implementations and 25 problems we show that parameter space tends to be rife with viable parameters, at least for the problems studied herein. We discuss the implications of this finding in practice for the researcher employing EC.
Article
Full-text available
Recent empirical studies show that the performance of GenProg is not satisfactory, particularly for Java. In this paper, we propose ARJA, a new GP based repair approach for automated repair of Java programs. To be specific, we present a novel lower-granularity patch representation that properly decouples the search subspaces of likely-buggy locations, operation types and potential fix ingredients, enabling GP to explore the search space more effectively. Based on this new representation, we formulate automated program repair as a multi-objective search problem and use NSGA-II to look for simpler repairs. To reduce the computational effort and search space, we introduce a test filtering procedure that can speed up the fitness evaluation of GP and three types of rules that can be applied to avoid unnecessary manipulations of the code. Moreover, we also propose a type matching strategy that can create new potential fix ingredients by exploiting the syntactic patterns of the existing statements. We conduct a large-scale empirical evaluation of ARJA along with its variants on both seeded bugs and real-world bugs in comparison with several state-of-the-art repair approaches. Our results verify the effectiveness and efficiency of the search mechanisms employed in ARJA and also show its superiority over the other approaches. In particular, compared to jGenProg (an implementation of GenProg for Java), an ARJA version fully following the redundancy assumption can generate a test-suite adequate patch for more than twice the number of bugs (from 27 to 59), and a correct patch for nearly four times of the number (from 5 to 18), on 224 real-world bugs considered in Defects4J. Furthermore, ARJA is able to correctly fix several real multi-location bugs that are hard to be repaired by most of the existing repair approaches.
Article
Full-text available
Genetic improvement uses automated search to find improved versions of existing software. We present a comprehensive survey of this nascent field of research with a focus on the core papers in the area published between 1995 and 2015. We identified core publications including empirical studies, 96% of which use evolutionary algorithms (genetic programming in particular). Although we can trace the foundations of genetic improvement back to the origins of computer science itself, our analysis reveals a significant upsurge in activity since 2012. Genetic improvement has resulted in dramatic performance improvements for a diverse set of properties such as execution time, energy and memory consumption, as well as results for fixing and extending existing system functionality. Moreover, we present examples of research work that lies on the boundary between genetic improvement and other areas, such as program transformation, approximate computing, and software repair, with the intention of encouraging further exchange of ideas between researchers in these fields.
Conference Paper
Full-text available
We genetically improve BarraCUDA using a BNF grammar incorporating C scoping rules with GP. Barracuda maps next generation DNA sequences to the human genome using the Burrows-Wheeler algorithm (BWA) on nVidia Tesla parallel graphics hardware (GPUs). GI using phenotypic tabu search with manually grown code can graft new features giving more than 100 fold speed up on a performance critical kernel without loss of accuracy.
Article
Genetic improvement uses computational search to improve existing software while retaining its partial functionality. Genetic improvement has previously been concerned with improving a system with respect to all possible usage scenarios. In this paper, we show how genetic improvement can also be used to achieve specialisation to a specific set of usage scenarios. We use genetic improvement to evolve faster versions of a C++ program, a Boolean satisfiability solver called MiniSAT, specialising it for three applications. Our specialised solvers achieve between 4% and 36% execution time improvement, which is commensurate with efficiency gains achievable using human expert optimisation for the general solver. We also use genetic improvement to evolve faster versions of an image processing tool called ImageMagick, utilising code from GraphicsMagick, another image processing tool which was forked from it. We specialise the format conversion functionality to black & white images and colour images only. Our specialised versions achieve up to 3% execution time improvement.
Article
The practice of evolutionary algorithms involves a mundane yet inescapable phase, namely, finding parameters that work well. How big should the population be? How many generations should the algorithm run? What is the (tournament selection) tournament size? What probabilities should one assign to crossover and mutation? All these nagging questions need good answers if one is to embrace success. Through an extensive series of experiments over multiple evolutionary algorithm implementations and problems we show that parameter space tends to be rife with viable parameters. We aver that this renders the life of the practitioner that much easier, and cap off our study with an advisory digest for the weary.
Article
Virtualization technology has the potential to notably advance the automation process in the domain of cyber-physical systems (CPS). It can improve both dependability and availability as well as significantly reduce the procurement, operation and maintenance costs of such systems. However, in the context of virtualization, research has put the most emphasis on topics of hardware utilization and fault-tolerance. There is little literature on how to model, integrate and consolidate a CPS by means of virtualization. In this paper we present a methodology for planning safe and efficient virtualized cyber-physical compute and control clusters – execution platforms for time-constrained virtual machines (VMs) that encapsulate CPS applications. We discuss the used methods, describe the corresponding models and the required system architecture. In contrast to typical resource allocation problems from other domains (e.g. cloud computing), in this case, the planning process must take real-time requirements of applications into account. In order to achieve this, we combine evolutionary algorithms with formal system performance analysis – in particular algorithms considered in classical scheduling theory. Such an approach allows not only to optimally dimension the compute and control clusters, but also provides strict guarantees regarding the timing predictability of the integrated CPS. Further, the embedment of a formal performance analysis technique notably eases the modeling of a system. As a consequence, the modeling process is fast, flexible and accessible not only to experts but also to system designers as they do not have to struggle with complex and time consuming mathematical formulations. Finally, our approach also provides answers to several practical questions that arise when integrating a CPS by means of virtualization.
Conference Paper
Genetic programming (GP) is an evolutionary-based search paradigm that is well suited to automatically solve difficult design problems. The general principles of GP have been used to evolve mathematical functions, models, image operators, programs, and even antennas and lenses. Since GP evolves the syntax and structure of a solution, the evolutionary process can be carried out in one environment and the solution can then be ported to another. However, given the nature of GP it is common that the evolved designs are unorthodox compared to traditional approaches used in the problem domain. Therefore, efficiently porting, improving or optimizing an evolved design might not be a trivial task. In this work we argue that the same GP principles used to evolve the solution can then be used to optimize a particular new implementation of the design, following the Genetic Improvement approach. In particular, this paper presents a case study where evolved image operators are ported from Matlab to OpenCV, and then the source code is optimized an improved using Genetic Improvement of Software for Multiple Objectives (GISMOE). In the example we show that functional behavior is maintained (output image) while improving non-functional properties (computation time). Despite the fact that this first example is a simple case, it clearly illustrates the possibilities of using GP principles in two distinct stages of the software development process, from design to improved implementation.
Conference Paper
Genetic programming (GP) can increase computer program’s functional and non-functional performance. It can automatically port or refactor legacy code written by domain experts. Working with programmers it can grow and graft (GGGP) new functionality into legacy systems and parallel Bioinformatics GPGPU code. We review Genetic Improvement (GI) and SBSE research on evolving software.
Conference Paper
Grow and graft genetic programming greatly improves GPGPU dynamic programming software for predicting the minimum binding energy for folding of RNA molecules. The parallel code inserted into the existing CUDA version of pknots was grown using a BNF grammar. On an nVidia Tesla K40 GPU GGGP gives a speed up of up to 10000 times.