No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Cartesian Genetic Programming on the GPU
Abstract
Cartesian Genetic Programming is a form of Genetic Programming based on evolving graph structures. It has a fixed genotype length and a genotype–phenotype mapping that introduces neutrality into the representation. It has been used for many applications and was one of the first Genetic Programming techniques to be implemented on the GPU. In this chapter, we describe the representation in detail and discuss various GPU implementations of it. Later in the chapter, we discuss a recent implementation based on the GPU.net framework.
... To represent our functional programs we use the Cartesian Genetic Programming framework [16]. CGP is a form of Genetic Programming in which computations are organized using a matrix of nodes as depicted in Figure 1. ...
... Depending on the various C ij , not all nodes affects the output so that only the active nodes need to be actually computed. The connections C ij are also here associated to multiplicative factors (weights) The general form of a CGP as described in [16]. In our version weights are also assigned to the connections Cij so that the program is defined by some integer values (the connections, the functions and the outputs) and by some floating point values (the connection weights) w i,j . ...
... Let us consider a CGP expression having n = 3, m = 1, r = 1, c = 10, l = 3, a = 2, w i,j = 1 and the following kernel functions: +,*,/,σ where σ is the sigmoid function. Using the same convention used in [16] for the node numbering, the chromosome: ...
We introduce the use of high order automatic differentiation, implemented via the algebra of truncated Taylor polynomials, in genetic programming. Using the Cartesian Genetic Programming encoding we obtain a high-order Taylor representation of the program output that is then used to back-propagate errors during learning. The resulting machine learning framework is called differentiable Cartesian Genetic Programming (dCGP). In the context of symbolic regression, dCGP offers a new approach to the long unsolved problem of constant representation in GP expressions. On several problems of increasing complexity we find that dCGP is able to find the exact form of the symbolic expression as well as the constants values. We also demonstrate the use of dCGP to solve a large class of differential equations and to find prime integrals of dynamical systems, presenting, in both cases, results that confirm the efficacy of our approach.
... CGP was introduced by Miller [1] to evolve digital circuits, which is still an active research field as of today [2,3]. Furthermore, its concept is used in a diverse field of problem domains like classification or regression [4]. Another more prominent application for CGP is on the subject of neural architecture search [5,6]. ...
... Using nodes, it is possible to feed-forward an arbitrary amount of program inputs via partially connected nodes to calculate any desired amount of outputs. However, with today's standard, a CGP model consists of only one row for most applications [4]. That means, r = 1. ...
... As is standard for most CGP configurations, an elitist (µ + λ) evolutionary strategy (ES) with µ = 1 and λ = 4 is used [4,11]. Furthermore, CGP usually does not rely on crossover, as it generally does not improve CGP's performance [26]. ...
Cartesian Genetic Programming (CGP) suffers from a specific limitation: Positional bias, a phenomenon in which mostly genes at the start of the genome contribute to a program output, while genes at the end rarely do. This can lead to an overall worse performance of CGP. One solution to overcome positional bias is to introduce reordering methods, which shuffle the current genotype without changing its corresponding phenotype. There are currently two different reorder operators that extend the classic CGP formula and improve its fitness value. In this work, we discuss possible shortcomings of these two existing operators. Afterwards, we introduce three novel operators which reorder the genotype of a graph defined by CGP. We show empirically on four Boolean and four symbolic regression benchmarks that the number of iterations until a solution is found and/or the fitness value improves by using CGP with a reorder method. However, there is no consistently best performing reorder operator. Furthermore, their behaviour is analysed by investigating their convergence plots and we show that all behave the same in terms of convergence type.
... Cartesian Genetic Programming (CGP) [Miller 2011] is a bio-inspired technique, in which programs are directed acyclic graphs (DAGs). CGP was firstly developed for evolving digital circuits. ...
... Continuous lines define the phenotype. Considering a situation which all nodes carry out some computational function, representing functions in the form of graphs is more concise than traditional GP trees, because they allow the reuse of previously calculated subgraphs [Miller 2011] CGP commonly uses an evolutionary strategy (1+λ) as evolutionary mechanism, in which the best individual generates λ offspring. Usually λ = 4 [Miller 2011], presented in Algorithm 2. ...
... Considering a situation which all nodes carry out some computational function, representing functions in the form of graphs is more concise than traditional GP trees, because they allow the reuse of previously calculated subgraphs [Miller 2011] CGP commonly uses an evolutionary strategy (1+λ) as evolutionary mechanism, in which the best individual generates λ offspring. Usually λ = 4 [Miller 2011], presented in Algorithm 2. ...
Evolutionary techniques have been used in the design and optimization of combinational logic circuits. This procedure is called evolvable hardware and Cartesian Genetic Programming (CGP) is the evolutionary technique with the best performance in this context. Despite the good results obtained by CGP techniques, its search procedure usually evolves a single candidate solution by an evolution strategy and this approach tends to be trapped in local optima. On the other hand, clonal selection techniques in general, and CLONALG in particular, were designed to avoid converging to a low-quality local optimum. Thus, we propose here using the representation of CGP with the search procedure of a Clonal Selection Algorithm to minimize the number of transistors of combinational logic circuits. Furthermore, a parameter sensitivity analysis is performed. The results are assessed considering a benchmark from the literature and showed a reduction in the number of transistors when compared to the baseline ESPRESSO.
... Neural Architecture Search (NAS) uses different approaches like EAs, reinforcement learning, and Bayesian optimization to find near-optimal architectures for Deep Neural Networks (DNNs) (Miikkulainen et al., 2019;Zoph et al., 2018;Lu et al., 2019). Genetic Programming (GP) and Grammatical Evolution (GE) (Ryan et al., 1998) enable solution generation and optimization simultaneously (Koza, 1994;Ryan et al., 1998;Miller, 2011), which makes them suitable candidates for simultaneous generation and training of neural networks (Tsoulos et al., 2008;Ahmadizar et al., 2015;Assunção et al., 2017a,b;Miller, 2020). In particular, GE (Ryan et al., 1998) is a variant of GP that takes a user-defined Context Free Grammar (CFG) as its input to determine how genotypes are mapped to phenotypes. ...
... There are several studies that investigate the importance of modularity in the performance of GPs (Koza, 1994;Miller, 2011;Ellefsen et al., 2020;O'Neill and Ryan, 2000;Hemberg E., 2009;Harper and Blair, 2006;Swafford et al., 2011a,b). For example, Koza (1994) demonstrates that adding modularity to GP highly increases scalability. ...
... For example, Koza (1994) demonstrates that adding modularity to GP highly increases scalability. Cartesian Genetic Programming (CGP) (Miller, 2011) is one of the well-known modular GPs that applies to many applications like circuit design, GPU programming, ANN synthesis (Miller, 2020), and DNNs design (Ellefsen et al., 2020). CGP is modular in that it represents sub-functions of the target function as distinct sub-trees in predefined layers where the output sub-tree composes the final candidate solution. ...
This paper presents a novel method, called Modular Grammatical Evolution (MGE), towards validating the hypothesis that restricting the solution space of NeuroEvolution to modular and simple neural networks enables the efficient generation of smaller and more structured neural networks while providing acceptable (and in some cases superior) accuracy on large data sets. MGE also enhances the state-of-the-art Grammatical Evolution (GE) methods in two directions. First, MGE's representation is modular in that each individual has a set of genes, and each gene is mapped to a neuron by grammatical rules. Second, the proposed representation mitigates two important drawbacks of GE, namely the low scalability and weak locality of representation, towards generating modular and multi-layer networks with a high number of neurons. We define and evaluate five different forms of structures with and without modularity using MGE and find single-layer modules with no coupling more productive. Our experiments demonstrate that modularity helps in finding better neural networks faster. We have validated the proposed method using ten well-known classification benchmarks with different sizes, feature counts, and output class count. Our experimental results indicate that MGE provides superior accuracy with respect to existing NeuroEvolution methods and returns classifiers that are significantly simpler than other machine learning generated classifiers. Finally, we empirically demonstrate that MGE outperforms other GE methods in terms of locality and scalability properties.
... Similarly, a transfer learning-based GP technique can easily solve simple classification tasks but struggles to completely learn complex (e.g., 7-bit parity) problems [20]. Cartesian GP (CGP) is another GP-based technique that implicitly reuses graph nodes to address large-scale problems [21], [22]. Although CGP can produce many-toone genotype-to-phenotype mapping, it generates an arbitrary sequence of computer programs that can solve only a particular problem. ...
... The program has the ability to implicitly reuse nodes in the graph. Consequently, the mapping from genotype to phenotype in such a program is many-to-one [21], [47]. The CGP technique generates an arbitrary sequence of computer programs that can solve only a particular problem; the learned solution cannot be reused to solve other complex problems. ...
... Consequently, the system efficiently learns new problems and acquires the ability to resolve higher level complex problems. The experimental results generated by the lateralized system (LateralXCS) are compared with the results generated by the conventional LCS (XCS), CF-based LCS (XCSCFC) [25], XCS/BOA [56], conventional GP, LLGP [18], and CGP [21] techniques. ...
Modern classifier systems can effectively classify targets that consist of simple patterns. However, they can fail to detect hierarchical patterns of features that exist in many real-world problems, such as understanding speech or recognizing object ontologies. Biological nervous systems have the ability to abstract knowledge from simple and small-scale problems in order to then apply it to resolve more complex problems in similar and related domains. It is thought that lateral asymmetry of biological brains allows modular learning to occur at different levels of abstraction, which can then be transferred between tasks. This work develops a novel evolutionary machine-learning (EML) system that incorporates lateralization and modular learning at different levels of abstraction. The results of analyzable Boolean tasks show that the lateralized system has the ability to encapsulate underlying knowledge patterns in the form of building blocks of knowledge (BBK). Lateralized abstraction transforms complex problems into simple ones by reusing general patterns (e.g., any parity problem becomes a sequence of the 2-bit parity problem). By enabling abstraction in evolutionary computation, the lateralized system is able to identify complex patterns (e.g., in hierarchical multiplexer (HMux) problems) better than existing systems.
... Neural Architecture Search (NAS) uses different approaches like EAs, reinforcement learning, and Bayesian optimization to find near-optimal architectures for Deep Neural Networks (DNNs) (Miikkulainen et al., 2019;Zoph et al., 2018;Lu et al., 2019). Genetic Programming (GP) and Grammatical Evolution (GE) (Ryan et al., 1998) enable solution generation and optimization simultaneously (Koza, 1994;Ryan et al., 1998;Miller, 2011), which makes them suitable candidates for simultaneous generation and training of neural networks (Tsoulos et al., 2008;Ahmadizar et al., 2015;Assunção et al., 2017aAssunção et al., , 2017bMiller, 2020). In particular, GE (Ryan et al., 1998) is a variant of GP that takes a user-defined Context Free Grammar (CFG) as its input to determine how genotypes are mapped to phenotypes. ...
... There are several studies that investigate the importance of modularity in the performance of GPs (Koza, 1994;Miller, 2011;Ellefsen et al., 2020;O'Neill and Ryan, 2000;Hemberg, 2009;Harper and Blair, 2006;Swafford, Hemberg et al., 2011;. For example, Koza (1994) demonstrates that adding modularity to GP highly increases scalability. ...
... For example, Koza (1994) demonstrates that adding modularity to GP highly increases scalability. Cartesian Genetic Programming (CGP) (Miller, 2011) is one of the well-known modular GPs that applies to many applications like circuit design, GPU programming, ANN synthesis (Miller, 2020), and DNNs design (Ellefsen et al., 2020). CGP is modular in that it represents sub-functions of the target function as distinct sub-trees in predefined layers where the output sub-tree composes the final candidate solution. ...
This article presents a novel method, called Modular Grammatical Evolution (MGE), toward validating the hypothesis that restricting the solution space of NeuroEvolution to modular and simple neural networks enables the efficient generation of smaller and more structured neural networks while providing acceptable (and in some cases superior) accuracy on large data sets. MGE also enhances the state-of-the-art Grammatical Evolution (GE) methods in two directions. First, MGE's representation is modular in that each individual has a set of genes, and each gene is mapped to a neuron by grammatical rules. Second, the proposed representation mitigates two important drawbacks of GE, namely the low scalability and weak locality of representation, toward generating modular and multilayer networks with a high number of neurons. We define and evaluate five different forms of structures with and without modularity using MGE and find single-layer modules with no coupling more productive. Our experiments demonstrate that modularity helps in finding better neural networks faster. We have validated the proposed method using ten well-known classification benchmarks with different sizes, feature counts, and output class counts. Our experimental results indicate that MGE provides superior accuracy with respect to existing NeuroEvolution methods and returns classifiers that are significantly simpler than other machine learning generated classifiers. Finally, we empirically demonstrate that MGE outperforms other GE methods in terms of locality and scalability properties.
... The proposed approximation methodology, illustrated in Figure 3, consists of three main phases. In Phase 1, Cartesian genetic programming (CGP) [33] is used to evolve circuits that approximate the popcount function. In Phase 2, we identify Pareto-optimal combinations of these circuits to deliver the necessary approximate popcount-compare functionality for the hidden-layer neurons. ...
... An -bit popcount (PC) operation sums binary inputs, typically implemented using a tree structure of adders. To approximate this exact circuit, we employ the evolutionary CGP algorithm [33]. In CGP, a simple search method is deployed over an integer address-based genetic representation of the circuit to create random mutations of an initial population that contains the accurate PC circuit. ...
Printed electronics offer ultra-low manufacturing costs and the potential for on-demand fabrication of flexible hardware. However, significant intrinsic constraints stemming from their large feature sizes and low integration density pose design challenges that hinder their practicality. In this work, we conduct a holistic exploration of printed neural network accelerators, starting from the analog-to-digital interface - a major area and power sink for sensor processing applications - and extending to networks of ternary neurons and their implementation. We propose bespoke ternary neural networks using approximate popcount and popcount-compare units, developed through a multi-phase evolutionary optimization approach and interfaced with sensors via customizable analog-to-binary converters. Our evaluation results show that the presented designs outperform the state of the art, achieving at least 6x improvement in area and 19x in power. To our knowledge, they represent the first open-source digital printed neural network classifiers capable of operating with existing printed energy harvesters.
... CGP encodes computer programs as graph representation [18] and is an enhancement of a method originally developed for evolving digital circuits [19,20]. CGP already demonstrated its capabilities in synthesizing complex functions in several different use cases for example for image processing [3], or neural network training [21]. ...
... Not all outputs of computational nodes are used as input for other functions. In fact, usually many of such unused computational nodes occur in evolved CGP [19]. These nodes are inactive, do not contribute to the encoded program's output, and are not executed during interpretation of the program. ...
Cartesian genetic programming is a popular version of genetic programming and has meanwhile proven its performance in many use cases. This paper introduces an algorithmic level decomposition of program evolution that can be solved by a multi-agent system in a fully distributed manner. A heuristic for distributed combinatorial problem solving is adapted to evolve programs. The applicability of the approach and the effectiveness of the multi-agent approach as well as of the evolved genetic programs are demonstrated using symbolic regression, n-parity, and classification problems.
... One research direction involves improving the scalability of the evaluation. As a straightforward solution, various FPGA-based, GPU-based and CPU-based accelerators have been employed [10,12,24,32]. Besides of that, modification of the fitness evaluation procedure has been proposed. ...
... The mutation of a node input connection consists of several steps. First, some inactive nodes are modified (lines [8][9][10][11]. After that, one input of a mutated node c is chosen randomly (line 12). ...
Cartesian genetic programming (CGP) represents the most efficient method for the evolution of digital circuits. Despite many successful applications, however, CGP suffers from limited scalability, especially when used for evolutionary circuit design, i.e. design of circuits from a randomly initialized population. Considering the multiplier design problem, for example, the 55-bit multiplier represents the most complex circuit designed by the evolution from scratch. The efficiency of CGP highly depends on the performance of the point mutation operator, however, this operator is purely stochastic. This contrasts with the recent developments in genetic programming (GP), where advanced informed approaches such as semantic-aware operators are incorporated to improve the search space exploration capability of GP. In this paper, we propose a semantically-oriented mutation operator () suitable for the evolutionary design of combinational circuits. In contrast to standard point mutation modifying the values of the mutated genes randomly, the proposed operator uses semantics to determine the best value for each mutated gene. Compared to the common CGP and its variants, the proposed method converges on common Boolean benchmarks substantially faster while keeping the phenotype size relatively small. The successfully evolved instances presented in this paper include 10-bit parity, 10 + 10-bit adder and 55-bit multiplier. The most complex circuits were evolved in less than one hour with a single-thread implementation running on a common CPU.
... Schmidt and Lipson [26] simultaneously coevolved solutions and predictors for a symbolic-regression problem, a popular task in the field of Genetic Programming (GP), wherein solutions are represented in various forms, including trees [17], two-dimensional grids of computational nodes [22], grammars [23], and more. Their results showed significant runtime improvement and reduced the size of the trees, leading to faster fitness computations. ...
We present a novel approach to performing fitness approximation in genetic algorithms (GAs) using machine-learning (ML) models, focusing on evolutionary agents in Gymnasium (game) simulators -- where fitness computation is costly. Maintaining a dataset of sampled individuals along with their actual fitness scores, we continually update throughout an evolutionary run a fitness-approximation ML model. We compare different methods for: 1) switching between actual and approximate fitness, 2) sampling the population, and 3) weighting the samples. Experimental findings demonstrate significant improvement in evolutionary runtimes, with fitness scores that are either identical or slightly lower than that of the fully run GA -- depending on the ratio of approximate-to-actual-fitness computation. Our approach is generic and can be easily applied to many different domains.
... It is well-known that stochastic gradient descent (SGD) is the most effective method to train ANNs [7]. To account for this key element, we adopt dCGPANN [9,12], a variation of Cartesian Genetic Programming (CGP) [13,14]. dCGPANN allows us to differentiate weights, and so, we can use SGD instead of evolving the ANNs' weights. ...
Surrogate-assisted evolutionary algorithms (SAEAs) aim to use efficient computational models with the goal of approximating the fitness function in evolutionary computation systems. This area of research has been active for over two decades and has received significant attention from the specialised research community in different areas, for example, single and many objective optimisation or dynamic and stationary optimisation problems. An emergent and exciting area that has received little attention from the SAEAs community is in neuroevolution. This refers to the use of evolutionary algorithms in the automatic configuration of artificial neural network (ANN) architectures, hyper-parameters and/or the training of ANNs. However, ANNs suffer from two major issues: (a) the use of highly-intense computational power for their correct training, and (b) the highly specialised human expertise required to correctly configure ANNs necessary to get a well-performing network. This work aims to fill this important research gap in SAEAs in neuroevolution by addressing these two issues. We demonstrate how one can use a Kriging Partial Least Squares method that allows efficient computation of good approximate surrogate models compared to the well-known Kriging method, which normally cannot be used in neuroevolution due to the high dimensionality of the data.
... The modified algorithm can be represented with the following pseudocode: create λ child solutions using perturbation on the parent 5: select the best solution 6: set the best solution as the parent 7: end for 8: until TerminationCriterion Using algorithm parameters µ and λ, we can balance between the diversification and intensification segments of the search. In our experiments, we have used the value of 4 for the parameter λ, which is a common choice in diverse ES applications, see, e.g., [33], [35], [42]. With this parameter value, the parent for the next iteration is selected among five solutions in total (the parent and four child solutions). ...
... To address the bloat issue, Cartesian GP (CGP) uses an integer-array genome with a fixed length to encode an executable tree graph [129], as depicted in Figure 3b. All nodes are placed in a grid manner and are indexed sequentially. ...
Reinforcement learning (RL) is a machine learning approach that trains agents to maximize cumulative rewards through interactions with environments. The integration of RL with deep learning has recently resulted in impressive achievements in a wide range of challenging tasks, including board games, arcade games, and robot control. Despite these successes, there remain several crucial challenges, including brittle convergence properties caused by sensitive hyperparameters, difficulties in temporal credit assignment with long time horizons and sparse rewards, a lack of diverse exploration, especially in continuous search space scenarios, difficulties in credit assignment in multi-agent reinforcement learning, and conflicting objectives for rewards. Evolutionary computation (EC), which maintains a population of learning agents, has demonstrated promising performance in addressing these limitations. This article presents a comprehensive survey of state-of-the-art methods for integrating EC into RL, referred to as evolutionary reinforcement learning (EvoRL). We categorize EvoRL methods according to key research fields in RL, including hyperparameter optimization, policy search, exploration, reward shaping, meta-RL, and multi-objective RL. We then discuss future research directions in terms of efficient methods, benchmarks, and scalable platforms. This survey serves as a resource for researchers and practitioners interested in the field of EvoRL, highlighting the important challenges and opportunities for future research. With the help of this survey, researchers and practitioners can develop more efficient methods and tailored benchmarks for EvoRL, further advancing this promising cross-disciplinary research field.
... Cartesian Genetic Programming (CGP) (Miller, 2011) is a form of genetic programming that encodes programs as directed acyclic graphs. An individual CGP graph is composed of three components: input nodes, output nodes and computation nodes. ...
Given the current context of climate change and the increasing population densities at coastal zones around the globe, there is an increasing need to be able to predict the development of our coasts. Recent advances in artificial intelligence allow for automatic analysis of observational data. Symbolic Regression (SR) is a type of Machine Learning algorithm that aims to find interpretable symbolic expressions that can explain the relations in the data. In this work, we aim to study the problem of forecasting shoreline change using SR. We make use of Cartesian Genetic Programming (CGP) in order to encode and improve upon ShoreFor, a shoreline prediction model. Coupled with NSGA-II, the CGP individuals are evaluated and selected during evolution according to their predictive skills at five different coastal sites. This work presents a comparison between the CGP-evolved models and the base ShoreFor model. In addition to its ability to produce well-performing models, the work demonstrates the usefulness of CGP as a research tool to gain insight into the behaviors of shorelines at different points around the globe.
... The modified algorithm can be represented with the following pseudocode: create λ child solutions using perturbation on the parent 5: select the best solution 6: set the best solution as the parent 7: end for 8: until TerminationCriterion Using algorithm parameters µ and λ, we can balance between the diversification and intensification segments of the search. In our experiments, we have used the value of 4 for the parameter λ, which is a common choice in diverse ES applications, see, e.g., [33], [35], [42]. With this parameter value, the parent for the next iteration is selected among five solutions in total (the parent and four child solutions). ...
Fuzzing is an automated software testing technique broadly adopted by the industry. A popular variant is mutation-based fuzzing, which discovers a large number of bugs in practice. While the research community has studied mutation-based fuzzing for years now, the algorithms' interactions within the fuzzer are highly complex and can, together with the randomness in every instance of a fuzzer, lead to unpredictable effects. Most efforts to improve this fragile interaction focused on optimizing seed scheduling. However, real-world results like Google's FuzzBench highlight that these approaches do not consistently show improvements in practice. Another approach to improve the fuzzing process algorithmically is optimizing mutation scheduling. Unfortunately, existing mutation scheduling approaches also failed to convince because of missing real-world improvements or too many user-controlled parameters whose configuration requires expert knowledge about the target program. This leaves the challenging problem of cleverly processing test cases and achieving a measurable improvement unsolved. We present DARWIN, a novel mutation scheduler and the first to show fuzzing improvements in a realistic scenario without the need to introduce additional user-configurable parameters, opening this approach to the broad fuzzing community. DARWIN uses an Evolution Strategy to systematically optimize and adapt the probability distribution of the mutation operators during fuzzing. We implemented a prototype based on the popular general-purpose fuzzer AFL. DARWIN significantly outperforms the state-of-the-art mutation scheduler and the AFL baseline in our own coverage experiment, in FuzzBench, and by finding 15 out of 21 bugs the fastest in the MAGMA benchmark. Finally, DARWIN found 20 unique bugs (including one novel bug), 66% more than AFL, in widely-used real-world applications.
... We ran two series of experiments by using a (1 + 4) and a (20 + 20) evolutionary strategy. The former is commonly used to evolve digital circuits and graph structures Miller and Thomson 2000;Miller, 2011). The latter is commonly used to evolve robots (Pagliuca et al., 2018). ...
The propensity of evolutionary algorithms to generate compact solutions have advantages and disadvantages. On one side, compact solutions can be cheaper, lighter, and faster than less compact ones. On the other hand, compact solutions might lack evolvability, i.e. might have a lower probability to improve as a result of genetic variations. In this work we study the relation between phenotypic complexity and evolvability in the case of soft-robots with varying morphology. We demonstrate a correlation between phenotypic complexity and evolvability. We demonstrate that the tendency to select compact solutions originates from the fact that the fittest robots often correspond to phenotypically simple robots which are robust to genetic variations but lack evolvability. Finally, we demonstrate that the efficacy of the evolutionary process can be improved by increasing the probability of genetic variations which produce a complexification of the agents’ phenotype or by using absolute mutation rates.
... Uma das vantagens dessa representaçãoé que a CGP pode lidar com uma ampla gama de estruturas computacionais. Além disso, estaé uma representação mais compacta quando comparadaàś arvores da Programação Genética tradicional [Miller 2011]. A estratégia evolutiva comumente utilizada pela CGPé apresentada no Algoritmo 1. Seleciona o melhor dos (1 + λ) indivíduos para ser progenitor 7: end while A CGP pode ser usada em diversos problemas e um delesé a evolução de Redes Neurais Artificiais (RNAs), como em [Khan et al. 2010]. ...
A inferência de Redes de Regulação Gênica (GRNs) é importante em Biologia Sistêmica, pois permite o entendimento de padrões de interações entre genes. Essas descobertas sãoúteis para fornecer compreensão sobre doenças e ajudar no desenvolvimento de fármacos. Técnicas de computação evolutiva, como a Programação Genética Cartesiana (CGP), têm sido utilizadas para inferir GRNs com resultados promissores. Entretanto, a CGP tem problemas de escalabilidade. Aqui, GRNs são inferidas de forma eficiente usando abordagens de computação de alto desempenho. Experimentos computacionais mostram que o método desenvolvido nesta iniciação científica é capaz de inferir GRNs mais rapidamente do que outros da literatura com soluções simbólicas. O ganho em tempo de processamento da técnica paralela apresentada em relação ao formato sequencial é de até 104%.
... The use of the genetic programming (GP) algorithms in the behavioral synthesis makes it possible to synthesize the processors of any complexity [2,4]. This approach is often used for the program development but sometimes for the hardware synthesis [5]. ...
A method of the pipelined datapath synthesis which is based on the genetic programming is used for the discrete cosine transform (DCT) processor synthesis. The method is based on the representation of the algorithm by the spaced synchronous data flow graph (SDF), transferring it to a chromosome, and using the genetic optimization process. The method is implemented in the SDFCAD program system which is able to perform the input of algorithms specified by the SDF and automatically perform the synthesis of pipelined processors. The DCT processors synthesized by this system have less hardware volume and higher throughput than the similar processors configured in FPGA.
... Programs in CGP are represented using computational nodes that form either one or two-dimensional structures and work as DAG. In CGP, a genotype is represented in the form of integers given by (Miller, 2011), There are three types of gene nodes in CGP -function gene, connection genes, and output genes. Function gene 'Fi' provides the address of the function that the node represents in the function table, while connection genes 'Cij' decides from where the input to the current node will be taken, and finally output genes 'Oi' define from which nodes outputs will be taken. ...
Stock market is one of the attractive domains for researchers as well as academicians. It represents highly complex non-linear fluctuating market behaviours where traders, investors, and organizers look forward to reliable future predictions of the market indices. Such prediction problems can be computationally addressed using various machine learning, deep learning, sentiment analysis, as well as mining approaches. However, the internal parameters configuration can play an important role in the prediction performance; also, feature selection is a crucial task. Therefore, to optimize such approaches, the evolutionary computation-based algorithms can be integrated in several ways. In this article, we systematically conduct a focused survey on genetic algorithm (GA) and its applications for stock market prediction; GAs are known for their parallel search mechanism to solve complex real-world problems; various genetic perspectives are also integrated with machine learning and deep learning methods to address financial forecasting. Thus, we aim to analyse the potential extensibility and adaptability of GAs for stock market prediction. We review stock price and stock trend prediction, as well as portfolio optimization, approaches over the recent years (2013–2022) to signify the state-of-the-art of GA-based optimization in financial markets. We broaden our discussion by briefly reviewing other genetic perspectives and their applications for stock market forecasting. We balance our survey with the consideration of competitiveness and complementation of GAs, followed by highlighting the challenges and potential future research directions of applying GAs for stock market prediction.
For real-time applications in embedded systems, an efficient image filter is not defined solely by its accuracy but by the delicate balance it strikes between precision and computational cost. While one approach to manage an algorithm’s computing demands involves evaluating its complexity, an alternative strategy employs a multi-objective algorithm to optimize both precision and computational cost.
In this paper, we introduce a multi-objective adaptation of Cartesian Genetic Programming aimed at enhancing image filter performance. We refine the existing Cartesian Genetic Programming framework for image processing by integrating the elite Non-dominated Sorting Genetic Algorithm into the evolutionary process, thus enabling the generation of a set of Pareto front solutions that cater to multiple objectives.
To assess the effectiveness of our framework, we conduct a study using a Urban Traffic dataset and compare our results with those obtained using the standard framework employing a mono-objective evolutionary strategy. Our findings reveal two key advantages of this adaptation. Firstly, it generates individuals with nearly identical precision in one objective while achieving a substantial enhancement in the other objective. Secondly, the use of the Pareto front during the evolution process expands the research space, yielding individuals with improved fitness.
Unconventional functions, including activation functions and power functions, are extremely hard-to-realize primarily due to the difficulty in arriving at the hierarchical design. The hierarchical design allows the synthesis tool to map the functionality with that of standard cells employed through the regular ASIC synthesis flow. For conventional functions, the hierarchical design is structured and then supplied to the synthesis flow, whereas, for unconventional functions, the same method is not reliable, since the current synthesis method does not offer any design-space exploration scheme to arrive at an easy-to-realize design entity. The unconventional functions either take a long synthesis run-time or additional efforts are spent in restructuring the hierarchical design for the desired function to synthesizable ones. Cartesian genetic programing (CGP) allows to not only incorporate custom logic gates for synthesizing the hierarchical design but also aids in the design-space exploration for the targeted function through the custom gates. The CGP configuration evolves difficult-to-realize complex functions with multiple solutions, and filtering through desired Pareto-optimal requirements offers a unique hierarchical design. Incorporating CGP-derived hierarchical designs into the traditional synthesis flow is instrumental for implementing and evaluating higher-order designs comprising nonlinear functional constructs. Six activation functions and power functions that fall in the category of unconventional functions are realized by the CGP method using custom cells to demonstrate the capability. Further, the hierarchical design of these unconventional functions is flattened and compared with the same function that is directly synthesized using basic gates. The CGP-derived synthesis method reports 3× less synthesis time for realizing the complex functions at the hierarchical level compared to the synthesis using basic gate cells. Hardware characteristics and error metrics are also investigated for the CGP realized complex functions and are made freely available for further usage to the research and designers’ community.
The inference of gene regulatory networks (GRNs) is a widely addressed problem in Systems Biology. GRNs can be modeled as Boolean networks, which is the simplest approach for this task. However, Boolean models need binarized data. Several approaches have been developed for the discretization of gene expression data (GED). Also, the advance of data extraction technologies, such as single-cell RNA-Sequencing (scRNA-Seq), provides a new vision of gene expression and brings new challenges for dealing with its specificities, such as a large occurrence of zero data. This work proposes a new discretization approach for dealing with scRNA-Seq time-series data, named Distribution and Successive Spline Points Discretization (DSSPD), which considers the data distribution and a proper preprocessing step. Here, Cartesian Genetic Programming (CGP) is used to infer GRNs using the results of DSSPD. The proposal is compared with CGP with the standard data handling and five state-of-the-art algorithms on curated models and experimental data. The results show that the proposal improves the results of CGP in all tested cases and outperforms the state-of-the-art algorithms in most cases.
Cartesian genetic programming is a popular version of classical genetic programming, and it has now demonstrated a very good performance in solving various use cases. Originally, programs evolved by using a centralized optimization approach. Recently, an algorithmic level decomposition of program evolution has been introduced that can be solved by a multi-agent system in a fully distributed manner. A heuristic for distributed combinatorial problem-solving was adapted to evolve these programs. The applicability of the approach and the effectiveness of the used multi-agent protocol as well as of the evolved genetic programs for the case of full enumeration in local agent decisions has already been successfully demonstrated. Symbolic regression, n-parity, and classification problems were used for this purpose. As is typical of decentralized systems, agents have to solve local sub-problems for decision-making and for determining the best local contribution to solving program evolution. So far, only a full enumeration of the solution candidates has been used, which is not sufficient for larger problem sizes. We extend this approach by using CMA-ES as an algorithm for local decisions. The superior performance of CMA-ES is demonstrated using Koza’s computational effort statistic when compared with the original approach. In addition, the distributed modality of the local optimization is scrutinized by a fitness landscape analysis.
Automatic programming is a type of programming that has the ability to analyze and solve problems using the principles of symbolic regression analysis. These methods can solve complex problems regardless of whether they have a specific pattern or not. In this work, we are going to introduce the difference-based firefly programming (DFP) method as an improved version of the standard firefly programming method. We have analyzed the performance of this new improved method, which will be described in detail within the scope of this work. In order to evaluate the performance of the newly presented method, the results have been compared to the results of the standard method and the results of other methods that are used to solve the same type of problems. DFP has been used also in forecasting and modeling a real-world time-series problem, where it showed good performance too. In general, the results demonstrated the improved performance of the newly introduced method and showed its ability to efficiently solve complex problems.
Logic Synthesis From Partial Specifications (LSFPS) is the problem of finding the hardware implementation of a Boolean function from a partial knowledge of its care set. The elements missing from the specifications are named don’t knows. The exact solution of LSFPS is the minimum size circuit of the corresponding problem in which the don’t knows set is void. Hence, in addition to the traditional objective of size minimization, the goal is to maximize the test accuracy, i.e., the accuracy of the circuit when evaluated over a subset of the don’t knows. This problem is relevant because efficient solutions can lead to hardware friendly machine learning models, not relying on black-box approaches. Indeed, LSFPS maps directly to the problem of the automatic generation of optimized topologies for Binarized Neural Networks. Furthermore, combining the exact solution with modern logic synthesis techniques would unlock unprecedented optimization capabilities. Previous works proved the effectiveness of approximate logic synthesis (ALS) for designing circuits with high test accuracy. Nonetheless, these methods sacrifice accuracy on the specifications, which banishes them from the legitimate candidates for LSFPS. In this paper, we propose accuracy recovery, a procedure to map an approximate version of the circuit to a new one that satisfies the exact functionality of the specifications. The proposed approach relies on an extension of a disjoint support decomposition algorithm. Relative experiments on the IWLS2020 benchmarks show that, on average, the addition of the designed decomposition to a synthesis flow reduces by 17.38% the number of gates and by 12.02% the depth. The usage of accuracy recovery, based on such a decomposition, yields a 95.73% accuracy in the binary MNIST problem, beating the state-of-the-art in ALS of 92.76%.
This article presents a comprehensive review of stacking methods commonly used to address the challenge of automatic semantic similarity measurement in the literature. Since more than two decades of research have left various semantic similarity measures, scientists and practitioners often find many difficulties in choosing the best method to put into production. For this reason, a novel generation of strategies has been proposed to use basic semantic similarity measures using base estimators to achieve a better performance than could be gained from any of the semantic similarity measures. In this work, we analyze different stacking techniques, ranging from the classical algebraic methods to the most powerful ones based on hybridization, including blending, neural, fuzzy, and genetic-based stacking. Each technique excels in aspects such as simplicity, robustness, accuracy, interpretability, transferability, or a favorable combination of several of those aspects. The goal is that the reader can have an overview of the state-of-the-art in this field.
Grammar-Guided Genetic Programming (G3P) is widely recognised as one of the most successful approaches for program synthesis, i.e., the task of automatically discovering an executable piece of code given user intent. G3P has been shown capable of successfully evolving programs in arbitrary languages that solve several program synthesis problems based only on a set of input/output examples. Despite its success, the restriction on the evolutionary system to only leverage input/output error rate during its assessment of the programs it derives limits its scalabil-ity to larger and more complex program synthesis problems. With the growing number and size of open software repositories and generative artificial intelligence approaches, there is a sizeable and growing number of approaches for retrieving/generating source code (potentially several partial snippets) based on textual problem descriptions. Therefore, it is now, more than ever, time to introduce G3P to other means of user intent (particularly textual problem descriptions). In this paper, we would like to assess the potential for G3P to evolve programs based on their similarity to particular target codes of interest (obtained using some code retrieval/generative approach). Through our experimental evaluation on a well-known program synthesis benchmark, we have shown that G3P successfully manages to evolve some of the desired programs with all four considered similarity measures. However, in its default configuration, G3P is not as successful with similarity measures as it is with the classical input/output error rate when solving program synthesis problems. Therefore, we propose a novel multi-objective G3P approach that combines the similarity to the target program and the traditional input/output error rate. Our experiments show that compared to the error-based G3P, the multi-objective G3P approach could improve the success rate of specific problems and has great potential to improve on the traditional G3P system.
Based on a solid mathematical background, this paper proposes a method for Symbolic Regression that enables the extraction of mathematical expressions from a dataset. Contrary to other approaches, such as Genetic Programming, the proposed method is deterministic and, consequently, does not require the creation of a population of initial solutions. Instead, a simple expression is grown until it fits the data. This method has been compared with four well-known Symbolic Regression techniques with a large number of datasets. As a result, on average, the proposed method returns better performance than the other techniques, with the advantage of returning mathematical expressions that can be easily used by different systems. Additionally, this method makes it possible to establish a threshold at the complexity of the expressions generated, i.e., the system can return mathematical expressions that are easily analyzed by the user, as opposed to other techniques that return very large expressions.
The previous studies on the bending behavior of Fiber Reinforced Polymers (FRP) Reinforced Concrete (RC) beams proved that there is a distinct difference between the cracking and deflection behavior of these structural members and conventional RC beams with steel reinforcing rebar. Accordingly, the existing relations for the prediction of deflection and crack width of RC members cannot be used for serviceability controls of FRP-RC beams. The present study proposes new and more accurate predictive relations for the effective moment of inertia and the crack width of GFRP-RC beams using a collection of experimental test results from literature an Evolutionary Algorithm (EA) called Multi-Expression Programming (MEP). An EA is a subset of generic population-based metaheuristic optimization, which performs mechanisms inspired by biological operations to find the best solutions for different types of approximating problems. Numerical evaluations proved that the proposed relation for the effective moment of inertia has higher accuracy than the other existing relations with an R² value of 0.49 and RMSE, MAE, and IAE error indices of 0.31, 0.72, and 0.51. The proposed relation for the crack width also outperforms the other existing counterparts. Since the proposed relations are developed based on the results of the four-point bending test on simple beams, the accuracy of the proposed relation for the effective moment of inertia was also compared with the experimental and numerical results for continuous beams, which again verified the high accuracy of the proposed model. In addition to the experimental database, a numerical model is also developed to verify the accuracy of the suggested models. The attained results demonstrate considerably higher accuracy of the proposed relations in comparison with the other existing counterparts.
Continuous adaptation allows survival in an ever-changing world. Adjustments in the synaptic coupling strength between neurons are essential for this capability, setting us apart from simpler, hard-wired organisms. How these changes can be mathematically described at the phenomenological level, as so-called ‘plasticity rules’, is essential both for understanding biological information processing and for developing cognitively performant artificial systems. We suggest an automated approach for discovering biophysically plausible plasticity rules based on the definition of task families, associated performance measures and biophysical constraints. By evolving compact symbolic expressions, we ensure the discovered plasticity rules are amenable to intuitive understanding, fundamental for successful communication and human-guided generalization. We successfully apply our approach to typical learning scenarios and discover previously unknown mechanisms for learning efficiently from rewards, recover efficient gradient-descent methods for learning from target signals, and uncover various functionally equivalent STDP-like rules with tuned homeostatic mechanisms.
This chapter provides a detailed description of different symbolic regression methods. Some methods differ directly in the form of coding, as well as variational methods are based on the principle of small variations of the basic solution. By analogy with deep learning, the technology of the multilayer symbolic regression method is presented. We deliberately did not include detailed historical references in the description of the methods, focusing only on practically significant entities. The description of each method includes the encoding procedures with examples and the main features of the searching algorithm for finding the optimal solution in the code space with an emphasis on the implementation of the crossover operation of the genetic algorithm, which differs depending on the type of encoding. We do not pretend to provide a comprehensive overview of symbolic regression methods, but present only those symbolic regression methods that have already been applied in machine learning control problems, or we managed to apply them to the class of machine learning problems under consideration. As new symbolic regression methods appear for solving machine learning control problems, we will be happy to supplement the presented description.
Approximate circuits that trade the chip area for the quality of results play a key role in the development of energy-aware systems. Designing complex approximate circuits is, however, a very difficult and computationally demanding process. Evolutionary approximation—in particular, the method of Cartesian Genetic Programming (CGP)—currently represents one of the most successful approaches for automated circuit approximation. In this paper, we thoroughly investigate mutation operators for CGP with respect to the performance of circuit approximation. We design a novel dedicated operator that combines the classical single active gene mutation with a node deactivation operation (eliminating a part of the circuit forming a tree from an active gate). We show that our new operator significantly outperforms other operators on a wide class of approximation problems (such as 16 bit multipliers and dividers) and thus improves the performance of the state-of-the-art approximation techniques. Our results are grounded on a rigorous statistical evaluation including 39 approximation scenarios and 14,000 runs.
The paper introduces an approach to solve the optimal control problem applying symbolic regression methods. All known approaches to solve the optimal control problem are based on direct or indirect methods. Indirect methods may provide analytical solution but impose restrictions on the form and dimension of control object. In direct ones the optimal control problem is reduced to the nonlinear programming problem which provides general numerical solution. Also direct methods impose no restrictions on control object and have a wide range of application to complex applied problems. The approach proposed combines the advantages of both direct and indirect methods not involving the disadvantages mentioned previously. The suggested approach provides solution to the optimal control problem as a control function of time in explicit form. This approach benefits in case of applied optimal control problem requiring a solution in explicit form while indirect methods are inapplicable due to the restrictions imposed on the control system. The computational experiment of solving the optimal control problem for a mobile robot with phase constraints is presented. The results illustrate that the proposed approach finds an optimal control as a function of time which provides a mobile robot’s motion along the optimal trajectory avoiding phase constraints.
Using of a cluster of Graphics Processing Unit (GPU) equipped computers, it is possible to accelerate the evaluation of individuals in Genetic Programming. Pro-gram compilation, fitness case data and fitness execution are spread over the cluster of computers, allowing for the efficient processing of very large datasets. Here, the im-plementation is demonstrated on datasets containing over 10 million rows and several hundred megabytes in size. Populations of candidate individuals are compiled into NVidia CUDA programs and executed on a set of client computers -each with a different subset of the dataset. The paper discusses the implementation of the system and acts as a tutorial for other researchers experimenting with genetic programming and GPUs.
The evolution of image filters using Genetic Pro-gramming is a relatively unexplored task. This is most likely due to the high computational cost of evaluating the evolved programs. We use the parallel processors available on modern graphics cards to greatly increase the speed of evaluation. Previous papers in this area dealt with noise reduction and edge detection. Here we demonstrate that other more complicated processes can also be successfully evolved, and that we can "reverse engineer" the output from filters used in common graphics manipulation programs.
Self-Modifying Cartesian Genetic Programming (SMCGP) is a general purpose, graph-based, developmental form of Cartesian Genetic
Programming. In addition to the usual computational functions found in CGP, SMCGP includes functions that can modify the evolved
program at run time. This means that programs can be iterated to produce an infinite sequence of phenotypes from a single
evolved genotype. Here, we discuss the results of using SMCGP on a variety of different problems, and see that SMCGP is able
to solve tasks that require scalability and plasticity. We demonstrate how SMCGP is able to produce results that would be
impossible for conventional, static Genetic Programming techniques.
KeywordsCartesian genetic programming-developmental systems
Graphics processor units are fast, inexpensive parallel computing devices. Recently there has been great interest in harnessing this power for various types of scientific computation, including genetic programming. In previous work, we have shown that using the graphics processor provides dramatic speed improvements over a standard CPU in the context of fitness evaluation. In this work, we use Cartesian Genetic Programming to generate shader programs that implement image filter operations. Using the GPU, we can rapidly apply these programs to each pixel in an image and evaluate the performance of a given filter. We show that we can successfully evolve noise removal filters that produce better image quality than a standard median filter.
The majority of genetic programming implementations build expressions that only use a single data type. This is in contrast to human engineered programs that typically make use of multiple data types, as this provides the ability to express solutions in a more natural fashion. In this paper, we present a version of Cartesian Genetic Programming that handles multiple data types. We demonstrate that this allows evolution to quickly find competitive, compact, and human readable solutions on multiple classification tasks.
In this paper, we describe our work to investigate how much cyclic graph based Genetic Programming (GP) can be accelerated on one machine using currently available mid-range Graphics Processing Units (GPUs).
Cyclic graphs pose different problems for evaluation than do trees and we describe how our CUDA based, "population parallel" evaluator tackles these problems.
Previous similar work has focused on the evaluation alone. Unfortunately large reductions in the evaluation time do not necessarily translate to similar reductions in the total run time because the time spent on other tasks becomes more significant. We show that this problem can be tackled by having the GPU execute in parallel with the Central Processing Unit (CPU) and with memory transfers. We also demonstrate that it is possible to use a second graphics card to further improve the acceleration of one machine.
These additional techniques are able to reduce the total run time of the GPU system by up to 2.83 times. The combined architecture completes a full cyclic GP run 434.61 times faster than the single-core CPU equivalent. This involves evaluating at an average rate of 3.85 billion GP operations per second over the course of the whole run.
Abstract In this paper we demonstrate,the use of the Graphics Processing Unit (GPU) to accelerate Evolutionary Compu- tation applications, in particular Genetic Programming ap- proaches. We show that it is possible to get speed increases of several hundred,times over a typical CPU implemen- tation, catapulting GPU processing for these applications into the realm of HPC. This increase in performance,also extends to artificial developmental systems, where evolved programs,are used to construct cellular systems. Feasibil- ity of this approach to efficiently evaluate artificial develop- mental systems based on cellular automata is demonstrated.
We present a case study showing a human-competitive design of an evolved antenna that was deployed on a NASA spacecraft in 2006. We were fortunate to develop our antennas in parallel with another group using traditional design methodologies. This allowed us to demonstrate that our techniques were human-competitive because our automatically designed antenna could be directly compared to a human-designed antenna. The antennas described below were evolved to meet a challenging set of mission requirements, most notably the combination of wide beamwidth for a circularly polarized wave and wide bandwidth. Two evolutionary algorithms were used in the development process: one used a genetic algorithm style representation that did not allow branching in the antenna arms; the second used a genetic programming style tree-structured representation that allowed branching in the antenna arms. The highest performance antennas from both algorithms were fabricated and tested, and both yielded very similar performance. Both antennas were comparable in performance to a hand-designed antenna produced by the antenna contractor for the mission, and so we consider them examples of human-competitive performance by evolutionary algorithms. Our design was approved for flight, and three copies of it were successfully flown on NASA's Space Technology 5 mission between March 22 and June 30, 2006. These evolved antennas represent the first evolved hardware in space and the first evolved antennas to be deployed.
Self-modifying Cartesian Genetic Programming (SMCGP) is a general purpose, graph-based, developmental form of Genetic Programming
founded on Cartesian Genetic Programming. In addition to the usual computational functions, it includes functions that can
modify the program encoded in the genotype. This means that programs can be iterated to produce an infinite sequence of programs
(phenotypes) from a single evolved genotype. It also allows programs to acquire more inputs and produce more outputs during
this iteration. We discuss how SMCGP can be used and the results obtained in several different problem domains, including
digital circuits, generation of patterns and sequences, and mathematical problems. We find that SMCGP can efficiently solve
all the problems studied. In addition, we prove mathematically that evolved programs can provide general solutions to a number
of problems: n-input even-parity, n-input adder, and sequence approximation to π.
We present a general method for deploying parallel linear genetic programming (LGP) to the PC and Xbox 360 video game console
by using a publicly available common framework for the devices called XNA (for “XNA’s Not Acronymed”). By constructing the
LGP within this framework, we effectively produce an LGP “game” for PC and XBox 360 that displays results as they evolve.
We use the GPU of each device to parallelize fitness evaluation and the mutation operator of the LGP algorithm, thus providing
a general LGP implementation suitable for parallel computation on heterogeneous devices. While parallel GP implementations
on PCs are now common, both the implementation of GP on a video game console using GPU and the construction of a GP around
a framework for heterogeneous devices are novel contributions. The objective of this work is to describe how to implement
the parallel execution of LGP in order to use the underlying hardware (especially GPU) on the different platforms while still
maintaining loyalty to the general methodology of the LGP algorithm built for the common framework. We discuss the implementation
of texture-based data structures and the sequential and parallel algorithms built for their use on both CPU and GPU. Following
the description of the general algorithm, the particular tailoring of the implementations for each hardware platform is described.
Sequential (CPU) and parallel (GPU-based) algorithm performance is compared on both PC and video game platforms using the
metrics of GP operations per second, actual time elapsed, speedup of parallel over sequential implementation, and percentage
of execution time used by the GPU versus CPU.
The availability of low cost powerful parallel graphics cards has stimulated the port of Genetic Programming (GP) on Graphics
Processing Units (GPUs). Our work focuses on the possibilities offered by Nvidia G80 GPUs when programmed in the CUDA language.
In a first work we have showed that this setup allows to develop fine grain parallelization schemes to evaluate several GP
programs in parallel, while obtaining speedups for usual training sets and program sizes. Here we present another parallelization
scheme and optimizations about program representation and use of GPU fast memory. This increases the computation speed about
three times faster, up to 4 billion GP operations per second. The code has been developed within the well known ECJ library
and is open source.
Evolving a Self-Repairing, Self-Regulating, French Flag Organism
This paper presents an empirical study of the variation of program size over time, for a form of Genetic Programming called Cartesian Genetic Programming. Two main types of Cartesian genetic programming are examined: one uses a fully connected graph, with no redundant nodes, while the other allows partial connectedness and has redundant nodes. Studies are reported here for fitness based search and for a flat fitness landscape.
A new form of Genetic Programming (GP) called Cartesian Genetic Programming (CGP) is proposed in which programs are represented by linear integer chromosomes in the form of connections and functionalities of a rectangular array of primitive functions. The effectiveness of this approach is investigated for boolean even-parity functions (3,4,5), and the 2-bit multiplier. The minimum number of evaluations required to give a 0.99 probability of evolving a target function is used to measure the efficiency of the new approach. It is found that extremely low populations are most effective. A simple probabilistic hillclimber (PH) is devised which proves to be even more effective. For these boolean functions either method appears to be much more efficient than the GP and Evolutionary Programming (EP) methods reported. The efficacy of the PH suggests that boolean function learning may not be an appropriate problem for testing the effectiveness of GP and EP. 1. Introduction Since the original d...
. This paper presents a new form of Genetic Programming called Cartesian Genetic Programming in which a program is represented as an indexed graph. The graph is encoded in the form of a linear string of integers. The inputs or terminal set and node outputs are numbered sequentially. The node functions are also separately numbered. The genotype is just a list of node connections and functions. The genotype is then mapped to an indexed graph that can be executed as a program. Evolutionary algorithms are used to evolve the genotype in a symbolic regression problem (sixth order polynomial) and the Santa Fe Ant Trail. The computational effort is calculated for both cases. It is suggested that hit effort is a more reliable measure of computational efficiency. A neutral search strategy that allows the fittest genotype to be replaced by another equally fit genotype (a neutral genotype) is examined and compared with non-neutral search for the Santa Fe ant problem. The neutral search...
A Single Instruction Multiple Thread CUDA interpreter pro- vides SIMD like parallel evaluation of the whole GP popu- lation of 1 4 million RPN expressions on graphics cards and nVidia Tesla T10P. Using sub-machine code GP a sustain peak performance of 212 billion GP operations per second (3300 speed up) and an average of 4.5 peta GP ops per day is reported for a single card on a Boolean induction benchmark never attempted before, let alone solved.
A Single Instruction Multiple Thread CUDA interpreter provides SIMD like parallel evaluation of the whole GP population of
\frac14\frac{1}{4}million reverse polish notation (RPN) expressions on graphics cards and nVidia Tesla. Using sub-machine code tree GP a sustain
peak performance of 665 billion GP operations per second (10,000 speed up) and an average of 22 peta GP ops per day is reported
for a single GPU card on a Boolean induction benchmark never attempted before, let alone solved.
GPUs are difficult to program for general-purpose uses. Programmers can either learn graphics APIs and convert their applications to use graphics pipeline operations or they can use stream programming abstractions of GPUs. We describe Accelerator, a system that uses data parallelism to program GPUs for general-purpose uses instead. Programmers use a conventional imperative programming language and a library that provides only high-level data-parallel operations. No aspects of GPUs are exposed to programmers. The library implementation compiles the data-parallel operations on the fly to optimized GPU pixel shader code and API calls.We describe the compilation techniques used to do this. We evaluate the effectiveness of using data parallelism to program GPUs by providing results for a set of compute-intensive benchmarks. We compare the performance of Accelerator versions of the benchmarks against hand-written pixel shaders. The speeds of the Accelerator versions are typically within 50% of the speeds of hand-written pixel shader code. Some benchmarks significantly outperform C versions on a CPU: they are up to 18 times faster than C code running on a CPU.
In recent years the computing power of graphics cards has increased significantly. Indeed, the growth in the computing power of these graphics cards is now several orders of magni- tude greater than the growth in the power of computer pro- cessor units. Thus these graphics cards are now beginning to be used by the scientific community as low cost, high perfor- mance computing platforms. Traditional genetic program- ming is a highly computer intensive algorithm but due to its parallel nature it can be distributed over multiple processors to increase the speed of the algorithm considerably. This is not applicable for single processor architectures but graph- ics cards provide a mechanism for developing a data parallel implementation of genetic programming. In this paper we will describe the technique of general purpose computing using graphics cards and how to extend this technique to genetic programming. We will demonstrate the improve- ment in the performance of genetic programming on single processor architectures which can be achieved by harness- ing the computing power of these next generation graphics cards.
With over 800 million pages covering most areas of human endeavor, the World-wide Web is a fertile ground for data mining research to make a difference to the effectiveness of information search. Today, Web surfers access the Web through two dominant ...
Most simulations of biological evolution depend on a rather restricted set of properties. In this paper a richer model, based on differential gene expression is introduced to control developmental processes in an artificial evolutionary system. Differential gene expression is used to get different cell types and to modulate cell division and cell death. One of the advantages using developmental processes in evolutionary systems is the reduction of the information needed in the genome to encode e.g. shapes or cell types which results in better scaling behavior of the system. My result showed that the shaping of multicellular organisms in 3d is possible with the proposed system. 1 Introduction In the field of artificial evolution, current research tries to imitate biological concepts of evolution and development to simulate or build artificial organisms. This paper reports on a biologically inspired model that has been used to evolve 3d shapes of simulated, multicellular organisms. The...
Fast genetic programming on GPUs
- S Harding
- W Banzhaf
- M Ebner
- M Neill
- A Ekárt
- L Vanneschi
- Esparcia-Alcázar
A survey of self modifying CGP. Genetic Programming Theory and Practice
- S Harding
- J F Miller
- W Banzhaf
Distributed genetic programming on GPUs using CUDA Workshop on Parallel Architectures and Bioinspired Algorithms
- S L Harding
- W Banzhaf
Data: Third international knowledge discovery and data mining tools competition
- Kdd Cup
Optimizing shape design with distributed parallel genetic programming on GPUs
- S Harding
- W Banzhaf
- S. Harding
Genetic and Evolutionary Computation Conference Late Breaking Papers
- J Miller