No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Cartesian genetic programming
Abstract
We are interested in engineering smart machines that enable backtracking of emergent behaviors. Our SSNNS simulator consists of hand-picked tools to explore spiking neural networks in more depth with flexibility. SSNNS is based on the Spike Response ...
... Cartesian graph representation from the popular Cartesian genetic programming or CGP [29] is used in this work for candidate circuit representation. In this representation, an array of n c columns × n r rows of programmable nodes models an n i -input, n o -output combinational circuit. ...
... Since SYFR only has to focus on finding the best auxiliary circuits for logic masking, the design space exploration becomes easier. Note that other than the standard Cartesian graph allelic constraints from [29] and those proposed above, we enforce two additional constraints that ensure that: 1) all graph outputs are unique connections and 2) all the graph inputs are utilized. ...
In the event of an upset, fault-resilient circuits maintain correct functionality allowing the system to remain fully operational or at least operate with a graceful degradation. Every circuit has a certain level of inherent resilience to faults. Often times, this inherent resilience to faults is insufficient for the given application. This is because conventional synthesis tools generally only focus on optimizing a circuit with respect to area, power or timing budgets. There is a wide range of applications where faulty circuit behavior can lead to fatal results. Fault injection analyses are reported and show that even a single fault can be critical to the desired circuit operation. To which end, this paper presents SYFR, an evolutionary method for automated synthesis of increased fault-resilience digital circuits suitable for fine-grained use. Test results for synthesis of up to 60 input circuits with SYFR are reported. SYFR can be repeatedly applied to a circuit to obtain various design trade-offs between fault-resilience and implementation costs. SYFR can also be flexibly applied to build circuits which are selectively fault-resilient, i.e., their tolerance to faults is workload-aware. In addition, a novel population seeding mechanism to reduce the design space is introduced and experimentally validated. In summary, this paper demonstrates that SYFR can be considered a competitive synthesis methodology for constructing fault-resilient circuits.
... The most commonly used program representation formats of GP in the literature are tree, linear and graphical representations. For instance, one can consider standard GP [1] for tree-based representation, Cartesian Genetic Programming (CGP) [29] for graphical representation, and Linear Genetic Programming (LGP) [30] for linear representation. ...
... Cartesian Genetic Programming(CGP) [31] was first added to the literature by Miller in 1999. It was adopted as an innovative genetic programming type in the early 2000s [29]. As shown in Figure 3, programs in CGP are directly represented by grid graphics. ...
Genetic Programming (GP) is one of the evolutionary computation (EC) methods followed with great interest by many researchers. When GP first appeared, it has become a popular computational intelligence method because of its successful applications and its potentials to find effective solutions for difficult practical problems of many different disciplines. With the use of GP in a wide variety of areas, numerous variants of GP methods have emerged to provide more effective solutions for computation problems of diverse application fields. Therefore, GP has a very rich literature that is progressively growing. Many GP software tools developed along with process of GP algorithms. There is a need for an inclusive survey of GP literature from the beginning to today of GP in order to reveal the role of GP in the computational intelligence field. This survey study aims to provide an overview of the growing GP literature in a systematic way. The researchers, who need to implement GP methods, can gain insight of potentials in GP methods, their essential drawbacks and prevalent superiorities. Accordingly, taxonomy of GP methods is given by a systematic review of popular GP methods. In this manner, GP methods are analyzed according to two main categories, which consider the discrepancies in their program (chromosome) representation styles and their methodologies. Besides, GP applications in diverse problems are summarized. This literature survey is especially useful for new researchers to gain the required broad perspective before implementing a GP method in their problems.
... Typically, Cartesian Genetic Programming (CGP) [117,118,119] is used instead of working indirectly encoding graphs. In CGP, programs are represented by a two dimensional grid of program primitives. ...
... It is recommenced not to use crossover [118]. Due to the fixed maximum program size and the way connections are allowed, CGP programs are inherently sparse [119] and do not suffer the same problems with bloat as its tree based counterpart. Representing another dimension is as simple as adding another output gene without further constraints on possible program manipulations. ...
In the present work, we use symbolic regression for automated modeling of dynamical systems. Symbolic regression is a powerful and general method suitable for data-driven identification of mathematical expressions. In particular, the structure and parameters of those expressions are identified simultaneously. We consider two main variants of symbolic regression: sparse regression-based and genetic programming-based symbolic regression. Both are applied to identification, prediction and control of dynamical systems. We introduce a new methodology for the data-driven identification of nonlinear dynamics for systems undergoing abrupt changes. Building on a sparse regression algorithm derived earlier, the model after the change is defined as a minimum update with respect to a reference model of the system identified prior to the change. The technique is successfully exemplified on the chaotic Lorenz system and the van der Pol oscillator. Issues such as computational complexity, robustness against noise and requirements with respect to data volume are investigated. We show how symbolic regression can be used for time series prediction. Again, issues such as robustness against noise and convergence rate are investigated us- ing the harmonic oscillator as a toy problem. In combination with embedding, we demonstrate the prediction of a propagating front in coupled FitzHugh-Nagumo oscillators. Additionally, we show how we can enhance numerical weather predictions to commercially forecast power production of green energy power plants. We employ symbolic regression for synchronization control in coupled van der Pol oscillators. Different coupling topologies are investigated. We address issues such as plausibility and stability of the control laws found. The toolkit has been made open source and is used in turbulence control applications. Genetic programming based symbolic regression is very versatile and can be adapted to many optimization problems. The heuristic-based algorithm allows for cost efficient optimization of complex tasks. We emphasize the ability of symbolic regression to yield white-box models. In contrast to black-box models, such models are accessible and interpretable which allows the usage of established tool chains.
... We resort to EC for optimization. Indeed, EAs have already proven capable of dealing with graph-like structures Miller & Harding, 2008;K. O. Stanley & Miikkulainen, 2002), provided that a fitness function and an appropriate representation are given. ...
Modularity is a desirable property for embodied agents, as it could foster their suitability to different domains by disassembling them into transferable modules that can be reassembled differently. We focus on a class of embodied agents known as voxel-based soft robots (VSRs). They are aggregations of elastic blocks of soft material; as such, their morphologies are intrinsically modular. Nevertheless, controllers used until now for VSRs act as abstract, disembodied processing units: Disassembling such controllers for the purpose of module transferability is a challenging problem. Thus, the full potential of modularity for VSRs still remains untapped. In this work, we propose a novel self-organizing, embodied neural controller for VSRs. We optimize it for a given task and morphology by means of evolutionary computation: While evolving, the controller spreads across the VSR morphology in a way that permits emergence of modularity. We experimentally investigate whether such a controller (i) is effective and (ii) allows tuning of its degree of modularity, and with what kind of impact. To this end, we consider the task of locomotion on rugged terrains and evolve controllers for two morphologies. Our experiments confirm that our self-organizing, embodied controller is indeed effective. Moreover, by mimicking the structural modularity observed in biological neural networks, different levels of modularity can be achieved. Our findings suggest that the self-organization of modularity could be the basis for an automatic pipeline for assembling, disassembling, and reassembling embodied agents.
... Therefore, many researchers have specified the maximum allowable tree depth to control the growth of the search space even though the shape and length of the final rule are usually unknown ). There are many other GP representations presented in the literature to restrict the search space of the GP algorithm, such as grammar-based GP (Whigham 1995), cartesian GP (Miller and Harding 2008), dimensionally aware GP , and GEP (Ferreira 2001). ...
Dispatching rules are one of the most widely applied methods for solving Dynamic Job Shop Scheduling problems (DJSSP) in real-world manufacturing systems. Hence, the automated design of effective rules has been an important subject in the scheduling literature for the past several years. High computational requirements and difficulty in interpreting generated rules are limitations of literature methods. Also, feature selection approaches in the field of automated design of scheduling policies have been developed for the tree-based GP approach only. Therefore, the aim of this study is to propose a feature selection approach for the Gene Expression Programming (GEP) algorithm to evolve high-quality rules in simple structures with an affordable computational budget. This integration speeds up the search process by restricting the GP search space using the linear representation of the GEP algorithm and creates concise rules with only meaningful features using the feature selection approach. The proposed algorithm is compared with five algorithms and 30 rules from the literature under different processing conditions. Three performance measures are considered including total weighted tardiness, mean tardiness, and mean flow time. The results show that the proposed algorithm can generate smaller rules with high interpretability in a much shorter training time.
... What's worse, the number of nodes is usually limited and fixed. To represent variablelength network structures, Suganuma et al. [19] use Cartesian genetic programming (CGP) [99], [100] encoding scheme to represent the network as directed acyclic graphs, for the CNN architecture representation. In [16], individual architectures are also encoded as a graph, where vertices indicate rank-3 tensors or activations (batch normalization with ReLUs or plain linear units), and edges indicate identity connection or convolutions. ...
Deep learning (DL) techniques have penetrated all aspects of our lives and brought us great convenience. However,building a high-quality DL system for a specific task highly relies on human expertise, hindering the applications of DLto more areas. Automated machine learning (AutoML) becomes a promising solution to build a DL system withouthuman assistance, and a growing number of researchers focus on AutoML. In this paper, we provide a comprehensive andup-to-date review of the state-of-the-art (SOTA) in AutoML. First, we introduce AutoML methods according to thepipeline, covering data preparation, feature engineering, hyperparameter optimization, and neural architecture search(NAS). We focus more on NAS, as it is currently very hot sub-topic of AutoML. We summarize the performance of therepresentative NAS algorithms on the CIFAR-10 and ImageNet datasets and further discuss several worthy studyingdirections of NAS methods: one/two-stage NAS, one-shot NAS, and joint hyperparameter and architecture optimization.Finally, we discuss some open problems of the existing AutoML methods for future research.
... Node functions are also numbered separately. CGP has three parameters to be chosen by the user: number of rows n r , number of columns n c , and levels-back l [16]. ...
In this paper, we ask a question whether evolutionary algorithms can evolve cryptographic algorithms when no precise design criteria are given. Our strategy utilizes Cartesian Genetic Programming in the bi-level optimization setting with multiple populations trying to evolve a cryptographic algorithm and break it. To challenge our design paradigm, we consider a number of scenarios with varying criteria on the system and its security. We are able to obtain interesting results in several scenarios where the attacker is not able to understand the text with more than a random chance. Interestingly, our system is able to develop various versions of one-time pads, which are the only systems that ensure perfect secrecy. Although our system is far from practical, we consider it interesting since it gives good results that are also human-readable.
... CGP is a type of Genetic Programming (GP) invented by Miller and Thomson [8], where computer programs are encoded as directed acyclic graphs. Any GP model consists of a Genotype and a Phenotype where a Genotype is a collection of nodes represented as a graph, and the Phenotype is the decoding of the Genotype as a computer program. ...
Human nervous system has evolved for over 600 million years and can accomplish a wide variety of tasks effortlessly - telling whether a visual scene contains animals or buildings feels trivial to us, for example. For Artificial Neural Networks (ANNs) to carry out activities like these, requires careful design of networks by experts over years of difficult research, and typically address one specific task, such as to find what's in a photograph, or to help diagnose a disease. Preferably for any given task, one would want an automated technique to generate the right architecture. One approach to generate these architectures is through the use of evolutionary techniques. In this work, we test three methods i.e. CGP technique, ECGP technique, and a new crossover technique of CGP, for generating CNN architectures and report the results. To our knowledge, this is the first attempt on using either ECGP or the new crossover technique of CGP for evolving CNN architectures. This study is still in progress.
... Harding and Shepherd [44] describe a method of encoding a parametric model within a single numeric list, opening the topology of a parametric model itself for evolution. Encoding the model is similar to that used in Cartesian Genetic Programming (CGP) [45], which has been shown to be suitable for evolutionary methods due to its developmental mapping from genotype to phenotype. Such 'Meta-Parametric' methods have recently been investigated Joyce and Ibrahim [46] for the combinatorial search of parametric models. ...
Combining graph-based parametric design with metaheuristic solvers has to date focused solely on performance-based criteria and solving clearly defined objectives. In this article, we outline a new method for combining a parametric modelling environment with an interactive Cluster-Orientated Genetic Algorithm. In addition to performance criteria, evolutionary design exploration can be guided through choice alone, with user motivation that cannot be easily defined. As well as numeric parameters forming a genotype, the evolution of whole parametric definitions is discussed through the use of genetic programming. Visualisation techniques that enable mixing small populations for interactive evolution with large populations for performance-based optimisation are discussed, with examples from both academia and industry showing a wide range of applications.
Engineering design optimization poses a significant challenge, usually requiring human expertise to discover superior solutions. Although various search techniques have been employed to generate diverse designs, their effectiveness is often limited by problem-specific parameter tuning, making them less generalizable and scalable. This article introduces a framework inspired by evolutionary and developmental (evo-devo) concepts, aiming to automate the evolution of structural engineering designs. In biological systems, evo-devo governs the growth of single-cell organisms into multicellular organisms through the use of gene regulatory networks (GRNs). GRNs are inherently complex and highly nonlinear, and this article explores the use of neural networks and genetic programming as artificial representations of GRNs to emulate such behaviors. To evolve a wide range of Pareto fronts for artificial GRNs, this article introduces a new technique, a real value–encoded neuroevolutionary method termed real-encoded NEAT (RNEAT). The performance of RNEAT is compared with that of two well-known evolutionary search techniques across different 2-D and 3-D problems. The experimental results demonstrate two key findings. First, the proposed framework effectively generates a population of GRNs that can produce diverse structures for both 2-D and 3-D problems. Second, the proposed RNEAT algorithm outperforms its competitors on more than 50% of the problems examined. These results validate the proof of concept underlying the proposed evo-devo-based engineering design evolution.
A typical machine learning development cycle maximizes performance during model training and then minimizes the memory and area footprint of the trained model for deployment on processing cores, graphics processing units, microcontrollers or custom hardware accelerators. However, this becomes increasingly difficult as machine learning models grow larger and more complex. Here we report a methodology for automatically generating predictor circuits for the classification of tabular data. The approach offers comparable prediction performance to conventional machine learning techniques as substantially fewer hardware resources and power are used. We use an evolutionary algorithm to search over the space of logic gates and automatically generate a classifier circuit with maximized training prediction accuracy, which consists of no more than 300 logic gates. When simulated as a silicon chip, our tiny classifiers use 8–18 times less area and 4–8 times less power than the best-performing machine learning baseline. When implemented as a low-cost chip on a flexible substrate, they occupy 10–75 times less area, consume 13–75 times less power and have 6 times better yield than the most hardware-efficient ML baseline.
In recent years, the rapid advances in neural networks for Natural Language Processing (NLP) have led to the development of Large Language Models (LLMs), able to substantially improve the state-of-the-art in many NLP tasks, such as question answering and text summarization. Among them, one particularly interesting application is automatic code generation based only on the problem description. However, it has been shown that even the most effective LLMs available often fail to produce correct code. To address this issue, we propose an evolutionary-based approach using Genetic Improvement (GI) to improve the code generated by an LLM using a collection of user-provided test cases. Specifically, we employ Grammatical Evolution (GE) using a grammar that we automatically specialize—starting from a general one—for the output of the LLM. We test 25 different problems and 5 different LLMs, showing that the proposed method is able to improve in a statistically significant way the code generated by LLMs. This is a first step in showing that the combination of LLMs and evolutionary techniques can be a fruitful avenue of research.
Recently, finding the mathematical equations that match with data from any function has been considered a significant challenge for artificial intelligence and is known as symbolic regression. In a nutshell, symbolic regression is a subset of regression analysis that uses mathematical equation space to look for the best paradigm that matches the data and thus can match a much broader range of data sets than other paradigms, like linear regression. Explainable artificial intelligence has recently appeared where symbolic regression methods have been used for a long time to build models that are both understandable and tractable mathematically. Two symbolic regression methods, a network operator (NOP) and cartesian genetic programming (CGP), are discussed in detail. This study presents approaches for coding a mathematical equation and the basic collections of elementary functions that must be generated to perform this task. A comparative study for solving classical symbolic regression equations (benchmarks) has been carried out between the network operator method and cartesian genetic programming. It has been demonstrated through numerical results that the network operator outperforms cartesian genetic programming.
This study aims to determine the mechanical response of a brushless DC motor (BLDC) used in two-wheeled electric vehicles by analyzing torque values through finite element analysis. The motor features a three-phase stator structure and four permanent magnets on its rotor. The study investigates the relationship between torque, pulse degree, excitation voltage, and stator current using RMxprt software. A parametric dataset consisting of 600 data points is generated to model the BLDC motor system. Genetic programming (GP) is employed to establish a formula that correlates the motor’s output torque with the input variables. The resulting simplified formula, created with GP, achieves a mean absolute percentage error (MAPE) of 0.085 and an R-squared (R2) value of 0.989, indicating high accuracy in torque prediction based on simulation parameters. This research provides a torque formulation based on parametric finite element analysis, offering potential benefits for electric bicycles and potentially eliminating the need for certain sensors. Thus, before experimental studies in the process of determining BLDC motor torque behavior, a dataset approach based on FEA parametric simulation studies and a GP formulation developed based on the dataset obtained from parametric simulations were proposed.
Convolutional Neural Networks (CNNs) have shown a competitive performance in medical imaging applications, such as image segmentation. However, choosing an existing architecture capable of adapting to a specific dataset is challenging and requires design expertise. Neural Architecture Search (NAS) is employed to overcome these limitations. NAS uses techniques to design the Neural Networks architecture. Typically, the models’ weights optimization is carried out using a continuous loss function, unlike model topology optimization, which is highly influenced by the specific problem. Genetic Programming (GP) is an Evolutionary Algorithm (EA) capable of adapting to the topology optimization problem of CNNs by considering the attributes of its representation. A tree representation can express complex connectivity and apply variation operations. This paper presents a tree-based GP algorithm for evolving CNNs based on the well-known U-Net architecture producing compact and flexible models for medical image segmentation across multiple domains. This proposal is called Neural Architecture Search / Genetic Programming / U-Net (NASGP-Net). NASGP-Net uses a cell-based encoding and U-Net architecture as a backbone to construct CNNs based on a hierarchical arrangement of primitive operations. Our experiments indicate that our approach can produce remarkable segmentation results with fewer parameters regarding fixed architectures. Moreover, NASGP-Net presents competitive results against NAS methods. Finally, we observed notable performance improvements based on several evaluation metrics, including Dice similarity coefficient (DSC), Intersection over union (IoU), and Hausdorff Distance (HD).
Controllers are mission-critical components of any electronic design. By sending control signals, it decides which and when other data path elements must operate. Faults, especially Single Event Upset (SEU) occurrence in these components, can lead to functional/mission failure of the system when deployed in harsh environments. Hence, a competence to self-heal from SEU is highly required in the control path of the digital system. Reconfiguration is critical for recovering from a faulty state to a non-faulty state. Compared to native reconfiguration, the Virtual Reconfigurable Circuit (VRC) is an FPGA-generic reconfiguration mechanism. The non-partial reconfiguration in VRC and extensive architecture are considered hindrances in extending the VRC-based Evolvable Hardware (EHW) to real-time fault mitigation. To confront this challenge, we have proposed an intrinsic constrained evolution to improve the scalability and accelerate the evolution process for VRC-based fault mitigation in mission-critical applications. Experimentation is conducted on complex ACM/SIGDA benchmark circuits and real-time circuits used in space missions, which are not included in related works. In addition, a comparative study is made between existing and proposed methodologies for brushless DC motor control circuits. The hardware utilization in the multiplexer has been significantly reduced, resulting in up to 77% reduction in the existing VRC architecture. The proposed methodology employs a fault localization approach to narrow the search space effectively. This approach has yielded an 87% improvement on average in convergence speed, as measured by the evolution time compared to the existing work.
Grammar-Guided Genetic Programming (G3P) is widely recognised as one of the most successful approaches to program synthesis. Using a set of input/output tests, G3P evolves programs that fit a defined BNF grammar and that are capable of solving a wide range of program synthesis problems. However, G3P's inability to scale to more complex problems has limited its applicability. Recently, Generative Pre-trained Transformers (GPTs) have shown promise in revolutionizing program synthesis by generating code based on natural language prompts. However, challenges such as ensuring correctness and safety still need to be addressed as some GPT-generated programs might not work while others might include security vulnerabilities or blacklisted library calls. In this work, we proposed to combine GPT (in our case ChatGPT) with a G3P system, forcing any synthesised program to fit the BNF grammar-thus offering an opportunity to evolve/fix incorrect programs and reducing security threats. In our work, we leverage GPT-generated programs in G3P's initial population. However, since GPT-generated programs have an arbitrary structure, the initial work that we undertake is to devise a technique that maps such programs to a predefined BNF grammar before seeding the code into G3P's initial population. By seeding the grammar-mapped code into the population of our G3P system, we were able to successfully improve some of the desired programs using a well-known program synthesis benchmark. However, in its default configuration, G3P is not successful in fixing some incorrect GPT-generated programs-even when they are close to a correct program. We analysed the performance of our approach in depth and discussed its limitations and possible future improvements.
The approach known as Grammar-Guided Genetic Programming (G3P) is widely acknowledged as a highly effective method for program synthesis, which involves automatically generating code based on high-level formal specifications. Given the increasing quantity and scale of open software repositories and generative artificial intelligence techniques, there exists a significant range of methods for retrieving or generating source code using textual problem descriptions. Therefore, in light of the prevailing circumstances, it becomes imperative to introduce G3P into alternative means of user intent, with a specific focus on textual depictions. In our previous work, we assessed the potential for G3P to evolve programs based on bi-objectives that combine the similarity to the target program using four different similarity measures and the traditional input/output error rate. The result showed that such an approach improved the success rate for generating correct solutions for some of the considered problems. Nevertheless, it is noteworthy that despite the inclusion of various similarity measures, there is no single measure that uniformly improves the success rate of G3P across all problems. Instead, certain similarity measures exhibit effectiveness in addressing specific problems while demonstrating limited efficacy in others. In this paper, we would like to expand the bi-objective framework with different similarity measures to a many-objective framework to enhance the general performance of the algorithm to more range of problems. Our experiments show that compared to the bi-objective G3P (BOG3P), the Many-objective G3P (MaOG3P) approach could achieve the best result of all BOG3P algorithms with different similarity measures.
Evolutionary feature construction is a key technique in evolutionary machine learning, with the aim of constructing high-level features that enhance performance of a learning algorithm. In real-world applications, engineers typically construct complex features based on a combination of basic features, re-using those features as modules. However, modularity in evolutionary feature construction is still an open research topic. This paper tries to fill that gap by proposing a modular and hierarchical multitree genetic programming (GP) algorithm that allows trees to use the output values of other trees, thereby representing expressive features in a compact form. Based on this new representation, we propose a macro parent-repair strategy to reduce redundant and irrelevant features, a macro crossover operator to preserve interactive features, and an adaptive control strategy for crossover and mutation rates to dynamically balance the trade-off between exploration and exploitation. A comparison with seven bloat control methods on 98 regression datasets shows that the proposed modular representation achieves significantly better results in terms of test performance and smaller model size. Experimental results on the state-of-the-art symbolic regression benchmark demonstrate that the proposed symbolic regression method outperforms 22 existing symbolic regression and machine learning algorithms, providing empirical evidence for the superiority of the modularized evolutionary feature construction method.
Given the stringent requirements of energy efficiency for Internet-of-Things edge devices, approximate multipliers, as a basic component of many processors and accelerators, have been constantly proposed and studied for decades, especially in error-resilient applications. The computation error and energy efficiency largely depend on how and where the approximation is introduced into a design. Thus, this article aims to provide a comprehensive review of the approximation techniques in multiplier designs ranging from algorithms and architectures to circuits. We have implemented representative approximate multiplier designs in each category to understand the impact of the design techniques on accuracy and efficiency. The designs can then be effectively deployed in high-level applications, such as machine learning, to gain energy efficiency at the cost of slight accuracy loss.
Symbolic Regression searches for a parametric model with the optimal value of the parameters that best fits a set of samples to a measured target. The desired solution has a balance between accuracy and interpretability. Commonly there is no constraint in the way the functions are composed in the expression nor where the numerical parameters are placed, this can potentially lead to expressions that require a nonlinear optimization to find the optimal parameters. The representation called Interaction-Transformation alleviates this problem by describing expressions as a linear regression of the composition of functions applied to the interaction of the variables. One advantage is that any model that follows this representation is linear in its parameters, allowing an efficient computation. More recently, this representation was extended by applying a univariate function to the rational function of two Interaction-Transformation expressions, called Transformation-Interaction-Rational (TIR). The use of this representation was shown to be competitive with the current literature of Symbolic Regression. In this paper, we make a detailed analysis of these results using the SRBench benchmark. For this purpose, we split the datasets into different categories to understand the algorithm behavior in different settings. We also test the use of nonlinear optimization to adjust the numerical parameters instead of Ordinary Least Squares. We find through the experiments that TIR has some difficulties handling high-dimensional and noisy data sets, especially when most of the variables are composed of random noise. These results point to new directions for improving the evolutionary search of TIR expressions.
Image reconstruction is an interesting yet challenging optimization problem that has several potential applications. The task is to reconstruct an image using a fixed number of transparent polygons. Traditional gradient-based algorithms cannot be applied to the problem since the optimization objective has no explicit expression and cannot be represented by computational graphs. Metaheuristic search algorithms are powerful optimization techniques for solving complex optimization problems, especially in the context of incomplete information or limited computational capability. In this paper, we developed a novel metaheuristic search algorithm named progressive learning hill climbing (ProHC) for image reconstruction. Instead of placing all the polygons on a blank canvas at once, ProHC starts from one polygon and gradually adds new polygons to the canvas until reaching the number limit. Furthermore, an energy-map-based initialization operator was designed to facilitate the generation of new solutions. To assess the performance of the proposed algorithm, we constructed a benchmark problem set containing four different types of images. The experimental results demonstrated that ProHC was able to produce visually pleasing reconstructions of the benchmark images. Moreover, the time consumed by ProHC was much shorter than that of the existing approach.
Grammar-Guided Genetic Programming is widely recognised as one of the most successful approaches for program synthesis, i.e., the task of automatically discovering an executable piece of code given user intent. Grammar-Guided Genetic Programming has been shown capable of successfully evolving programs in arbitrary languages that solve several program synthesis problems based only on a set of input-output examples. Despite its success, the restriction on the evolutionary system to only leverage input/output error rate during its assessment of the programs it derives limits its scalability to larger and more complex program synthesis problems. With the growing number and size of open software repositories and generative artificial intelligence approaches, there is a sizeable and growing number of approaches for retrieving/generating source code based on textual problem descriptions. Therefore, it is now, more than ever, time to introduce G3P to other means of user intent (particularly textual problem descriptions). In this paper, we would like to assess the potential for G3P to evolve programs based on their similarity to particular target codes of interest (obtained using some code retrieval/generative approach). We particularly assess 4 similarity measures from various fields: text processing (i.e., FuzzyWuzzy), natural language processing (i.e., Cosine Similarity based on term frequency), software clone detection (i.e., CCFinder), plagiarism detector(i.e., SIM). Through our experimental evaluation on a well-known program synthesis benchmark, we have shown that G3P successfully manages to evolve some of the desired programs with three of the used similarity measures. However, in its default configuration, G3P is not as successful with similarity measures as with the classical input/output error rate at evolving solving program synthesis problems.KeywordsProgram synthesisGrammar-guided genetic programmingCode similarityTextual descriptionText to code
With a growing availability of ambient computing power as well as sensor data, networked systems are emerging in all areas of daily life. Coordination and optimization in complex cyber-physical systems demand for decentralized and self-organizing algorithms to cope with problem size and distributed information availability. Self-organization often relies on emergent behavior. Local observations and decisions aggregate to some global behavior without any apparent, explicitly programmed rule. Systematically designing algorithms with emergent behavior suitably for a new orchestration or optimization task is, at best, tedious and error prone. Appropriate design patterns are scarce so far. It is demonstrated that a machine learning approach based on Cartesian Genetic Programming is capable of learning the emergent mechanisms that are necessary for swarm-based optimization. Targeted emergent behavior can be evolved by evolution strategies. The learned swarm behavior is already significantly better than just random search. The encountered pitfalls as well as remaining challenges on the research agenda are discussed in detail. An additional fitness landscape analysis gives insight in obstructions during evolutions and clues for future improvements.
Membrane computing is a discipline that aims to perform computation by mimicking nature at the cellular level. Spiking Neural P (in short, SN P) systems are a subset of membrane computing methodologies that combine spiking neurons with membrane computing techniques, where “P” means that the system is intrinsically parallel. While these methodologies are very powerful, being able to simulate a Turing machine with only few neurons, their design is time-consuming and it can only be handled by experts in the field, that have an in-depth knowledge of such systems. In this work, we use the Neuroevolution of Augmenting Topologies (NEAT) algorithm, usually employed to evolve multi-layer perceptrons and recurrent neural networks, to evolve SN P systems. Unlike existing approaches for the automatic design of SN P systems, NEAT provides high flexibility in the type of SN P systems, removing the need to specify a great part of the system. To test the proposed method, we evolve Spiking Neural P systems as policies for two classic control tasks from OpenAI Gym. The experimental results show that our method is able to generate efficient (yet extremely simple) Spiking Neural P systems that can solve the two tasks. A further analysis shows that the evolved systems act on the environment by performing a kind of “if-then-else” reasoning.
Neural architecture search (NAS), which automatically designs the architectures of deep neural networks, has achieved breakthrough success over many applications in the past few years. Among different classes of NAS methods, evolutionary computation-based NAS (ENAS) methods have recently gained much attention. Unfortunately, the development of ENAS is hindered by unfair comparison between different ENAS algorithms due to different training conditions and high computational cost caused by expensive performance evaluation. This article develops a platform named BenchENAS, in short for benchmarking evolutionary NAS, to address these issues. BenchENAS makes it easy to achieve fair comparisons between different algorithms by keeping them under the same settings. To accelerate the performance evaluation in a common lab environment, BenchENAS designs a novel and generic efficient evaluation method for the population characteristics of evolutionary computation. This method has greatly improved the efficiency of the evaluation. Furthermore, BenchENAS is easy to install and highly configurable and modular, which brings benefits in good usability and easy extensibility. This article conducts efficient comparison experiments on eight ENAS algorithms with high GPU utilization on this platform. The experiments validate that the fair comparison issue does exist in the current ENAS algorithms, and BenchENAS can alleviate this issue. A Website has been built to promote BenchENAS at
https://benchenas.com
, where interested researchers can obtain the source code and document of BenchENAS for free.
Many practically relevant computing artifacts are forms of graphs, as, e.g., neural networks, mathematical expressions, finite automata. This great generality of the graph abstraction makes it desirable a way for searching in the space of graphs able to work effectively regardless of the graph form and application. In this paper, we propose GraphEA, a modular evolutionary algorithm (EA) for evolving graphs. GraphEA is modular in the sense that it can be tailored to any graph optimization task by providing components for specifying how to vary edges and nodes, and how to check the validity of a graph. We designed GraphEA by building on previous EAs dealing with particular kinds of graphs and included a speciation mechanism for the preservation of structural innovations and one for the gradual complexification of the solutions structure. To validate the generality of GraphEA, we applied it to 3 radically different tasks (regressions, in two flavors, text extraction from examples, evolution of controllers for soft robots) and compared its effectiveness against less general approaches. The results are promising and indicate some directions for further improvement of GraphEA.
We investigate the addition of constraints on the function image and its derivatives for the incorporation of prior knowledge in symbolic regression. The approach is called shape-constrained symbolic regression and allows us to enforce, for example, monotonicity of the function over selected inputs. The aim is to find models which conform to expected behavior and which have improved extrapolation capabilities. We demonstrate the feasibility of the idea and propose and compare two evolutionary algorithms for shape-constrained symbolic regression: (i) an extension of tree-based genetic programming which discards infeasible solutions in the selection step, and (ii) a two-population evolutionary algorithm that separates the feasible from the infeasible solutions. In both algorithms we use interval arithmetic to approximate bounds for models and their partial derivatives. The algorithms are tested on a set of 19 synthetic and four real-world regression problems. Both algorithms are able to identify models which conform to shape constraints which is not the case for the unmodified symbolic regression algorithms. However, the predictive accuracy of models with constraints is worse on the training set and the test set. Shape-constrained polynomial regression produces the best results for the test set but also significantly larger models.
This paper presents a new method with a set of desirable properties for multi-pitch estimation of piano recordings. We propose a framework based on a set of classifiers to analyze audio input and to identify piano notes present in a given audio signal. Our system’s classifiers are evolved using Cartesian genetic programming: we take advantage of Cartesian genetic programming to evolve a set of mathematical functions that act as independent classifiers for piano notes. Two significant improvements are described: the use of a harmonic mask for better fitness values and a data augmentation process for improving the training stage. The proposed approach achieves competitive results using F-measure metrics when compared to state-of-the-art algorithms. Then, we go beyond piano and show how it can be directly applied to other musical instruments, achieving even better results. Our system’s architecture is also described to show the feasibility of its parallelization and its implementation as a real-time system. Our methodology is also a white-box optimization approach that allows for clear analysis of the solutions found and for researchers to learn and test improvements based on the new findings.
Interaction–Transformation (IT) is a new representation for Symbolic Regression that reduces the space of solutions to a set of expressions that follow a specific structure. The potential of this representation was illustrated in prior work with the algorithm called SymTree. This algorithm starts with a simple linear model and incrementally introduces new transformed features until a stop criterion is met. While the results obtained by this algorithm were competitive with the literature, it had the drawback of not scaling well with the problem dimension. This article introduces a mutation-only Evolutionary Algorithm, called ITEA, capable of evolving a population of IT expressions. One advantage of this algorithm is that it enables the user to specify the maximum number of terms in an expression. In order to verify the competitiveness of this approach, ITEA is compared to linear, nonlinear, and Symbolic Regression models from the literature. The results indicate that ITEA is capable of finding equal or better approximations than other Symbolic Regression models while being competitive to state-of-the-art nonlinear models. Additionally, since this representation follows a specific structure, it is possible to extract the importance of each original feature of a data set as an analytical function, enabling us to automate the explanation of any prediction. In conclusion, ITEA is competitive when comparing to regression models with the additional benefit of automating the extraction of additional information of the generated models.
Multiclass classification is one of the most fundamental tasks in data mining. However, traditional data mining methods rely on the model assumption, they generally can suffer from the overfitting problem on high dimension and low sample size (HDLSS) data. Trying to address multiclass classification problems on HDLSS data from another perspective, we utilize Genetic Programming (GP), an intrinsic evolutionary classification algorithm that can implement feature construction automatically without model assumption. This paper develops an ensemble-based genetic programming classification framework, the Sigmoid-based Ensemble Gene Expression Programming (SEGEP). To relieve the problem of output conflict in GP-based multiclass classifiers, the proposed method employs a flexible probability representation with continuous relaxation to better integrate the output of all the binary classifiers, an effective data division strategy to further enhance the ensemble performance, and a novel sampling strategy to refine the existing GP-based binary classifier. The experiment results indicate that SE-GEP can attain better classification accuracy compared to other GP methods. Moreover, the comparison with other representative machine learning methods indicates that SE-GEP is a competitive method for multiclass classification in HDLSS data.
This paper proposes the use of Evolvable Hardware (EH) architectures as a countermeasure against power analysis attacks. It is inspired by the work of Sasdrich et al., in which the block cipher PRESENT is protected against power analysis attacks through the use of dynamic logic FPGA reconfiguration. The countermeasure consists of splitting the substitution boxes (S-boxes) into two parts with a register in between; the way the S-boxes are split is random and is altered before each new execution of the block cipher. This makes it very difficult (or even impossible) for an attacker to perform a Differential Power Analysis (DPA) attack by collecting many power traces of the same implementation.
During general anesthesia, anesthesiologists who provide anesthetic dosage traditionally play a fundamental role to regulate Bispectral Index (BIS). However, in this paper, an optimized type-2 Self-Organizing Fuzzy Logic Controller (SOFLC) is designed for Target Controlled Infusion (TCI) pump related to propofol dosing guided by BIS, to realize automatic control of general anesthesia. The type-2 SOFLC combines a type-2 fuzzy logic controller with a self-organizing (SO) mechanism to facilitate online training while able to contend with operational uncertainties. A novel data driven Surrogate Model (SM) and Genetic Programming (GP) based strategy is introduced for optimizing the type-2 SOFLC parameters offline to handle inter-patient variability. A pharmacological model is built for simulation in which different optimization strategies are tested and compared. Simulation results are presented to demonstrate the applicability of our approach and show that the proposed optimization strategy can achieve better control performance in terms of steady state error and robustness.
Cartesian Genetic Programming (CGP) is a variant of Genetic Programming with several advantages. During the last one and a half decades, CGP has been further extended to several other forms with lots of promising advantages and applications. This article formally discusses the classical form of CGP and its six different variants proposed so far, which include Embedded CGP, Self-Modifying CGP, Recurrent CGP, Mixed-Type CGP, Balanced CGP, and Differential CGP. Also, this article makes a comparison among these variants in terms of population representations, various constraints in representation, operators and functions applied, and algorithms used. Further, future work directions and open problems in the area have been discussed.
This paper formally introduces Recurrent Cartesian Genetic Programming (RCGP), an extension to Cartesian Genetic Programming (CGP) which allows recurrent connections. The presence of recurrent connections enables RCGP to be successfully applied to partially observable tasks. It is found that RCGP significantly outperforms CGP on two partially observable tasks: artificial ant and sunspot prediction. The paper also introduces a new parameter, recurrent connection probability, which biases the number of recurrent connections created via mutation. Suitable choices of this parameter significantly improve the effectiveness of RCGP.
A biologically inspired developmental model targeted at hardware implementation (off-shelf FPGA) is proposed which exhibits extremely robust transient fault-tolerant capability: in the software simulation of the experimen- tal application. In a 6x6 cell French Flag, some individuals were discovered us- ing evolution that have the ability to "recover" themselves from almost any kinds of transient faults, even in the worst case of only one "live" cell remain- ing. All cells in this model have identical genotype (physical structures), and only differ in internal states.
Techniques from the field of Evolutionary Computation are used to evolve a wide variety of aesthetically pleasing images using Cartesian Genetic Programming (CGP). The challenges that arise from employing a fitness function based on aesthetics, and the benefits that CGP can provide, are investigated and discussed. A significant piece of software was developed that places a focus on providing the user with efficient control over the evolutionary process. Several 'non-user' fitness functions that assess the phenotypes and genotypes of the chromosomes were also employed with varying success. To improve these results, methods of maintaining diversity within the population that take advantage of the neutrality of CGP are implemented and tested.
We argue that there is an upper limit on the complexity of software that can be constructed using current methods. Furthermore, this limit is orders of mag-nitude smaller than the complexity of living systems. We argue that many of the ad-vantages of autonomic computing will not be possible unless fundamental aspects of living systems are incorporated into a new paradigm of software construction. Truly self-healing and maintaining software will require methods of construction that mimic the biological development of multi-cellular organisms. We demonstrate a prototype system which is capable of autonomous repair and regeneration without using engineered methods. A method for evolving programs that construct multi-cellular structures (organisms) is described.
As is typical in evolutionary algorithms, fitness evaluation in GP takes the majority of the computational effort. In this
paper we demonstrate the use of the Graphics Processing Unit (GPU) to accelerate the evaluation of individuals. We show that
for both binary and floating point based data types, it is possible to get speed increases of several hundred times over a
typical CPU implementation. This allows for evaluation of many thousands of fitness cases, and hence should enable more ambitious
solutions to be evolved using GP.
Structure-based virtual screening is a technology increasingly used in drug discovery. Although successful at estimating binding
modes for input ligands, these technologies are less successful at ranking true hits correctly by binding free energy. This
chapter presents the automated removal of false positives from virtual hit sets, by evolving a post docking filter using Cartesian
Genetic Programming(CGP). We also investigate characteristics of CGP for this problem and confirm the absence of bloat and
the usefulness of neutral drift.
An evolutionary algorithm is used as an engine for discovering new designs of digital circuits, particularly arithmetic functions. These designs are often radically different from those produced by top-down, human, rule-based approaches. It is argued that by studying evolved designs of gradually increasing scale, one might be able to discern new, efficient, and generalizable principles of design. The ripple-carry adder principle is one such principle that can be inferred from evolved designs for one and two-bit adders. Novel evolved designs for three-bit binary multipliers are given that are 20% more efficient (in terms of number of two-input gates used) than the most efficient known conventional design.
A biologically inspired developmental model targeted at hardware implementation (off-shelf FPGA) is proposed which exhibits
extremely robust transient fault-tolerant capability. All cells in this model have identical genotype (physical structures),
and only differ in internal states. In a 3x3 cell digital organism, some individuals which implement a 2-bit multiplier were
discovered using evolution that have the ability to “recover” themselves from almost any kinds of transient faults. An intrinsic
evolvable hardware platform based on FPGA was realized to speed up the evolution process.
In a previous work it was argued that by studying evolved designs of gradually increasing scale, one might be able to discern new, efficient, and generalisable principles of design. These ideas are tested in the context of designing digital circuits, particularly arithmetic circuits. This process of discovery is seen as a principle extraction loop in which the evolved data is analysed both phenotypically and genotypically by processes of data mining and landscape analysis. The information extracted is then fed back into the evolutionary algorithm to enhance its search capabilities and hence increase the likelihood of identifying new principles which explain how to build systems which are too large to evolve.
The paper introduces a new approach to automatic design of image filters for a given type of noise. The approach employs evolvable
hardware at simplified functional level and produces circuits that outperform conventional designs. If an image is available
both with and without noise, the whole process of filter design can be done automatically, without influence of a designer.
An intrinsic evolvable hardware platform was realized to accelerate the evolutionary search process of a biologically inspired developmental model targeted at off-shelf FPGA implementation. The model has the capability of exhibiting very large transient fault-tolerance. The evolved circuits make up a digital "organism" from identical cells which only differ in internal states. Organisms implementing a 2-bit multiplier were evolved that can "recover" from almost any kinds of transient faults. This paper focuses on the design concerns and details of the evolvable hardware system, including the digital organism/cell and the intrinsic FPGA-based evolvable hardware platform.
In this paper we present a new chromosome representation for evolving digital circuits. The representation is based very closely on the c hip architecture of the Xilinx 6216 FPGA. We e xamine the e ffectiveness of evolving circuit functionality by using randomly chosen examples taken from the truth table. We consider the merits of a cell architecture in which functional cells alternate with routing cells and compare this with an architecture in which any cell can implement a function o r be merely used for r outing signals. It i s noteworthy that t he presence of elitism significantly improves the Genetic Algorithm performance.
Embedded Cartesian Genetic Programming (ECGP) is a form of Ge- netic Programming based on an acyclic directed graph representation. In this paper we investigate the use of ECGP together with a technique called Product Reduction (PR) to reduce the time required to evolve a digital multiplier. The results are compared with Cartesian Genetic Programming (CGP) with and without PR and show that ECGP improves evolvability and also that PR im- proves the performance of both techniques by up to eight times on the digital multiplier problems tested.
A review is given o f approaches to g rowing n eural networks and electronic circuits. A new method for growing graphs and circuits using a de- velopmental process is discussed. The method is inspired by the view that the cell is the basic unit of biology. Programs that construct circuits are evolved to bu ild a sequence of digital circuits at user specified iterations. The pro- grams can be run for an arbitrary number of iterations so circuits of huge size could be created that could not be evolved. It is shown that the circuit build- ing programs are capable of correctly predicting the next circuit in a sequence of larger even parity functions. The new method however finds building spe- cific circuits more difficult than a non-developmental method.
The exploitation of the physical characteristics has already been dem- onstrated in the intrinsic evolution of electronic circuits. This paper is an initial attempt at creating a world in which "physics" can be exploited in simulation. As a starting point we investigate a model of gate-like components with added noise. We refer to this as a kind of messiness. The principal idea behind these messy gates is that artificial evolution makes a virtue of the untidiness. We are ultimately trying to study the question: What kind of components should we use in artificial evolution? Several experiments are described that show that the messy circuits have a natural robustness to noise, as well as an implicit fault- tolerance. In addition, it was relatively easy for evolution to generate novel cir- cuits that were surprisingly efficient.
The paper presents for the first time automatic module acquisition and evolution within the graph based Cartesian Genetic Programming method. The method has been tested on a set of even parity problems and compared with Cartesian Genetic Programming without modules. Results are given that show that the new modular method evolves solutions up to 20 times quicker than the original non-modular method and that the speedup is more pronounced on larger problems. Analysis of some of the evolved modules shows that often they are lower order parity functions. Prospects for further improvement of the method are discussed.
Prime generating polynomial functions are known that can produce sequences of prime numbers (e.g. Euler polynomials). However,
polynomials which produce consecutive prime numbers are much more difficult to obtain. In this paper, we propose approaches
for both these problems. The first uses Cartesian Genetic Programming (CGP) to directly evolve integer based prime-prediction
mathematical formulae. The second uses multi-chromosome CGP to evolve a digital circuit, which represents a polynomial. We
evolved polynomials that can generate 43 primes in a row. We also found functions capable of producing the first 40 consecutive
prime numbers, and a number of digital circuits capable of predicting up to 208 consecutive prime numbers, given consecutive
input values. Many of the formulae have been previously unknown.
We propose building neutral networks in needle-in-haystack fitness landscapes to assist an evolutionary algorithm to perform search. The experimental results on four different problems show that this approach improves the search success rates in most cases. In situations where neutral networks do not give performance improvement, no impairment occurs either. We also tested a hypothesis proposed in our previous work. The results support the hypothesis: when the ratio of adaptive/neutral mutations during neutral walk is close to the ratio of adaptive/neutral mutations at the fitness improvement step, the evolutionary search has a high success rate. Moreover, the ratio magnitudes indicate that more neutral mutations (than adaptive mutations) are required for the algorithms to find a solution in this type of search space.
This work is a study of neutrality in the context of Evolutionary Computation systems. In particular, we introduce the use of explicit neutrality with an integer string coding scheme to allow neutrality to be measured during evolution. We tested this method on a Boolean benchmark problem. The experimental results indicate that there is a positive relationship between neutrality and evolvability: neutrality improves evolvability. We also identify four characteristics of adaptive/neutral mutations that are associated with high evolvability. They may be the ingredients in designing effective Evolutionary Computation systems for the Boolean class problem.
Cartesian Genetic Programming is a graph based representa- tion that has many benefits over traditional tree based methods, includ- ing bloat free evolution and faster evolution through neutral search. Here, an integer based version of the representation is applied to a traditional problem in the field: evolving an obstacle avoiding robot controller. The technique is used to rapidly evolve controllers that work in a complex en- vironment and with a challenging robot design. The generalisation of the robot controllers in different environments is also demonstrated. A novel fitness function based on chemical gradients is presented as a means of improving evolvability in such tasks.
Embedded Cartesian Genetic Programming (ECGP) is an extension of Cartesian Genetic Programming (CGP) capable of acquiring,
evolving and re-using partial solutions. In this paper, we apply for the first time CGP and ECGP to the ones-max and order-3
deceptive problems, which are normally associated with Genetic Algorithms. Our approach uses CGP and ECGP to evolve a sequence
of commands for a tape-head, which produces an arbitrary length binary string on a piece of tape. Computational effort figures
are calculated for CGP and ECGP and our results compare favourably with those of Genetic Algorithms.
A cell based optimization (CBO) algorithm is proposed which takes inspiration from the collective behaviour of cellular slime molds (Dictyostellium discoideum). Experiments with CBO are conducted to study the ability of simple cell-like agents to collectively manage resources across a distributed network. Cells, or agents, only have local information can signal, move, divide, and die. Heterogeneous populations of the cells are evolved using Cartesian genetic programming (CGP). Several experiments were carried out to examine the adaptation of cells to changing user demand patterns. CBO performance was compared using various methods to change demand. The experiments showed that populations consistently evolve to produce effective solutions. The populations produce better solutions when user demand patterns fluctuated over time instead of environments with static demand. This is a surprising result that shows that populations need to be challenged during the evolutionary process to produce good results.
Simple digital FIR filters have recently been evolved directly in the reconfigurable gate array, ignoring thus a classical
method based on multiply–and–accumulate structures. This work indicates that the method is very problematic. In this paper,
the gate-level approach is extended to IIR filters, a new approach is proposed to the fitness calculation based on the impulse
response evaluation and a comparison is performed between the evolutionary FIR filter design utilizing a full set and a reduced
set of gates. The objective of these experiments is to show that the evolutionary design of digital filters at the gate level
does not produce filters that are useful in practice when linearity of filters is not guaranteed by the evolutionary design
method.
A study is made to learn h ow multicellular organisms might have e merged from single- celled organisms. An understanding of this phenomenon might provide a better understanding of natural and man-made multicellular systems. The experiment performed is an Artificial Life simulation that uses Cartesian Genetic Programming to evolve the behaviors of individual cells. Cells have the ability to sense c hemicals around them, signal, divide, and move towards or away from chemicals. Interesting g roup behavior emerged from the simple instruction sets used by the cells.
Genetic Programming was first introduced by Koza using tree representation together with a crossover technique in which random sub-branches of the parents' trees are swapped to create the offspring. Later Miller and Thomson introduced Cartesian Genetic Programming, which uses directed graphs as a representation to replace the tree structures originally introduced by Koza. Cartesian Genetic Programming has been shown to perform better than the traditional Genetic Programming; but it does not use cross- over to create offspring, it is implemented using mutation only. In this paper a new crossover method in Genetic Programming is introduced. The new technique is based on an adaptation of the Cartesian Genetic Programming representation and is tested on two simple regression prob- lems. It is shown that by implementing the new crossover technique, convergence is faster than that of using mutation only in the Cartesian Genetic Programming method.
A developmental model of neural network is presented and evaluated in the game of Checkers. The network is devel- oped using cartesian genetic programs (CGP) as genotypes. Two agents are provided with this network and allowed to co-evolve untill they start playing better. The network that occurs by running theses genetic programs has a highly dy- namic morphology in which neurons grow, and die, and neu- rite branches together with synaptic connections form and change in response to situations encountered on the check- ers board. The method has no board evaluation function, no explicit learning rules and no human expertise at play- ing checkers is used. The results show that, after a number of generations, by playing each other the agents begin to play much better and can easily beat agents that occur in earlier generations. Such learning abilities are encoded at a genetic level rather than at the phenotype level of neural connections.
Embedded Cartesian Genetic Programming (ECGP) is an extension of the directed graph based Cartesian Genetic Pr- ogramming (CGP), which is capable of automatically ac- quiring, evolving and re-using partial solutions in the form of modules. In this paper, we apply for the first time, CGP and ECGP to the well known Lawnmower problem and to the Hierarchical-if-and-Only-if problem. The latter is nor- mally associated with Genetic Algorithms. Computational eort figures are calculated from the results of both CGP and ECGP and our results compare favourably with other techniques.
Embedded Cartesian Genetic Programming (ECGP) is a form of the graph based Cartesian Genetic Programming (CGP) in which modules are automatically acquired and evolved. In this paper we compare the efficiencies of the ECGP and CGP techniques on three classes of problem: digital adders, digital multipliers and digital comparators. We show that in most cases ECGP shows a substantial improvement in performance over CGP and that the computational speedup is more pronounced on larger problems.
This paper shows that the evolutionary design of digital cir- cuits which is conducted at the gate level is able to produce human-competitive circuits at the transistor level. In addi- tion to standard gates, we utilize unconventional gates (such as the NAND/NOR gate and NOR/NAND gate) that con- sist of a few transistors but exhibit non-trivial 3-input logic functions. Novel implementations of adders and majority circuits evolved using these gates contain fewer transistors than the smallest existing implementations of these circuits. Moreover, it was shown that the use of these gates signi- cantly improves the success rate of the search process.
Classical Evolutionary Programming (CEP) and Fast Evo- lutionary Programming (FEP) have been applied to real- valued function optimisation. Both of these techniques di- rectly evolve the real-values that are the arguments of the real-valued function. In this paper we have applied a form of genetic programming called Cartesian Genetic Program- ming (CGP) to a number of real-valued optimisation bench- mark problems. The approach we have taken is to evolve a computer program that controls a writing-head, which moves along and interacts with a finite set of symbols that are interpreted as real numbers, instead of manipulating the real numbers directly. In other studies, CGP has already been shown to benefit from a high degree of neutrality. We hope to exploit this for real-valued function optimisation problems to avoid being trapped on local optima. We have also used an extended form of CGP called Embedded CGP (ECGP) which allows the acquisition, evolution and re-use of modules. The effectiveness of CGP and ECGP are com- pared and contrasted with CEP and FEP on the benchmark problems. Results show that the new techniques are very ef- fective.
A coevolutionary competitive learning environment for two antagonistic agents is presented. The agents are controlled by a new kind of computational network based on a com-partmentalised model of neurons. The genetic basis of neu-rons is an important [27] and neglected aspect of previous approaches. Accordingly, we have defined a collection of chromosomes representing various aspects of the neuron: soma, dendrites and axon branches, and synaptic connec-tions. Chromosomes are represented and evolved using a form of genetic programming (GP) known as Cartesian GP. The network formed by running the chromosomal programs, has a highly dynamic morphology in which neurons grow, and die, and neurite branches together with synaptic con-nections form and change in response to environmental in-teractions. The idea of this paper is to demonstrate the importance of the genetic transfer of learned experience and life time learning. The learning is a consequence of the com-plex dynamics produced as a result of interaction (coevolu-tion) between two intelligent agents. Our results show that both agents exhibit interesting learning capabilities.
ABSTRACT The paper focuses on the evolution of algorithms for control of a machine in the presence of sensor faults, using Carte- sian Genetic Programming. The key challenges in creating training sets and a fltness function that encourage a general solution are discussed. The evolved algorithms are analysed and discussed. It was found that highly novel, mathemati- cally elegant and hitherto unknown,solutions were found. Categories and Subject Descriptors
A method for evolving p rograms that construct multicellular structures (organisms) is described. The paper concentrates on the difficult problem of evolv- ing a cell program that constructs a fixed size French flag. We obtain and analyze an organism that shows a remarkable a bility to repair itself when subjected to severe damage. Its behaviour resembles the regenerative power of some living organisms.
Embedded Cartesian Genetic Programming (ECGP) is an extension of Cartesian Genetic Programming (CGP) that can automatically acquire, evolve and re-use partial solu- tions in the form of modules. In this paper, we introduce for the first time a new multi-chromosome approach to CGP and ECGP that allows dicult problems with multiple out- puts to be broken down into many smaller, simpler problems with single outputs, whilst still encoding the entire solution in a single genotype. We also propose a multi-chromosome evolutionary strategy which selects the best chromosomes from the entire population to form the new fittest individ- ual, which may not have been present in the population. The multi-chromosome approach to CGP and ECGP is tested on a number of multiple output digital circuits. Computational Eort figures are calculated for each problem and compared against those for CGP and ECGP. The results indicate that the use of multiple chromosomes in both CGP and ECGP provide a significant performance increase on all problems tested.
This paper describes experiments to determine how the architecture vis-a-vis routing and functional resources affect the ease with which combinational logic designs may be evolved on a field-programmable gate array (FPGA). We compare two chromosome representations with differing levels of connectivity, and show that the amount of routing and functional resource have a marked effect on the success of the evolutionary process.
This paper examines the idea of a genetic or evolutionary algorithm being an inspirational or discovery engine. This is illustrated in the particular context of designing electronic circuits. We argue that by connecting pieces of logic together and testing them to see if they carry out the desired function it may be possible to discover new principles of design, and new algebraic techniques. This is illustrated in the design of binary circuits, particularly arithmetic functions, where we demonstrate that by evolving a hierarchical series of examples, it becomes possible to re-discover the well known ripple-carry principle for building adder circuits of any size. We also examine the much harder case of multiplication. We show also that extending the work into the field of multiple-valued logic, the genetic algorithm is able to produce fully working circuits that lie outside conventional algebra. In addition we look at the issue of principle extraction from evolved data.
A perceived limitation of evolutionary art and design algorithms is that they rely on human intervention; the artist selects the most aesthetically pleasing variants of one generation to produce the next. This paper discusses how computer generated art and design can become more creatively human-like with respect to both process and outcome. As an example of a step in this direction, we present an algorithm that overcomes the above limitation by employing an automatic fitness function. The goal is to evolve abstract portraits of Darwin, using our 2nd generation fitness function which rewards genomes that not just produce a likeness of Darwin but exhibit certain strategies characteristic of human artists. We note that in human creativity, change is less choosing amongst randomly generated variants and more capitalizing on the associative structure of a conceptual network to hone in on a vision. We discuss how to achieve this fluidity algorithmically.
Evolving a Self-Repairing, Self-Regulating, French Flag Organism
A method for evolving a developmental program inside a cell to create multicellular organisms of arbitrary size and characteristics is described. The cell genotype is evolved so that the organism will organize itself into well defined patterns of differentiated cell types (e.g. the French Flag). In addition the cell genotypes are evolved to respond appropriately to environmental signals that cause metamorphosis of the whole organism. A number of experiments are described that show that the organisms exhibit emergent properties of self-repair and adaptation.
An evolutionary system that supports the interaction of neutral and adaptive mutations is investigated. Experimental results on a Boolean function and needle-in-haystack problems show that this system enables evolutionary search to find better solutions faster. Through a novel analysis based on the ratio of neutral to adaptive mutations, we identify this interaction as an engine that automatically adjusts the relative amounts of exploration and exploitation to achieve effective search (i.e., it is self-adaptive). Moreover, a hypothesis to describe the search process in this system is proposed and investigated. Our findings lead us to counter the arguments of those who dismiss the usefulness of neutrality. We argue that the benefits of neutrality are intimately related to its implementation, so that one must be cautious about making general claims about its merits or demerits.
We use evolutionary search to design combinational logic circuits. The technique is based on evolving the functionality and connectivity of a rectangular array of logic cells whose dimension is defined by the circuit layout. The main idea of this approach is to improve quality of the circuits evolved by the genetic algorithm (GA) by reducing the number of active gates used. We accomplish this by combining two ideas: 1) using multi-objective fitness function; 2) evolving circuit layout. It will be shown that using these two approaches allows us to increase the quality of evolved circuits. The circuits are evolved in two phases. Initially the genome fitness is given by the percentage of output bits that are correct. Once 100% functional circuits have been evolved, the number of gates actually used in the circuit is taken into account in the fitness function. This allows us to evolve circuits with 100% functionality and minimise the number of active gates in circuit structure. The population is initialised with heterogeneous circuit layouts and the circuit layout is allowed to vary during the evolutionary process. Evolving the circuit layout together with the function is one of the distinctive features of proposed approach. The experimental results show that allowing the circuit layout to be flexible is useful when we want to evolve circuits with the smallest number of gates used. We find that it is better to use a fixed circuit layout when the objective is to achieve the highest number of 100% functional circuits. The two-fitness strategy is most effective when we allow a large number of generations
The two-bit multiplier is a simple electronic circuit, small enough to be evolvable, and practically useful for the implementation of many digital systems. In this paper we study the structure of the two-bit multiplier fitness landscapes generated by circuit evolution on an idealised model of a field-programmable gate array. The two-bit multiplier landscapes are challenging. The difficulty in studying these landscapes stems from the genotype representation which allows us to evolve the functionality and connectivity of an array of logic cells. Here, the genotypes are simply strings defined over two completely different alphabets. This makes the study of the corresponding landscapes much more involved. We outline a model for studying the two-bit multiplier landscapes and estimate the amplitudes derived from the Fourier transform of these landscape. We show that the two-bit multiplier landscapes can be characterised in terms of subspaces, determined by the interactions between the genotype partitions
This paper presents a generalization of the graph- based genetic programming (GP) technique known as Cartesian genetic programming (CGP). We have extended CGP by utilizing automatic module acquisition, evolution, and reuse. To benchmark the new technique, we have tested it on: various digital circuit problems, two symbolic regression problems, the lawnmower problem, and the hierarchical if-and-only-if problem. The results show the new modular method evolves solutions quicker than the original nonmodular method, and the speedup is more pronounced on larger problems. Also, the new modular method performs favorably when compared with other GP methods. Analysis of the evolved modules shows they often produce recognizable functions. Prospects for further improvements to the method are discussed.
The graph-based Cartesian genetic programming system has an unusual genotype representation with a number of advantageous properties. It has a form of redundancy whose role has received little attention in the published literature. The representation has genes that can be activated or deactivated by mutation operators during evolution. It has been demonstrated that this "junk" has a useful role and is very beneficial in evolutionary search. The results presented demonstrate the role of mutation and genotype length in the evolvability of the representation. It is found that the most evolvable representations occur when the genotype is extremely large and in which over 95% of the genes are inactive.
Traditionally digital filters are designed using the concept of a linear difference equation with the output response being a weighted sum of signal samples with usually floating point coefficients.
The development of an entire organism from a single cell is one of the most profound and awe inspiring phenomena in the whole of the natural world. The complexity of living systems itself dwarfs anything that man has produced. This is all the more the case for the processes that lead to these intricate systems. In each phase of the development of a multi-cellular being, this living system has to survive, whether stand-alone or supported by various structures and processes provided by other living systems. Organisms construct themselves, out of humble single-celled beginnings, riding waves of interaction between the information residing in their genomes - inherited from the evolutionary past of their species via their progenitors - and the resources of their environment. Permanent renewal and self-repair are natural extrapolations of developmental recipes, as is adaptation to different environmental conditions. Multi-cellular organisms consist of a huge amount of cells, the atoms of life, modular structures used to perform all the functions of a body. It is estimated that there are of the order of 1013 cells in the human body. Some of them are dying and
This paper describes the application of intrinsic evolvable hardware to combinational circuit design and synthesis, as an alternative to conventional approaches. This novel reconfigurable architecture is inspired by Cartesian genetic programming and dedicated for implementing high performance digital image filters on a custom Xilinx Virtex FPGA xcv1000, together with a flexible local interconnection hierarchy. As a highly parallel architecture, it scales linearly with the filter complexity. It is reconfigured by an external genetic reconfiguration processing unit with a hardware GA implementation embedded. Due to pipelining, parallelization and no function call overhead, it yields a significant speedup of one to two orders of magnitude over a software implementation, which is especially useful for the real-time applications. The experimental results conclude that in terms of computational effort, filtered image signal and implementation cost, the intrinsic evolvable hardware solution outperforms traditional approaches.
The development of mathematical neural networks was based on an analogy with biological neural networks found in nature. Recently there has been a resurgence in research and understanding in self-organizing networks that are based on other metaphors: genetics, immune systems etc. In this paper a new methodology is presented for creating complex adaptive functional networks (CAFN) that are based on the particle swarm social-psychological metaphor. The proposed social programming methodology is based on combining the particle swarm methodology with the group method of data handling and Cartesian programming.