Conference PaperPDF Available

GECCO 2012 tutorial: Cartesian Genetic Programming

July 2011

DOI:10.1145/2001858.2002136

Source
DBLP

Conference: 13th Annual Genetic and Evolutionary Computation Conference, GECCO 2011, Companion Material Proceedings, Dublin, Ireland, July 12-16, 2011

Authors:

Julian Francis Miller

University of York

Simon Lawrence Harding

Machine Intelligence Ltd.

Cartesian Genetic Programming (CGP) is an increasingly popular and efficient form of Genetic Programming that was developed by Julian Miller in 1999 and 2000. In its classic form, it uses a very simple integer based genetic representation of a program in the form of a directed graph. Graphs are very useful program representations and can be applied to many domains (e.g. electronic circuits, neural networks). In a number of studies, CGP has been shown to be comparatively efficient to other GP techniques. It is also very simple to program. Since then, the classical form of CGP has been developed made more efficient in various ways. Notably by including automatically defined functions (modular CGP) and self-modification operators(self-modifying CGP). SMCGP was developed by Julian Miller, Simon Harding and Wolfgang Banzhaf. It uses functions that cause the evolved programs to change themselves as a function of time. Using this technique it is possible to find general solutions to classes of problems and mathematical algorithms (e.g. arbitrary parity, n-bit binary addition, sequences that provably compute pi and e to arbitrary precision, and so on). The tutorial will cover the basic technique, advanced developments and applications to a variety of problem domains.

Genotype-phenotype mapping a) Phenotype. b) Genotype. For a program with six inputs and 3 outputs, and three functions (0, 1, 2 inside square nodes, in italics in genotype). The grey squares indicate unconnected nodes.

…

The computational effort for neutral and non-neutral strategies for various mutation rates.

…

The number of hits for neutral and non-neutral strategies for various mutation rates.

…

The hit effort for neutral and non-neutral strategies for various mutation rates.

…

Figures - uploaded by Julian Francis Miller

Content may be subject to copyright.

Content uploaded by Julian Francis Miller

Content may be subject to copyright.

Content uploaded by Julian Francis Miller

Content may be subject to copyright.

Content uploaded by Julian Francis Miller

Content may be subject to copyright.

A preview of the PDF is not available

gecco2011-tutorial-miller

Data

June 2013

Julian Francis Miller · Simon Lawrence Harding

Download

Design of Power-Efficient Approximate Multipliers for Approximate Artificial Neural Networks

Conference Paper

Full-text available

Nov 2016

Artificial neural networks (NN) have shown a significant promise in difficult tasks like image classification or speech recognition. Even well-optimized hardware implementations of digital NNs show significant power consumption. It is mainly due to non-uniform pipeline structures and inherent redundancy of numerous arithmetic operations that have to be performed to produce each single output vector. This paper provides a methodology for the design of well-optimized power-efficient NNs with a uniform structure suitable for hardware implementation. An error resilience analysis was performed in order to determine key constraints for the design of approximate multipliers that are employed in the resulting structure of NN. By means of a search based approximation method, approximate multipliers showing desired tradeoffs between the accuracy and implementation cost were created. Resulting approximate NNs, containing the approximate multipliers, were evaluated using standard benchmarks (MNIST dataset) and a real-world classification problem of Street-View House Numbers. Significant improvement in power efficiency was obtained in both cases with respect to regular NNs. In some cases, 91% power reduction of multiplication led to classification accuracy degradation of less than 2.80%. Moreover, the paper showed the capability of the back propagation learning algorithm to adapt with NNs containing the approximate multipliers.

Tiny Classifier Circuits: Evolving Accelerators for Tabular Data

Preprint

Full-text available

Feb 2023

A typical machine learning (ML) development cycle for edge computing is to maximise the performance during model training and then minimise the memory/area footprint of the trained model for deployment on edge devices targeting CPUs, GPUs, microcontrollers, or custom hardware accelerators. This paper proposes a methodology for automatically generating predictor circuits for classification of tabular data with comparable prediction performance to conventional ML techniques while using substantially fewer hardware resources and power. The proposed methodology uses an evolutionary algorithm to search over the space of logic gates and automatically generates a classifier circuit with maximised training prediction accuracy. Classifier circuits are so tiny (i.e., consisting of no more than 300 logic gates) that they are called "Tiny Classifier" circuits, and can efficiently be implemented in ASIC or on an FPGA. We empirically evaluate the automatic Tiny Classifier circuit generation methodology or "Auto Tiny Classifiers" on a wide range of tabular datasets, and compare it against conventional ML techniques such as Amazon's AutoGluon, Google's TabNet and a neural search over Multi-Layer Perceptrons. Despite Tiny Classifiers being constrained to a few hundred logic gates, we observe no statistically significant difference in prediction performance in comparison to the best-performing ML baseline. When synthesised as a Silicon chip, Tiny Classifiers use 8-56x less area and 4-22x less power. When implemented as an ultra-low cost chip on a flexible substrate (i.e., FlexIC), they occupy 10-75x less area and consume 13-75x less power compared to the most hardware-efficient ML baseline. On an FPGA, Tiny Classifiers consume 3-11x fewer resources.

Extension des Programmes Génétiques pour l’apprentissage supervisé à partir de très larges Bases de Données (Big data)

Thesis

Oct 2019

Hmida Hmida

Dans cette thèse, nous étudions l'adaptation des Programmes Génétiques (GP) pour surmonter l'obstacle du volume de données dans les problèmes Big Data. GP est une méta‐heuristique qui a fait ses preuves pour les problèmes de classification. Néanmoins, son coût de calcul est un frein à son utilisation avec les larges bases d’apprentissage. Tout d'abord, nous effectuons une revue approfondie enrichie par une étude comparative expérimentale des algorithmes d'échantillonnage utilisés avec GP. Puis, à partir des résultats de l'étude précédente, nous proposons quelques extensions basées sur l'échantillonnage hiérarchique. Ce dernier combine des algorithmes d'échantillonnage actif à plusieurs niveaux et s’est prouvé une solution appropriée pour mettre à l’échelle certaines techniques comme TBS et pour appliquer GP à un problème Big Data (cas de la classification des bosons de Higgs). Par ailleurs, nous formulons une nouvelle approche d'échantillonnage appelée échantillonnage adaptatif, basée sur le contrôle de la fréquence d'échantillonnage en fonction du processus d'apprentissage, selon les schémas fixe, déterministe et adaptatif. Enfin, nous présentons comment transformer une implémentation GP existante (DEAP) en distribuant les évaluations sur un cluster Spark. Nous démontrons comment cette implémentation peut être exécutée sur des clusters à nombre de nœuds réduit grâce à l’échantillonnage. Les expériences montrent les grands avantages de l'utilisation de Spark pour la parallélisation de GP.

Compact version of Cartesian Genetic Programming

Conference Paper

Sep 2014

Petr Burian

This paper deals with the design of the compact version of Cartesian Genetic Programming. The focus is given to the search algorithm of type (1+1). The paper presents the approach that detects changes in the phenotype and, based on that, the algorithm can omit the evaluation of a candidate solution. The author uses the evolutionary design of multipliers as benchmark to present the efficiency of the algorithm.

Evolutionary Methods for the Construction of Cryptographic Boolean Functions

Conference Paper

Apr 2015
Lect Notes Comput Sci

Boolean functions represent an important primitive when constructing many stream ciphers. Since they are often the only nonlinear element of such ciphers, without them the algorithm would be trivial to break. Therefore, it is not surprising there exist a substantial body of work on the methods of constructing Boolean functions. Among those methods, evolutionary computation (EC) techniques play a significant role. Previous works show it is possible to use EC methods to generate high-quality Boolean functions that even surpass those built by algebraic constructions. However, up to now, there was no work investigating the use of Cartesian Genetic Programming (CGP) for producing Boolean functions suitable for cryptography. In this paper we compare Genetic Programming (GP) and CGP algorithms in order to reach the conclusion which algorithm is better suited to evolve Boolean functions suitable for cryptographic usage. Our experiments show that CGP performs much better than the GP when the goal is obtaining as high as possible nonlinearity. Our results indicate that CGP should be further tested with different fitness objectives in order to check the boundaries of its performance.

Investigating vesicular selection: A selection operator in in vitro evolution

Article

Dec 2011
APPL SOFT COMPUT

Directed protein evolution has led to major advances in organic chemistry, enabling the development of highly optimised proteins. The SELEX method has also been highly effective in evolving ribose nucleic acid (RNA) or deoxy-ribose nucleic acid (DNA) molecules; variants have been proposed which allow SELEX to be used in protein evolution. All of these methods can be viewed as evolutionary algorithms implemented in chemistry.A number of methods rely on selection of natural cells, or of artificial bubbles. These methods result in a new form of selection mechanism, which we call vesicular selection (VS). It is not, prima facie, clear whether VS is an effective selection mechanism, or how its performance is affected by changes in vesicle size. It is difficult to investigate this in vitro, so we use in silico methods derived from evolutionary computation. The primary aim is to test whether this selection method hinders biochemical evolutionary search (in which case, it might be worth investing research effort in discovering alternative selection methods).An in silico implementation of this selection method, embedded in an otherwise-typical evolutionary computation system, shows reasonable ability to solve tough optimisation problems, together with an acceptable ability to concentrate the solutions found. We compare it with tournament selection (TS), a standard evolutionary computation method, which can be finely tuned for high selection pressure, but only coarsely tuned for low selection pressure. By contrast, the new selection mechanism VS is highly tunable at low selection pressures. It is thus particularly suited to problem domains where extensive exploration capabilities are required. Since there is very good reason to believe that protein search spaces require highly exploratory search, the selection mechanism is well matched to its application in combinatorial chemistry.

An Updated Taxonomy of Evolutionary Computation Problems using Graph-based Evolutionary Algorithms

Conference Paper

Full-text available

Jan 2006

Graph based evolutionary algorithms use combinatorial graphs to impose a topology or "geographic structure" on an evolving population. It has been demonstrated that, for a fixed problem, time to solution varies substantially with the choice of graph. This variation is not simple with very different graphs yielding faster solution times for different problems. Normalized time to solution for many graphs thus forms an objective character that can be used for classifying the type of a problem, separate from its hardness measured with average time to solution. This study uses fifteen combinatorial graphs to classify 40 evolutionary computation problems. The resulting classification is done using neighbor joining, and the results are also displayed using non-linear projection. The different methods of grouping evolutionary computation problems into similar types exhibit substantial agreement. Numerical optimization problems form a close grouping while some other groups of problems scatter across the taxonomy. This paper updates an earlier taxonomy of 23 problems and introduces new classification techniques.

Evolution of sorting algorithm using graph structured program evolution

Conference Paper

Nov 2007

In this paper, we apply graph structured program evolution (GRAPE) to evolution of general sorting algorithm. GRAPE is a new Automatic Programming technique. The representation of GRAPE is graph structure, therefore it can express complex programs (e.g. branches and loops) using its graph structure. Each program is constructed as an arbitrary directed graph of nodes and data set. GRAPE handles multiple data types using data set for each type, and the genotype of GRAPE is the form of a linear string of integers. The aim of this work is to evolve a program which correctly sort any sequence of numbers. We demonstrate that GRAPE constructs general sorting algorithm automatically.

Comparison of CGP and Age-Layered CGP Performance in Image Operator Evolution

Conference Paper

Apr 2009
Lect Notes Comput Sci

Karel Slaný

This paper analyses the efficiency of the Cartesian Genetic Programming (CGP) methodology in the image operator design problem at the functional level. The CGP algorithm is compared with an age layering enhancement of the CGP algorithm by the means of achieved best results and their computational effort. Experimental results show that the Age-Layered Population Structure (ALPS) algorithm combined together with CGP can perform better in the task of image operator design in comparison with a common CGP algorithm.

GRACE: Generative Robust Analog Circuit Exploration

Conference Paper

Full-text available

Apr 2006
Lect Notes Comput Sci

We motivate and describe an analog evolvable hardware design platform named GRACE (i.e. Generative Robust Analog Circuit Exploration). GRACE combines coarse-grained, topological circuit search with intrinsic testing on a Commercial Off-The-Shelf (COTS) field programmable device, the Anadigm AN221E04. It is suited for adaptive, fault tolerant system design as well as CAD flow applications.

Recurrent Cartesian Genetic Programming

Conference Paper

Full-text available

Sep 2014
Lect Notes Comput Sci

This paper formally introduces Recurrent Cartesian Genetic Programming (RCGP), an extension to Cartesian Genetic Programming (CGP) which allows recurrent connections. The presence of recurrent connections enables RCGP to be successfully applied to partially observable tasks. It is found that RCGP significantly outperforms CGP on two partially observable tasks: artificial ant and sunspot prediction. The paper also introduces a new parameter, recurrent connection probability, which biases the number of recurrent connections created via mutation. Suitable choices of this parameter significantly improve the effectiveness of RCGP.

Intelligent agents capable of developing memory of their environment

Article

Full-text available

We investigate the emergence of intelligent behaviour of agents in the classic AI learning environment known as Wumpus World. The agent is given a 'brain' that is a new type of developmental neuro-inspired computational network. The neurons are defined by a genotype consisting of seven computational functions that represent electrical and develop-mental processes inside neuron. The computational functions are evolved using a form of Genetic Programming known as Cartesian Genetic Pro-gramming. The 'brain' that occurs by running the genetic programs has a highly dynamic morphology in which neurons grow, and die, and neurite branches together with synaptic connections form and change in response to situations encountered in the external environment. We present results and analyse characteristics of the model and find that the agents appear to exhibit 'instinctive' behaviour and develop a form of memory.

Parallel Distributed Genetic Programming

Conference Paper

Full-text available

Jan 1999

Riccardo Poli

An Intrinsic Robust Transient Fault-Tolerant Developmental Model for Digital Systems

Article

Full-text available

A biologically inspired developmental model targeted at hardware implementation (off-shelf FPGA) is proposed which exhibits extremely robust transient fault-tolerant capability: in the software simulation of the experimen- tal application. In a 6x6 cell French Flag, some individuals were discovered us- ing evolution that have the ability to "recover" themselves from almost any kinds of transient faults, even in the worst case of only one "live" cell remain- ing. All cells in this model have identical genotype (physical structures), and only differ in internal states.

Cartesian Genetic Programming for Image Processing (CGP-IP)

Chapter

Full-text available

Mar 2013

Combining domain knowledge about both imaging processing and machine learning techniques can expand the abilities of Genetic Programming when used for image processing. We successfully demonstrate our new approach on several different problem domains. We show that the approach is fast, scalable and robust. In addition, by virtue of using off-the-shelf image processing libraries we can generate human readable programs that incorporate sophisticated domain knowledge.

Evolutionary Art with Cartesian Genetic Programming

Article

Full-text available

Jan 2004

Techniques from the field of Evolutionary Computation are used to evolve a wide variety of aesthetically pleasing images using Cartesian Genetic Programming (CGP). The challenges that arise from employing a fitness function based on aesthetics, and the benefits that CGP can provide, are investigated and discussed. A significant piece of software was developed that places a focus on providing the user with efficient control over the evolutionary process. Several 'non-user' fitness functions that assess the phenotypes and genotypes of the chromosomes were also employed with varying success. To improve these results, methods of maintaining diversity within the population that take advantage of the neutrality of CGP are implemented and tested.

Beyond the complexity ceiling: Evolution, emergence and regeneration

Article

Full-text available

We argue that there is an upper limit on the complexity of software that can be constructed using current methods. Furthermore, this limit is orders of mag-nitude smaller than the complexity of living systems. We argue that many of the ad-vantages of autonomic computing will not be possible unless fundamental aspects of living systems are incorporated into a new paradigm of software construction. Truly self-healing and maintaining software will require methods of construction that mimic the biological development of multi-cellular organisms. We demonstrate a prototype system which is capable of autonomous repair and regeneration without using engineered methods. A method for evolving programs that construct multi-cellular structures (organisms) is described.

Evolutionary programming with tree mutations: Evolving computer programming without crossover

Article