No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Competitive Coevolution through Evolutionary Complexification
Abstract
Two major goals in machine learning are the discovery of complex multidimensional solutions and continual improvement of existing solutions. In this paper, we argue that complexification, i.e. the incremental elaboration of solutions through adding new structure, achieves both these goals. We demonstrate the power of complexification through the NeuroEvolution of Augmenting Topologies (NEAT) method, which evolves increasingly complex neural network architectures. NEAT is applied to an open-ended coevolutionary robot duel domain where robot controllers compete head to head. Because the robot duel domain supports a wide range of sophisticated strategies, and because coevolution benefits from an escalating arms race, it serves as a suitable testbed for observing the effect of evolving increasingly complex controllers. The result is an arms race of increasingly sophisticated strategies. When compared to the evolution of networks with fixed structure, complexifying networks discover significantly more sophisticated strategies. The results suggest that in order to realize the full potential of evolution, and search in general, solutions must be allowed to complexify as well as optimize.
... NeuroEvolution of Augmenting Topologies (NEAT) [23] is a well-known algorithm that evolves the weights and topologies of neural networks. NEAT was also successfully applied in a coevolution context [24]. The NEAT model was also expanded to work on larger search spaces, such as deep neural networks, in the DeepNEAT [16] method. ...
... Coevolution is the simultaneous evolution of at least two distinct species [8,18]. In [24], NEAT was applied in a competitive coevolution environment. In competitive coevolution, individuals of two or more species are competing between them. ...
... In competitive coevolution, individuals of two or more species are competing between them. Therefore, the fitness function represents the competition in order to represent a score that is inversely related between different species [18,22,24]. ...
Generative adversarial networks (GAN) present state-of-the-art results in the generation of samples following the distribution of the input dataset. However, GANs are difficult to train, and several aspects of the model should be previously designed by hand. Neuroevolution is a well-known technique used to provide the automatic design of network architectures which was recently expanded to deep neural networks.
COEGAN is a model that uses neuroevolution and coevolution in the GAN training algorithm to provide a more stable training method and the automatic design of neural network architectures. COEGAN makes use of the adversarial aspect of the GAN components to implement coevolutionary strategies in the training algorithm. Our proposal was evaluated in the Fashion-MNIST and MNIST dataset. We compare our results with a baseline based on DCGAN and also with results from a random search algorithm. We show that our method is able to discover efficient architectures in the Fashion-MNIST and MNIST datasets. The results also suggest that COEGAN can be used as a training algorithm for GANs to avoid common issues, such as the mode collapse problem.
... © 2021 Copyright held by the authors. and Adaptive Heuristic Critic for several RL problems like robotic control [17,26]. Compared to RL, a direct exploration in the search space is possible without the requirement for indirect inferences from value functions. ...
... In general, GAs can vary significantly. For example, the way a genotype is encoded can have a significant impact on the number of generations or result in no solution being found [26]. In addition, it is also significant how large the search space or how large the number of solutions is. ...
... As a search heuristic, GAs have some advantages compared to other search methods. While gradient descent methods can get trapped in the local minima in the error surface, GAs minimize this pitfall by sampling multiple points on the error surface [26]. In particular, feedback tends to cause unexpected effects in the error surface, so that local minima are more likely to occur at suboptimal points on the surface. ...
Machine Learning (ML) is becoming increasingly important in daily life. In this context, Artificial Neural Networks (ANNs) are a popular approach within ML methods to realize an artificial intelligence. Usually, the topology of ANNs is predetermined. However, there are problems where it is difficult to find a suitable topology. Therefore, Topology and Weight Evolving Artificial Neural Network (TWEANN) algorithms have been developed that can find ANN topologies and weights using genetic algorithms. A well-known downside for large-scale problems is that TWEANN algorithms often evolve inefficient ANNs and require long runtimes. To address this issue, we propose a new TWEANN algorithm called Artificial Life Form (ALF) with the following technical advancements: (1) speciation via structural and semantic similarity to form better candidate solutions, (2) dynamic adaptation of the observed candidate solutions for better convergence properties, and (3) integration of solution quality into genetic reproduction to increase the probability of optimization success. Experiments on large-scale ML problems confirm that these approaches allow the fast solving of these problems and lead to efficient evolved ANNs.
... NeuroEvolution of Augmenting Topologies (NEAT) [7] is a well-known neuroevolution method that evolves the weights and topologies of neural networks. In further experiments, NEAT was also successfully applied in a coevolution context [8]. Moreover, Deep-NEAT [9] was recently proposed to expand NEAT to larger search spaces, such as in deep neural networks. ...
... In competitive coevolution, individuals of different species are competing between them. Consequently, their fitness function directly represents this competition in a way that scores between species are inversely related [8,11,14]. ...
... We present in this paper a model that combines neuroevolution and coevolution in the coordination of the GAN training algorithm. To design the model, we took inspiration on previous evolutionary algorithms, such as NEAT [8] and DeepNeat [9], and on recent advances in GANs, such as [3]. ...
Generative adversarial networks (GAN) became a hot topic, presenting impressive results in the field of computer vision. However, there are still open problems with the GAN model, such as the training stability and the hand-design of architectures. Neuroevolution is a technique that can be used to provide the automatic design of network architectures even in large search spaces as in deep neural networks. Therefore, this project proposes COEGAN, a model that combines neuroevolution and coevolution in the coordination of the GAN training algorithm. The proposal uses the adversarial characteristic between the generator and discriminator components to design an algorithm using coevolution techniques. Our proposal was evaluated in the MNIST dataset. The results suggest the improvement of the training stability and the automatic discovery of efficient network architectures for GANs. Our model also partially solves the mode collapse problem.
... NeuroEvolution of Augmenting Topologies (NEAT) [7] is a well-known neuroevolution method that evolves the weights and topologies of neural networks. In further experiments, NEAT was also successfully applied in a coevolution context [8]. Moreover, Deep-NEAT [9] was recently proposed to expand NEAT to larger search spaces, such as in deep neural networks. ...
... In competitive coevolution, individuals of different species are competing between them. Consequently, their fitness function directly represents this competition in a way that scores between species are inversely related [8,11,14]. ...
... We present in this paper a model that combines neuroevolution and coevolution in the coordination of the GAN training algorithm. To design the model, we took inspiration on previous evolutionary algorithms, such as NEAT [8] and DeepNeat [9], and on recent advances in GANs, such as [3]. ...
Generative adversarial networks (GAN) became a hot topic, presenting impressive results in the field of computer vision. However, there are still open problems with the GAN model, such as the training stability and the hand-design of architectures. Neuroevolution is a technique that can be used to provide the automatic design of network architectures even in large search spaces as in deep neural networks. Therefore, this project proposes COEGAN, a model that combines neuroevolution and coevolution in the coordination of the GAN training algorithm. The proposal uses the adversarial characteristic between the generator and discriminator components to design an algorithm using coevolution techniques. Our proposal was evaluated in the MNIST dataset. The results suggest the improvement of the training stability and the automatic discovery of efficient network architectures for GANs. Our model also partially solves the mode collapse problem.
... NeuroEvolution of Augmenting Topologies (NEAT) [23] is a well-known algorithm that evolves the weights and topologies of neural networks. NEAT was also successfully applied in a coevolution context [24]. The NEAT model was also expanded to work on larger search spaces, such as deep neural networks, in the DeepNEAT [16] method. ...
... Coevolution is the simultaneous evolution of at least two distinct species [8,18]. In [24], NEAT was applied in a competitive coevolution environment. In competitive coevolution, individuals of two or more species are competing between them. ...
... In competitive coevolution, individuals of two or more species are competing between them. Therefore, the fitness function represents the competition in order to represent a score that is inversely related between different species [18,22,24]. ...
Generative adversarial networks (GAN) present state-of-the-art results in the generation of samples following the distribution of the input dataset. However, GANs are difficult to train, and several aspects of the model should be previously designed by hand. Neuroevolution is a well-known technique used to provide the automatic design of network architectures which was recently expanded to deep neural networks. COEGAN is a model that uses neuroevolution and coevolution in the GAN training algorithm to provide a more stable training method and the automatic design of neural network architectures. COEGAN makes use of the adversarial aspect of the GAN components to implement coevolutionary strategies in the training algorithm. Our proposal was evaluated in the Fashion-MNIST and MNIST dataset. We compare our results with a baseline based on DCGAN and also with results from a random search algorithm. We show that our method is able to discover efficient architectures in the Fashion-MNIST and MNIST datasets. The results also suggest that COEGAN can be used as a training algorithm for GANs to avoid common issues, such as the mode collapse problem.
... Herbivory has thoroughly determined ecosystem functioning and services, widely shaped biodiversity and favored complexification, since the actual diversity of life is the result not only from the diversification of species but also from the diversification of interactions among them (Huntly 1991, Stanley and Miikkulainen 2004, Thompson 2005 Agrawal 2009, Leimu et al. 2012, Valiente-Banuet et al. 2015, Guimarães et al. 2017, Levine et al. 2017. Insects have been recognized to be the most significant herbivores (Lawton 1983, Crawley 1989, Jaenike 1990, usually triggering an ongoing process of coevolution or reciprocal adaptations with plants (Ehrlich and Raven 1964, Janzen 1980, Gatehouse 2002, Strauss et al. 2004b, Futuyma and Agrawal 2009, Leimu et al. 2012. ...
... Complexification incremental elaboration of solutions through adding new structure (Stanley and Miikkulainen 2004) Selection relationship between a trait and fitness (Lande and Arnold 1983) Phenology timing of events in the life of an organism (Lieth 1974) Ontogeny developmental history of an organism within its own lifetime (Gould 1977) Phenotype the observable characteristics or traits of an organism (revisited by Dawkins ...
During the last decades, important advances have been made in the multifocal study of herbivory, although given its intrinsic complexity, many questions remain to be resolved. Even today we have a somewhat biased knowledge towards simpler systems such as those of crop plants, with few systems in which the set of natural herbivores of a given plant has been jointly studied with realistic consumption rates, and even fewer that have considered simultaneously its resistance and tolerance towards its community of herbivores. This is necessary for the advancement of this broad field and all its ramifications. In the present thesis we experimentally evaluated the interaction between the wild semi-arid herb Moricandia moricandioides (Brassicaceae) and various combinations of its main herbivores (ungulates and various pre-dispersal seed predator, florivore and root herbivore insects), the defensive response of the plant and the context in which both the damage inflicted on the plant and the ability of the plant to cope with its herbivores may vary. In addition, we delve into novel aspects such as transgenerational effects, herbivore-induced subindividual variation, and the potential of climate change to modulate plant-herbivore interactions.
... Therefore, a competitive model can be suitable to represent populations of individuals in GANs. In EAs, coevolution is the simultaneous evolution of at least two distinct species [19,35,42]. In competitive coevolution, individuals of these species are competing together, and their fitness function directly represents this competition. ...
... Here, their fitness function directly represents this competition in a way that scores between species are inversely related. For example, NEAT was successfully applied to a competitive coevolution environment [42]. ...
Generative Adversarial Networks (GAN) is an adversarial model that became relevant in the last years, displaying impressive results in generative tasks. A GAN combines two neural networks, a discriminator and a generator, trained in an adversarial way. The discriminator learns to distinguish between real samples of an input dataset and fake samples. The generator creates fake samples aiming to fool the discriminator. The training progresses iteratively, leading to the production of realistic samples that can mislead the discriminator. Despite the impressive results, GANs are hard to train, and a trial-and-error approach is generally used to obtain consistent results. Since the original GAN proposal, research has been conducted not only to improve the quality of the generated results but also to overcome the training issues and provide a robust training process. However, even with the advances in the GAN model, stability issues are still present in the training of GANs. Neuroevolution, the application of evolutionary algorithms in neural networks, was recently proposed as a strategy to train and evolve GANs. These proposals use the evolutionary pressure to guide the training of GANs to build robust models, leveraging the quality of results, and providing a more stable training. Furthermore, these proposals can automatically provide useful architectural definitions, avoiding the manual discovery of suitable models for GANs. We show the current advances in the use of evolutionary algorithms and GANs, presenting the state-of-the-art proposals related to this context. Finally, we discuss perspectives and possible directions for further advances in the use of evolutionary algorithms and GANs.
... COEGAN is inspired by DeepNEAT [17] to design the model, also using coevolution techniques presented in NEAT applied to competitive coevolution [26]. The genome of COEGAN is represented by a sequential array of genes. ...
Generative Adversarial Networks (GANs) are an adversarial model that achieved impressive results on generative tasks. In spite of the relevant results, GANs present some challenges regarding stability, making the training usually a hit-and-miss process. To overcome these challenges, several improvements were proposed to better handle the internal characteristics of the model, such as alternative loss functions or architectural changes on the neural networks used by the generator and the discriminator. Recent works proposed the use of evolutionary algorithms on GAN training, aiming to solve these challenges and to provide an automatic way to find good models. In this context, COEGAN proposes the use of coevolution and neuroevolution to orchestrate the training of GANs. However, previous experiments detected that some of the fitness functions used to guide the evolution are not ideal. In this work we propose the evaluation of a game-based fitness function to be used within the COEGAN method. Skill rating is a metric to quantify the skill of players in a game and has already been used to evaluate GANs. We extend this idea using the skill rating in an evolutionary algorithm to train GANs. The results show that skill rating can be used as fitness to guide the evolution in COEGAN without the dependence of an external evaluator.
... The new population is then randomly mutated by any of: Perturbation of weights, replacement of weights, addition of a new node, addition of a new connection, disabling a connection, intraspecies crossover, or interspecies crossover. NEAT has been applied to many problems, including the pole balancing problem [21], computer games [14,20], and robot control [22]. ...
Neuromodulation is a biologically-inspired technique that can adapt the per-connection learning rates of synaptic plasticity. Neuromodulation has been used to facilitate unsupervised learning by adapting neural network weights. Multiobjective evolution of neural network topology and weights has been used to design neurocontrollers for autonomous robots. This paper presents a novel multiobjective evolutionary neurocontroller with unsupervised learning for robot navigation. Multiobjective evolution of network weights and topologies (NEAT-MODS) is augmented with neuromodulated learning. NEAT-MODS is an NSGA-II based multiobjective neurocontroller that uses two conflicting objectives. The first rewards the robot when it moves in a direct manner with minimal turning; the second objective is to reach as many targets as possible. NEAT-MODS uses speciation, a selection process that aims to ensure Pareto-optimal genotypic diversity and elitism. The effectiveness of the design is demonstrated using a series of experiments with a simulated robot traversing a simple maze containing target goals. It is shown that when neuromodulated learning is combined with multiobjective evolution, better-performing neural controllers are synthesized than by evolution alone. Secondly, it is demonstrated that speciation is unnecessary in neuromodulated neuroevolution, as neuromodulation preserves topological innovation. The proposed neuromodulated approach is found to be statistically superior to NEAT-MODS alone when applied to solve a multiobjective navigation problem.
... This is accomplished by gradually increasing the size and complexity of the strategies available to both players. Using techniques like competitive fitness sharing, shared sampling, and "hall of fame" will result in the strategies of both players converging to a globally optimal Nash Equilibrium (NE) as the strategies coevolve [1][2][3][4]. ...
... Competitive co-evolutionary algorithms (CCEAs) can be seen as a superset of self-play, as rather than keeping only a solution and its predecessors, it is instead possible to keep and evaluate against an entire population of solutions. Like self-play, CEAs form a natural curriculum [7], but also confer an additional robustness as solutions are evaluated against a varied set of other solutions [15,18]. ...
In January 2019, DeepMind revealed AlphaStar to the world---the first artificial intelligence (AI) system to beat a professional player at the game of StarCraft II---representing a milestone in the progress of AI. AlphaStar draws on many areas of AI research, including deep learning, reinforcement learning, game theory, and evolutionary computation (EC). In this paper we analyze AlphaStar primarily through the lens of EC, presenting a new look at the system and relating it to many concepts in the field. We highlight some of its most interesting aspects---the use of Lamarckian evolution, competitive co-evolution, and quality diversity. In doing so, we hope to provide a bridge between the wider EC community and one of the most significant AI systems developed in recent times.
... A deep philosophical question exists in this realm: What are the necessary and sufficient characteristics required for such open-ended learning and environmental modifications (as opposed to saturation of learning and emergence over time, as seen in the non-living world)? This is explored in the field of "open-ended evolution" e.g., [54][55][56][57][58][59][60][61][62]. ...
We are embarking on a new age of astrobiology, one in which numerous interplanetary missions and telescopes will be designed, built, and launched with the explicit goal of finding evidence for life beyond Earth. Such a profound aim warrants caution and responsibility when interpreting and disseminating results. Scientists must take care not to overstate (or over-imply) confidence in life detection when evidence is lacking, or only incremental advances have been made. Recently, there has been a call for the community to create standards of evidence for the detection and reporting of biosignatures. In this perspective, we wish to highlight a critical but often understated element to the discussion of biosignatures: Life detection studies are deeply entwined with and rely upon our (often preconceived) notions of what life is, the origins of life, and habitability. Where biosignatures are concerned, these three highly related questions are frequently relegated to a low priority, assumed to be already solved or irrelevant to the question of life detection. Therefore, our aim is to bring to the fore how these other major astrobiological frontiers are central to searching for life elsewhere and encourage astrobiologists to embrace the reality that all of these science questions are interrelated and must be furthered together rather than separately. Finally, in an effort to be more inclusive of life as we do not know it, we propose tentative criteria for a more general and expansive characterization of habitability that we call genesity.
... This approach is limited, as the topology of the network, i.e. the size and structure of the network is not changed. Evolving the topology of the network can achieve better performance than evolving weights only (Harvey et al. 1997, Stanley andMiikkulainen, 2004). A third method involves evolving the individual components of a single network, rather than creating and evolving a population of networks. ...
The purpose of this study was to investigate the use and potential of evolutionary artificial neural networks as a tool for economic analysis, modelling and forecasting; and to determine how effective existing techniques for technology management are in managing innovation driven by evolutionary techniques.
To study the ability of evolutionary neural networks to perform economic predictions, a case study was undertaken. In this, a population of neural networks was created and trained and using the evolutionary principles of reproduction, mutation and selection the population was evolved through several generations to produce a network that performed significantly better than any member of the original population. The network was trained using historical time-series economic data on the UK co-operative sector, with the purpose of predicting those organisations that were at high risk of financial difficulty or failure.
A quantitative analysis of the output of the network then revealed how it was able to identify those organisations that later went on to experience financial troubles with an accuracy rate of 51.45%. This compared to an accuracy rate of 35.60% from the original (non-evolved) first population of networks, supporting the hypothesis that evolutionary AI techniques can be used to improve the accuracy of economic analysis over non-evolutionary AI techniques, albeit only in a single study.
Noting that technology innovation via evolution mechanisms is a significant departure from traditionally understood innovation methods, the study then goes on to examine how applicable existing technology management techniques are in this context. To do this two techniques from the field of innovation management are applied: firstly, the concept of degrees of innovation looks at how significant an innovation evolutionary artificial neural networks are/may be. This is combined with a second technique, technology road mapping, to estimate and project how the technology might evolve, and the barriers and timescales involved.
The roadmap showed how the issue of computing power had previously occurred, and paused most research into the technology during the 1970’s and early 1980’s. It also showed the development of other technologies which may be influential on evolutionary neurocomputing in future, such as quantum computing and physical neural networks.
... In reinforcement learning, this has often taken the form of adaptively growing the resolution of the state space considered by a piecewise constant discretised approximation [Moore, 1994, Munos and Moore, 2002, Whiteson et al., 2007. Stanley and Miikkulainen [2004] study continual complexification in the context of coevolution, growing the complexity of neural network architectures through the course of training. These works progressively increase the capabilities of the agent, but not with respect to its available actions. ...
In complex tasks, such as those with large combinatorial action spaces, random exploration may be too inefficient to achieve meaningful learning progress. In this work, we use a curriculum of progressively growing action spaces to accelerate learning. We assume the environment is out of our control, but that the agent may set an internal curriculum by initially restricting its action space. Our approach uses off-policy reinforcement learning to estimate optimal value functions for multiple action spaces simultaneously and efficiently transfers data, value estimates, and state representations from restricted action spaces to the full task. We show the efficacy of our approach in proof-of-concept control tasks and on challenging large-scale StarCraft micromanagement tasks with large, multi-agent action spaces.
... This neuroevolution method starts evolution using networks without any hidden nodes and subsequently adds neurons and connections by carefully designed mutation operators. It has been shown that complexification during evolution does indeed occur when using NEAT, and can create neural networks with in the order of up to a few dozens of neurons (Stanley and Miikkulainen, 2004). NEAT has subsequently been widely used for various evolutionary robotics experiments. ...
Many experiments have been performed that use evolutionary algorithms for learning the topology and connection weights of a neural network that controls a robot or virtual agent. These experiments are not only performed to better understand basic biological principles, but also with the hope that with further progress of the methods, they will become competitive for automatically creating robot behaviors of interest. However, current methods are limited with respect to the (Kolmogorov) complexity of evolved behavior. Using the evolution of robot trajectories as an example, we show that by adding four features, namely (1) freezing of previously evolved structure, (2) temporal scaffolding, (3) a homogeneous transfer function for output nodes, and (4) mutations that create new pathways to outputs, to standard methods for the evolution of neural networks, we can achieve an approximately linear growth of the complexity of behavior over thousands of generations. Overall, evolved complexity is up to two orders of magnitude over that achieved by standard methods in the experiments reported here, with the major limiting factor for further growth being the available run time. Thus, the set of methods proposed here promises to be a useful addition to various current neuroevolution methods.
... Concernant les CoEA coopératives, les individus des populations sont récompensés quand ils fonctionnent bien ensemble et pénalisés quand ils échouent ensemble [88]. Inversement pour les CoEA concurrentielles, les populations sont mises en compétition pour résoudre le problème à optimiser et les individus sont récompensés contrairement à ceux avec lesquels ils intéragissent qui sont pénalisés [105]. ...
Les problèmes d'optimisation continue sont nombreux, en économie, en traitement de signal, en réseaux de neurones, etc. L'une des solutions les plus connues et les plus employées est l'algorithme évolutionnaire, métaheuristique basée sur les théories de l'évolution qui emprunte des mécanismes stochastiques et qui a surtout montré de bonnes performances dans la résolution des problèmes d'optimisation continue. L’utilisation de cette famille d'algorithmes est très populaire, malgré les nombreuses difficultés qui peuvent être rencontrées lors de leur conception. En effet, ces algorithmes ont plusieurs paramètres à régler et plusieurs opérateurs à fixer en fonction des problèmes à résoudre. Dans la littérature, on trouve pléthore d'opérateurs décrits, et il devient compliqué pour l'utilisateur de savoir lesquels sélectionner afin d'avoir le meilleur résultat possible. Dans ce contexte, cette thèse avait pour objectif principal de proposer des méthodes permettant de remédier à ces problèmes sans pour autant détériorer les performances de ces algorithmes. Ainsi nous proposons deux algorithmes :- une méthode basée sur le maximum a posteriori qui utilise les probabilités de diversité afin de sélectionner les opérateurs à appliquer, et qui remet ce choix régulièrement en jeu,- une méthode basée sur un graphe dynamique d'opérateurs représentant les probabilités de passages entre les opérateurs, et en s'appuyant sur un modèle de la fonction objectif construit par un réseau de neurones pour mettre régulièrement à jour ces probabilités. Ces deux méthodes sont détaillées, ainsi qu'analysées via un benchmark d'optimisation continue
... Each seed of a specific robotic unit describes its basic sensorimotor capabilities/configuration in terms of the topology of the Artificial Neural Network (ANN) controlling it. The neuroevolution [11], [12] process will be applied to the mentioned seed configurations to create specialized ANN modules [13], depending on the sensorimotor dynamics of the system and the observed environmental characteristics. ...
In this paper, we look at how Artificial Swarm Intelligence can evolve using evolutionary algorithms that try to minimize the sensory surprise of the system. We will show how to apply the free-energy principle, borrowed from statistical physics, to quantitatively describe the optimization method (sensory surprise minimization), which can be used to support lifelong learning. We provide our ideas about how to combine this optimization method with evolutionary algorithms in order to boost the development of specialized Artificial Neural Networks, which define the proprioceptive configuration of particular robotic units that are part of a swarm. We consider how optimization of the free-energy can promote the homeostasis of the swarm system, i.e. ensures that the system remains within its sensory boundaries throughout its active lifetime. We will show how complex distributed cognitive systems can be build in the form of hierarchical modular system, which consists of specialized micro-intelligent agents connected through information channels. We will also consider the co-evolution of various robotic swarm units, which can result in development of proprioception and a comprehensive awareness of the properties of the environment. And finally, we will give a brief outline of how this system can be implemented in practice and of our progress in this area.
... Competitive co-evolutionary algorithms (CCEAs) can be seen as a superset of self-play, as rather than keeping only a solution and its predecessors, it is instead possible to keep and evaluate against an entire population of solutions. Like self-play, CEAs form a natural curriculum [8], but also confer an additional robustness as solutions are evaluated against a varied set of other solutions [16,19]. ...
In January 2019, DeepMind revealed AlphaStar to the world-the first artificial intelligence (AI) system to beat a professional player at the game of StarCraft II-representing a milestone in the progress of AI. AlphaStar draws on many areas of AI research, including deep learning, reinforcement learning, game theory, and evolutionary computation (EC). In this paper we analyze AlphaStar primarily through the lens of EC, presenting a new look at the system and relating it to many concepts in the field. We highlight some of its most interesting aspects-the use of Lamarckian evolution, competitive co-evolution, and quality diversity. In doing so, we hope to provide a bridge between the wider EC community and one of the most significant AI systems developed in recent times.
... Each seed of a specific robotic unit describes its basic sensorimotor capabilities/configuration in terms of the topology of the Artificial Neural Network (ANN) controlling it. The neuroevolution [11], [12] process will be applied to the mentioned seed configurations to create specialized ANN modules [13], depending on the sensorimotor dynamics of the system and the observed environmental characteristics. ...
In this paper, we look at how Artificial Swarm Intelligence can evolve using evolutionary algorithms that try to minimize the sensory surprise of the system. We will show how to apply the free-energy principle, borrowed from statistical physics, to quantitatively describe the optimization method (sensory surprise minimization), which can be used to support lifelong learning. We provide our ideas about how to combine this optimization method with evolutionary algorithms in order to boost the development of specialized Artificial Neural Networks , which define the proprioceptive configuration of particular robotic units that are part of a swarm. We consider how optimization of the free-energy can promote the homeostasis of the swarm system, i.e. ensures that the system remains within its sensory boundaries throughout its active lifetime. We will show how complex distributed cognitive systems can be build in the form of hierarchical modular system, which consists of specialized micro-intelligent agents connected through information channels. We will also consider the co-evolution of various robotic swarm units, which can result in development of proprioception and a comprehensive awareness of the properties of the environment. And finally, we will give a brief outline of how this system can be implemented in practice and of our progress in this area.
... For example, Hausknecht et al. [8,9] show that Neuroevolution has the potential to learn how to play Atari games directly from visual input. On their work, they develop several Neuroevolution architectures mainly using NEAT and HyperNEAT [18][19][20] that learn to play different Atari games from scratch. These learning algorithms require time and generations to adjust and learn. ...
Estimating the difficulty of a learning activity is crucial for smart learning systems to provide learners with most suitable activities for their abilities. Generally, difficulty is estimated by teachers according to their experience and based on students' results. For newly designed activities, this estimation is often inaccurate or complicated to obtain. Moreover, machine learning methods seem impossible to use on absence of data. This work proposes an innovative way to use machine learning in this scenario. We start hypothesizing that the effort a student has to make to perform an activity is correlated with the effort that a machine learning algorithm would require. We define the concept of difficulty in a formal way, implement two Neuroevolution algorithms that solve a specific type of activity, and then calculate and compare efforts for students and implemented algorithms. Results show that the correlation exists for the selected activity. Therefore, the effort of the algorithm can be used to estimate the difficulty of this learning activity.
... e AlexNet [13,14] model proposed by Alex for the improvement of convolutional neural network structure has exploded the application of neural networks. e author uses ReLU [15] activation function to replace sigmoid [16] and the dropout [17] technology to avoid overfitting. In addition, the maximum pooling method is applied to the lower sampling layer to build a deep convolutional neural network and win the champion of the image recognition contest, making CNN become the core algorithm model of image recognition and making the convolutional neural network development in a deeper direction. ...
With the development of industry and the progress of science and technology, more and more new technology is gradually applied to the movement of error correction. This not only relieves workers of unneeded burdens by making their task more straightforward and error-free, but it also improves production efficiency. Deep neural networks are one of these new technologies that have exploded in popularity in recent years, with applications in a variety of industries. Of course, the application in image recognition must not be less, image recognition technology based on deep neural network has become more mature, and the error rate of recognition is now much lower than human vision recognition. So at present, some industrial detection is gradually from human vision detection to computer vision detection. This study discloses a basketball action error correction method based on deep learning image recognition, which includes the following steps: receiving each frame of basketball image captured from the fitness video, recording the corresponding time of each frame of fitness image, and preprocessing each frame of fitness image; the preprocessed basketball image was fed into the human joint recognition model, and the human joint recognition model calculated each human joint in the fitness image and output its position coordinates. According to the coordinate position of each joint orderly line, the human skeleton diagram is obtained; the human skeleton diagram is compared and assessed in accordance with standard fitness action, and the nonstandard basketball image is generated to realize basketball action repair. A basketball action error correction system based on deep learning picture identification is also disclosed in the invention. The system and method are capable of efficiently addressing the difficult challenge of comparing fitness movements with and without music rhythm.
... Promising studies originate from the identification of synergies between evolutionary algorithms and artificial neural networks throughout neuroevolution [35,47]. Evolutionary computation is used to search for neural network hyper-parameters [30], topologies [36,56], or as replacement or hybridization of the neural learning algorithm [14,48,50] to achieve a good enough network performance in a given task. Evolutionary algorithms are inspired by natural evolution and adaptation of species to achieve the prevalence or subsistence of a population of individuals. ...
This article presents an evolutionary algorithm to autonomously construct full- connected multilayered feedforward neural architectures. This algorithm employs grammar-guided genetic programming with a context-free grammar that has been specifically designed to satisfy three important restrictions. First, the sentences that belong to the language produced by the grammar only encode all valid neural architectures. Second, full-connected feedforward neural architectures of any size can be generated. Third, smaller-sized neural architectures are favored to avoid overfitting. The proposed evolutionary neural architectures construction system is applied to calculate the terms of the two sequences that define the three-term recurrence relation associated with a sequence of orthogonal polynomials. This application imposes an important constraint: training datasets are always very small. Therefore, an adequate sized neural architecture has to be evolved to achieve satisfactory results, which are presented in terms of accuracy and size of the evolved neural architectures, and convergence speed of the evolutionary process.
... Competitive games have been used as benchmarks to evaluate the ability of algorithms to train an agent to make rational and strategic decisions [19,20,21]. Multi-agent reinforcement learning methods allow agents to learn emergent and complex behavior by interacting with each other and co-evolving together [22,23,24,25,26,27]. Many recent efforts have used multi-agent RL methods to learn continuous control polices that can achieve high complexity tasks. ...
This project aims to motivate research in competitive human-robot interaction by creating a robot competitor that can challenge human users in certain scenarios such as physical exercise and games. With this goal in mind, we introduce the Fencing Game, a human-robot competition used to evaluate both the capabilities of the robot competitor and user experience. We develop the robot competitor through iterative multi-agent reinforcement learning and show that it can perform well against human competitors. Our user study additionally found that our system was able to continuously create challenging and enjoyable interactions that significantly increased human subjects' heart rates. The majority of human subjects considered the system to be entertaining and desirable for improving the quality of their exercise.
... If the evolved structure is beneficial it is retained and utilized further to augment the network. NEAT has been used in automobile crash warning system (Stanley et al., 2005b), pole balancing (Stanley and Miikkulainen, 2002), computer games (Stanley et al., 2005a, Reisinger et al., 2007 and robot control (Stanley and Miikkulainen, 2004). ...
Concepts are of great value to humans because they are one of the building blocks of our
cognitive processes. They are involved in cognitive functions that are fundamental in decision
making such as classification and also capacitate us for contextual comprehension. By definition,
a concept refers to an idea or a combination of several ideas. In a computational context, a
concept can be a feature or a set of features. An individual concept is referred to as a concrete
concept, whereas a generalized form of a set of concepts can be perceived as an abstract concept.
Computational concepts can be characterized in three broad categories; i.e. symbolic (e.g.
Adaptive Control of Thought based approach), distributed (e.g. Neural Networks) and spatial
(e.g. Conceptual Space) representations. CLARION, a cognitive architecture, is an example
of a hybrid computational framework that combines symbolic and distributed representations.
Moreover, the symbolic, distributed, spatial and hybrid representations are mostly used on
representing concrete concepts, whereas the notion of an abstract concept is rarely explored.
In this thesis, we propose a computational cognitive model, named Regulated Activation Net-
work (RAN), capable of dynamically forming the abstract representations of concepts and to
unify the qualities of spatial, symbolic and distributed computational approaches. Our model
aims to simulate the cognitive processes of concept learning, creation and recall. In particular,
the RAN’s modeling has three learning mechanisms where two perform inter-layer learning that
helps in propagating activations from an input-to-output layer and vice versa. The third pro-
vides an intra-layer learning that is used to emulate regulation mechanism, which is inspired by
biological Axoaxonic synapse where one node in a layer induces excitatory, neutral or inhibitory
activation to other nodes in the layer. In this research, two different types of abstract con-
cepts are modeled: first, the convex abstract concepts where the geometrical convexity among
the concrete concepts was exploited to create the abstract concept; second, the non-convex ab-
stract concepts where the similarity relationships among the convex abstract concepts were used
to capture non-convexity and model it. The RAN uniquely unifies the qualities of symbolic,
distributed and spatial conceptual representation, where the model has a dynamic topology,
simulates cognitive process like learning and concept creation and performs machine learning
operations.
Experiments with 11 benchmarks demonstrated the classification capability of RAN’s modeling
and provided a proof-of-concept of convex and non-convex abstract concept modeling. In these
experiments, the study has shown that RAN performed satisfactorily when compared with five
different classifiers. One of the datasets was used to model the active and inactive states of
three students. Further, the results of this model of students were analyzed statistically to
infer students’ psychological and physiological conditions. The recall experiments with RAN
demonstrated the cued recall blend retrieval of abstract concepts. Besides cognitive function
simulation and machine learning, the RAN’s model was also useful in the data analysis task.
In one of the experiments, a RAN’s model was developed to have 7 layers showing dimension
reduction and expansion operations. Additionally, the data visualization of the 1st, 3rd, and
5th layers displayed how deep data analysis with the RAN model unearth the complexities in
the data.
The research work involved the study of topics from the fields of Mathematics, Computational
Modeling, Psychology, Cognition, and Neurology. Based upon the results of all the experiments
and analogical reasoning of RAN’s modeling processes, the hypotheses of the research work
were demonstrated. The abstract concept modeling was substantiated through classification
experiments, whereas the simulations of concept creation, learning, activation propagation, and
recall were justified through analogy and empirical outcomes. The research work also helped in
discovering new challenges, such as temporal learning and simulation of the cognitive process
of forgetting, which will be taken as research projects in the future.
... Competitive games like Chess, Checkers, and Go have been studied extensively by the AI researchers for the purpose of benchmarking the ability of algorithms to train an agent to learn, reason, and plan [3,23,27]. Many algorithms that solve multi-agent Markov games are designed under a multi-agent reinforcement learning scheme, where agents develop emergent and complex behavior through interacting with each other and co-evolving together [8,9,11,26,28,30]. The following works have successfully extended game RL to learn complex motor skills. ...
... Previous work [26][23] [24][21][2] [22] indicates that the use of crossover in NEAT does not improve the algorithm's performance. Therefore, we deemed the use of crossover unnecessary and possibly counterproductive for this experiment. ...
A network-based modelling technique, search trajectory networks (STNs), has recently helped to understand the dynamics of neuroevolution algorithms such as NEAT. Modelling and visualising variants of NEAT made it possible to analyse the dynamics of search operators. Thus far, this analysis was applied directly to the NEAT genotype space composed of neural network topologies and weights. Here, we extend this work, by illuminating instead the behavioural space, which is available when the evolved neural networks control the behaviour of agents. Recent interest in behaviour characterisation highlights the need for divergent search strategies. Quality-diversity and Novelty search are examples of divergent search, but their dynamics are not yet well understood. In this article, we examine the idiosyncrasies of three neuroevolution variants: novelty, random and objective search operating as usual on the genotypic search space, but analysed in the behavioural space. Results show that novelty is a successful divergent search strategy. However, its abilities to produce diverse solutions are not always consistent. Our visual analysis highlights interesting relationships between topological complexity and behavioural diversity which may pave the way for new characterisations and search strategies.
... The inclusion of historical markings in NEAT allowed the crossover operator to create valid offspring by identifying regions of each genome that were compatible. Each of these components was examined in a series of ablation experiments in Stanley and Miikkulaine [32] and each was determined to not only increase the performance of NEAT, but are independent and necessary for its application [33]. It was noted, however, that the nonmating NEAT, that is, NEAT without crossover, converged on the target fitness threshold significantly faster than the other ablation studies. ...
Neuroevolution has re-emerged as an active topic in the last few years. However, there is a lack of accessible tools to analyse, contrast and visualise the behaviour of neuroevolution systems. A variety of search strategies have been proposed such as Novelty search and Quality-Diversity search, but their impact on the evolutionary dynamics is not well understood. We propose using a data-driven, graph-based model, search trajectory networks (STNs) to analyse, visualise and directly contrast the behaviour of different neuroevolution search methods. Our analysis uses NEAT for solving maze problems with two search strategies: novelty-based and fitness-based, and including and excluding the crossover operator. We model and visualise the trajectories, contrasting and illuminating the behaviour of the studied neuroevolution variants. Our results confirm the advantages of novelty search in this setting, but challenge the usefulness of recombination.
... Competitive games have been used as benchmarks to evaluate the ability of algorithms to train an agent to make rational and strategic decisions [19,20,21]. Multi-agent reinforcement learning methods allow agents to learn emergent and complex behavior by interacting with each other and co-evolving together [22,23,24,25,26,27]. Many recent efforts have used multi-agent RL methods to learn continuous control polices that can achieve high complexity tasks. ...
This project aims to motivate research in competitive human-robot interaction by creating a robot competitor that can challenge human users in certain scenarios such as physical exercise and games. With this goal in mind, we introduce the Fencing Game, a human-robot competition used to evaluate both the capabilities of the robot competitor and user experience. We develop the robot competitor through iterative multi-agent reinforcement learning and show that it can perform well against human competitors. Our user study additionally found that our system was able to continuously create challenging and enjoyable interactions that significantly increased human subjects' heart rates. The majority of human subjects considered the system to be entertaining and desirable for improving the quality of their exercise.
... In RL, where the model learns by trial and error to try different action sequences to maximize the reward [17], the ability of NE to evolve the neural network topology along with weights has made them outperform baseline RL benchmark tasks [18,19]. Along with this, NE has successfully outperformed many tasks such as pole balancing [14], evolving neural network controllers for robots [20,21], video games [22,23], and an automobile crash warning system [24]. It is worth mentioning that evolutionary algorithms (EA) [25] have been gaining popularity for their ability to alleviate the shortcoming of BP [26]. ...
Artificial neural networks (ANN) and multilayer perceptrons (MLP) have proved to be efficient in terms of designing highly accurate semiconductor device compact models (CM). Their ability to update their weight and biases through the backpropagation method makes them highly useful in learning the task. To improve the learning, MLP usually requires large networks and thus a large number of model parameters, which significantly increases the simulation time in circuit simulation. Hence, optimizing the network architecture and topology is always a tedious yet important task. In this work, we tune the network topology using neuro-evolution (NE) to develop semiconductor device CMs. With input and output layers defined, we have allowed a genetic algorithm (GA), a gradient-free algorithm, to tune the network architecture in combination with Adam, a gradient-based backpropagation algorithm, for the network weight and bias optimization. In addition, we implemented the MLP model using a similar number of parameters as the baseline for comparison. It is observed that in most of the cases, the NE models exhibit a lower root mean square error (RMSE) and require fewer training epochs compared to the MLP baseline models. For instance, for patience number 10 with different number of model parameters, the RMSE for test dataset using NE and MLP are 3.
... A number of highly effective evolutionary-computation techniques owe their success to ecological dynamics. These largely fall into four categories: niching/speciation (e.g., Goldberg et al., 1987;Stanley and Miikkulainen, 2004), parent selection (e.g., De Jong, 1975;Mahfoud, 1992), dividing the population into subpopulations (e.g., Hu et al., 2005;Hornby, 2006), and adjusting the objective function to favor diversity and/or novelty (e.g., Mouret and Doncieux, 2009). In particular, fitness sharing and Eco-EA, both of which fall into the niching/speciation category, show promise as model systems for studying ecology. ...
In digital evolution, populations of computational organisms evolve via the same principles that govern natural selection in nature. These platforms have been used to great effect as a controlled system in which to conduct evolutionary experiments and develop novel evolutionary theory. In addition to their complex evolutionary dynamics, many digital evolution systems also produce rich ecological communities. As a result, digital evolution is also a powerful tool for research on eco-evolutionary dynamics. Here, we review the research to date in which digital evolution platforms have been used to address eco-evolutionary (and in some cases purely ecological) questions. This work has spanned a wide range of topics, including competition, facilitation, parasitism, predation, and macroecological scaling laws. We argue for the value of further ecological research in digital evolution systems and present some particularly promising directions for further research.
We investigate the application of a version of Genetic Programming with grammars, called Grammatical Evolution, and a multi-population competitive coevolutionary algorithm for anticipating tax evasion in the domain of U.S. Partnership tax regulations. A problem in tax auditing is that as soon as one evasion scheme is detected a new, slightly mutated, variant of that scheme appears. Multi-population competitive coevolutionary algorithms are disposed to explore adversarial problems, such as the arms-race between tax evader and auditor. In addition, we use Genetic Programming and grammars to represent and search the transactions of tax evaders and tax audit policies. Grammars are helpful for representing and biasing the search space. The feasibility of the method is studied with an example of adversarial coevolution in tax evasion. We study the dynamics and the solutions of the competing populations in this scenario, and note that we are able to replicate some of the expected behavior.
Despite the neuroevolution of augmenting topologies method strengths, like the capability to be used in cases where the formula for a cost function and the topology of the neural network are difficult to determine, one of the main problems of such methods is slow convergence towards optimal results, especially in cases with complex and challenging environments. This paper proposes the novel distributed implementation of neuroevolution of augmenting topologies method, which considering availability of sufficient computational resources allows drastically speed up the process of optimal neural network configuration search. Batch genome evaluation was implemented for the means of proposed solution performance optimization, fair, and even computational resources usage. The proposed distributed implementation benchmarking shows that the generated neural networks evaluation process gives a manifold increase of efficiency on the demonstrated task and computational environment.
Many experiments have been performed that use evolutionary algorithms for learning the topology and connection weights of a neural network that controls a robot or virtual agent. These experiments are not only performed to better understand basic biological principles, but also with the hope that with further progress of the methods, they will become competitive for automatically creating robot behaviors of interest. However, current methods are limited with respect to the (Kolmogorov) complexity of evolved behavior. Using the evolution of robot trajectories as an example, we show that by adding four features, namely (1) freezing of previously evolved structure, (2) temporal scaffolding, (3) a homogeneous transfer function for output nodes, and (4) mutations that create new pathways to outputs, to standard methods for the evolution of neural networks, we can achieve an approximately linear growth of the complexity of behavior over thousands of generations. Overall, evolved complexity is up to two orders of magnitude over that achieved by standard methods in the experiments reported here, with the major limiting factor for further growth being the available run time. Thus, the set of methods proposed here promises to be a useful addition to various current neuroevolution methods.
Artificial neural networks (ANN) and multilayer perceptrons (MLP) have proved to be efficient in terms of designing highly accurate semiconductor device compact models (CM). Their ability to update their weight and biases through the backpropagation method makes them highly useful in learning the task. To improve the learning, MLP usually requires large networks and thus a large number of model parameters, which significantly increases the simulation time in circuit simulation. Hence, optimizing the network architecture and topology is always a tedious yet important task. In this work, we tune the network topology using neuro-evolution (NE) to develop semiconductor device CMs. With input and output layers defined, we have allowed a genetic algorithm (GA), a gradient-free algorithm, to tune the network architecture in combination with Adam, a gradient-based backpropagation algorithm, for the network weight and bias optimization. In addition, we implemented the MLP model using a similar number of parameters as the baseline for comparison. It is observed that in most of the cases, the NE models exhibit a lower root mean square error (RMSE) and require fewer training epochs compared to the MLP baseline models. For instance, for patience number 10 with different number of model parameters, the RMSE for test dataset using NE and MLP in unit of log(ampere) are 0.1461, 0.0985, 0.1274, 0.0971, 0.0705, and 0.2254, 0.1423, 0.1429, 0.1425, 0.1391, respectively, for the 28nm technology node at foundry.
NeuroEvolution (NE) refers to a family of methods for optimizing Artificial Neural Networks (ANNs) using Evolutionary Computation (EC) algorithms. NeuroEvolution of Augmenting Topologies (NEAT) is considered one of the most influential algorithms in the field. Eighteen years after its invention, a plethora of methods have been proposed that extend NEAT in different aspects. In this paper we present a systematic literature review (SLR) to list and categorize the methods succeeding NEAT. Our review protocol identified 232 papers by merging the findings of two major electronic databases. Applying criteria that determine the paper's relevance and assess its quality, resulted in 61 methods that are presented in this paper. Our review paper proposes a new categorization scheme of NEAT's successors into three clusters. NEATbased methods are categorized based on 1) whether they consider issues specific to the search space or the fitness landscape, 2) whether they combine principles from NE and another domain or 3) based on the particular properties of the evolved ANNs. The clustering supports researchers 1) understanding the current state of the art that will enable them 2) exploring new research directions or 3) benchmarking their proposed method to the state of the art, if they are interested in comparing, and 4) positioning themselves in the domain or 5) selecting a method that is most appropriate for their problem.
The NKCS model was introduced to explore coevolutionary systems, that is, systems in which multiple species are closely interconnected. The fitness landscapes of the species are coupled to a controllable amount, where the underlying properties of the individual landscapes are also controllable. Previous work has assumed symmetry with respect to the controlling parameters. This paper explores the effects of reducing that symmetry on the behaviour of the coevolutionary system, including varying genome complexity, the degree of landscape coupling, and the use of local learning. Significant changes in behaviour from the traditional model are seen across the parameter space. These findings are suggested as particularly pertinent to symbiotic relationships.
In line with wicked problem literature, we argue that public management based on ordering societal issues to make them controllable and solvable (simplification) can be initially attractive, but in the long-term ineffective. We elaborate on an alternative management mode of increasing the complexity of both the problem and the approach to deal with the problem (complexification). Based on an in-depth case-study about transforming Utrecht Central Station (the Netherlands), we present complexification as a management strategy to revitalize processes that got locked by simplification management. Path-dependency shows up as a weakness of simplification, and path-creation as a strength of complexification.
Support vector machines (SVMs) have been exploited in a plethora of real-life classification and regression tasks, and are one of the most researched supervised learners. However, their generalization abilities strongly depend on the pivotal hyperparameters of the classifier, alongside its training dataset. Also, the training process is computationally and memory expensive, hence learning multiple SVMs to grid-search the hyperparameter space is infeasible in practice. In this paper, we address the problem of optimizing SVMs for binary classification of difficult datasets, including very large and extremely imbalanced cases. We propose an evolutionary technique that simultaneously optimizes the critical SVM aspects, including the training sample, kernel functions, and features. Also, we introduce a co-evolutionary scheme that allows us to guide the search in a competitive way to the highest-quality solutions. Our extensive experimental study performed over more than 120 benchmarks showed that the proposed algorithm outperforms popular supervised learners, as well as other techniques for optimizing SVMs reported in the literature.
Generative Adversarial Networks (GANs) are an adversarial model that achieved impressive results on generative tasks. In spite of the relevant results, GANs present some challenges regarding stability, making the training usually a hit-and-miss process. To overcome these challenges, several improvements were proposed to better handle the internal characteristics of the model, such as alternative loss functions or architectural changes on the neural networks used by the generator and the discriminator. Recent works proposed the use of evolutionary algorithms on GAN training, aiming to solve these challenges and to provide an automatic way to find good models. In this context, COEGAN proposes the use of coevolution and neuroevolution to orchestrate the training of GANs. However, previous experiments detected that some of the fitness functions used to guide the evolution are not ideal.
Evolutionary computation is inspired by the mechanisms of biological evolution. With algorithmic improvements and increasing computing resources, evolutionary computation has discovered creative and innovative solutions to challenging practical problems. This paper evaluates how today’s evolutionary computation compares to biological evolution and how it may fall short. A small number of well-accepted characteristics of biological evolution are considered: openendedness, major transitions in organizational structure, neutrality and genetic drift, multi-objectivity, complex genotype-to-phenotype mappings and co-evolution. Evolutionary computation exhibits many of these to some extent but more can be achieved by scaling up with available computing and by emulating biology more carefully. In particular, evolutionary computation diverges from biological evolution in three key respects: it is based on small populations and strong selection; it typically uses direct genotype-to-phenotype mappings; and it does not achieve major organizational transitions. These shortcomings suggest a roadmap for future evolutionary computation research, and point to gaps in our understanding of how biology discovers major transitions. Advances in these areas can lead to evolutionary computation that approaches the complexity and flexibility of biology, and can serve as an executable model of biological processes.
The complexity of problems has led to a shift toward the use of modular neural networks in place of traditional neural networks. The number of inputs to neural networks must be kept within manageable limits to escape from the curse of dimensionality. Attribute division is a novel concept to reduce the problem dimensionality without losing information. In this paper, the authors use Genetic Algorithms to determine the optimal distribution of the parameters to the various modules of the modular neural network. The attribute set is divided into the various modules. Each module computes the output using its own list of attributes. The individual results are then integrated by an integrator. This framework is used for the diagnosis of breast cancer. Experimental results show that optimal distribution strategy exceeds the well-known methods for the diagnosis of the disease.
Prevailing hypotheses concerning the origin of the ver- tebrate genome postulate successive genome duplications before the origin of fishes or tetrapods. These hypotheses predict episodic ex- pansion of gene families early in vertebrate evolution (mostly before the origin of fishes), tetralogous relationships between gene copies samples from invertebrates and vertebrates, and gene family trees with symmetrical shapes. None of these predictions were evident from a phylogenetic analysis of 35 gene families. Overall, the results do not refute the hypothesis that gene family evolution is governed by independent gene duplications occurring with identical proba- bility across gene lineages. These results suggest that the genome complexity of contemporary vertebrates mostly reflect small-scale (regional) DNA duplications instead of large-scale (genomic) duplications.
We discuss the methodological foundations for our work on the development of cognitive architectures, or control systems, for situated autonomous agents. Our focus is the problems of developing sensorimotor control systems for mobile robots, but we also discuss the applicability of our approach to the study of biological systems. We argue that, for agents required to exhibit sophisticated interactions with their environments, complex sensorimotor processing is necessary, and the design, by hand, of control systems capable of such processing is likely to become prohibitively difficult as complexity increases. We propose an automatic design process involving artificial evolution, wherein the basic building blocks used for evolving cognitive architectures are noise-tolerant dynamical neural networks. These networks may be recurrent and should operate in real time. The evolution should be incremental, using an extended and modified version of a genetic algorithm.
Practical constraints suggest that initial architecture evaluations should be done largely in simulation. To support our claims and proposals, we summarize results from some preliminary simulation experiments in which visually guided robots are evolved to operate in simple environments. Significantly, our results demonstrate that robust visually guided control systems evolve from evaluation functions that do not explicitly require monitoring visual input. We outline the difficulties involved in continuing with simulations and conclude by describing specialized visuorobotic equipment, designed to eliminate the need for simulated sensors and actuators.
An evolutionary algorithm is used as an engine for discovering new designs of digital circuits, particularly arithmetic functions. These designs are often radically different from those produced by top-down, human, rule-based approaches. It is argued that by studying evolved designs of gradually increasing scale, one might be able to discern new, efficient, and generalizable principles of design. The ripple-carry adder principle is one such principle that can be inferred from evolved designs for one and two-bit adders. Novel evolved designs for three-bit binary multipliers are given that are 20% more efficient (in terms of number of two-input gates used) than the most efficient known conventional design.
In a previous work it was argued that by studying evolved designs of gradually increasing scale, one might be able to discern new, efficient, and generalisable principles of design. These ideas are tested in the context of designing digital circuits, particularly arithmetic circuits. This process of discovery is seen as a principle extraction loop in which the evolved data is analysed both phenotypically and genotypically by processes of data mining and landscape analysis. The information extracted is then fed back into the evolutionary algorithm to enhance its search capabilities and hence increase the likelihood of identifying new principles which explain how to build systems which are too large to evolve.
In chordate phylogeny, changes in the nervous system, jaws, and appendages transformed meek filter feeders into fearsome predators. Gene duplication is thought to promote such innovation. Vertebrate ancestors probably had single copies of genes now found in multiple copies in vertebrates and gene maps suggest that this occurred by polyploidization. It has been suggested that one genome duplication event occurred before, and one after the divergence of ray-finned and lobe-finned fishes. Holland et al., however, have argued that because various vertebrates have several HOX clusters, two rounds of duplication occurred before the origin of jawed fishes. Such gene-number data, however, do not distinguish between tandem duplications and polyploidization events, nor whether independent duplications occurred in different lineages. To investigate these matters, we mapped 144 zebrafish genes and compared the resulting map with mammalian maps. Comparison revealed large conserved chromosome segments. Because duplicated chromosome segments in zebrafish often correspond with specific chromosome segments in mammals, it is likely that two polyploidization events occurred prior to the divergence of fish and mammal lineages. This zebrafish gene map will facilitate molecular identification of mutated zebrafish genes, which can suggest functions for human genes known only by sequence.
All vertebrates possess anatomical features not seen in their closest living relatives, the protochordates (tunicates and amphioxus). Some of these features depend on developmental processes or cellular behaviours that are again unique to vertebrates. We are interested in the genetic changes that may have permitted the origin of these innovations. Gene duplication, followed by functional divergence of new genes, may be one class of mutation that permits major evolutionary change. Here we examine the hypothesis that gene duplication events occurred close to the origin and early radiation of the vertebrates. Genome size comparisons are compatible with the occurrence of duplications close to vertebrate origins; more precise insight comes from cloning and phylogenetic analysis of gene families from amphioxus, tunicates and vertebrates. Comparisons of Hox gene clusters, other homeobox gene families, Wnt genes and insulin-related genes all indicate that there was a major phase of gene duplication close to vertebrate origins, after divergence from the amphioxus lineage; we suggest there was probably a second phase of duplication close to jawed vertebrate origins. From amphioxus and vertebrate homeobox gene expression patterns, we suggest that there are multiple routes by which new genes arising from gene duplication acquire new functions and permit the evolution of developmental innovations.
Duplicated genes are an important source of new protein functions and novel developmental and physiological pathways. Whereas most models for fate of duplicated genes show that they tend to be rapidly lost, models for pathway evolution suggest that many duplicated genes rapidly acquire novel functions. Little empirical evidence is available, however, for the relative rates of gene loss vs. divergence to help resolve these contradictory expectations. Gene families resulting from genome duplications provide an opportunity to address this apparent contradiction. With genome duplication, the number of duplicated genes in a gene family is at most 2n, where n is the number of duplications. The size of each gene family, e.g., 1, 2, 3, ..., 2n, reflects the patterns of gene loss vs. functional divergence after duplication. We focused on gene families in humans and mice that arose from genome duplications in early vertebrate evolution and we analyzed the frequency distribution of gene family size, i.e., the number of families with two, three or four members. All the models that we evaluated showed that duplicated genes are almost as likely to acquire a new and essential function as to be lost through acquisition of mutations that compromise protein function. An explanation for the unexpectedly high rate of functional divergence is that duplication allows genes to accumulate more neutral than disadvantageous mutations, thereby providing more opportunities to acquire diversified functions and pathways.
HOX genes specify cell fate in the anterior-posterior axis of animal embryos. Invertebrate chordates have one HOXcluster, but mammals have four, suggesting that cluster duplication facilitated the evolution of vertebrate body plans. This
report shows that zebrafish have seven hox clusters. Phylogenetic analysis and genetic mapping suggest a chromosome doubling event, probably by whole genome duplication,
after the divergence of ray-finned and lobe-finned fishes but before the teleost radiation. Thus, teleosts, the most species-rich
group of vertebrates, appear to have more copies of these developmental regulatory genes than do mammals, despite less complexity
in the anterior-posterior axis.
The origin of organismal complexity is generally thought to be tightly coupled to the evolution of new gene functions arising subsequent to gene duplication. Under the classical model for the evolution of duplicate genes, one member of the duplicated pair usually degenerates within a few million years by accumulating deleterious mutations, while the other duplicate retains the original function. This model further predicts that on rare occasions, one duplicate may acquire a new adaptive function, resulting in the preservation of both members of the pair, one with the new function and the other retaining the old. However, empirical data suggest that a much greater proportion of gene duplicates is preserved than predicted by the classical model. Here we present a new conceptual framework for understanding the evolution of duplicate genes that may help explain this conundrum. Focusing on the regulatory complexity of eukaryotic genes, we show how complementary degenerative mutations in different regulatory elements of duplicated genes can facilitate the preservation of both duplicates, thereby increasing long-term opportunities for the evolution of new gene functions. The duplication-degeneration-complementation (DDC) model predicts that (1) degenerative mutations in regulatory elements can increase rather than reduce the probability of duplicate gene preservation and (2) the usual mechanism of duplicate gene preservation is the partitioning of ancestral functions rather than the evolution of new functions. We present several examples (including analysis of a new engrailed gene in zebrafish) that appear to be consistent with the DDC model, and we suggest several analytical and experimental approaches for determining whether the complementary loss of gene subfunctions or the acquisition of novel functions are likely to be the primary mechanisms for the preservation of gene duplicates.
For a newly duplicated paralog, survival depends on the outcome of the race between entropic decay and chance acquisition of an advantageous regulatory mutation.
Sidow (1996, p. 717)
On one hand, it may fix an advantageous allele giving it a slightly different, and selectable, function from its original copy. This initial fixation provides substantial protection against future fixation of null mutations, allowing additional mutations to accumulate that refine functional differentiation. Alternatively, a duplicate locus can instead first fix a null allele, becoming a pseudogene.
Walsh (1995, p. 426)
Duplicated genes persist only if mutations create new and essential protein functions, an event that is predicted to occur rarely.
Nadeau and Sankoff (1997, p. 1259)
Thus overall, with complex metazoans, the major mechanism for retention of ancient gene duplicates would appear to have been the acquisition of novel expression sites for developmental genes, with its accompanying opportunity for new gene roles underlying the progressive extension of development itself.
Cooke et al. (1997, p. 362)
Biological life is in control of its own means of reproduction, which generally involves complex, autocatalysing chemical reactions. But this autonomy of design and manufacture has not yet been realized artificially. Robots are still laboriously designed and constructed by teams of human engineers, usually at considerable expense. Few robots are available because these costs must be absorbed through mass production, which is justified only for toys, weapons and industrial systems such as automatic teller machines. Here we report the results of a combined computational and experimental approach in which simple electromechanical systems are evolved through simulations from basic building blocks (bars, actuators and artificial neurons); the 'fittest' machines (defined by their locomotive ability) are then fabricated robotically using rapid manufacturing technology. We thus achieve autonomy of design and construction using evolution in a 'limited universe' physical simulation coupled to automatic fabrication.
We analyze a general model of multi-agent communication in which all agents communicate simultaneously to a message board. A genetic algorithm is used to evolve multi-agent languages for the predator agents in a version of the predator-prey pursuit problem. We show that the resulting behavior of the communicating multi-agent system is equivalent to that of a Mealy finite state machine whose states are determined by the agents' usage of the evolved language. Simulations show that the evolution of a communication language improves the performance of the predators. Increasing the language size (and thus increasing the number of possible states in the Mealy machine) improves the performance even further. Furthermore, the evolved communicating predators perform significantly better than all previous work on similar prey. We introduce a method for incrementally increasing the language size, which results in an effective coarse-to-fine search that significantly reduces the evolution time required to find a solution. We present some observations on the effects of language size, experimental setup, and prey difficulty on the evolved Mealy machines. In particular, we observe that the start state is often revisited, and incrementally increasing the language size results in smaller Mealy machines. Finally, a simple rule is derived that provides a pessimistic estimate on the minimum language size that should be used for any multi-agent problem.
In the research described here we extend past computational investigations of animal signaling by studying an artificial world in which a population of initially noncommunicating agents evolves to communicate about food sources and predators. Signaling in this world can be either beneficial (e.g., warning of nearby predators) or costly (e.g., attracting predators or competing agents). Our goals were twofold: to examine systematically environmental conditions under which grounded signaling does or does not evolve, and to determine how variations in assumptions made about the evolutionary process influence the outcome. Among other things, we found that agents warning of nearby predators were a common occurrence whenever predators had a significant impact on survival and signaling could interfere with predator success. The setting most likely to lead to food signaling was found to be difficult-to-locate food sources that each have relatively large amounts of food. Deviations from the selection methods typically used in traditional genetic algorithms were also found to have a substantial impact on whether communication evolved. For example, constraining parent selection and child placement to physically neighboring areas facilitated evolution of signaling in general, whereas basing parent selection upon survival alone rather than survival plus fitness measured as success in food acquisition was more conducive to the emergence of predator alarm signals. We examine the mechanisms underlying these and other results, relate them to existing experimental data about animal signaling, and discuss their implications for artificial life research involving evolution of communication.
Standard methods for simultaneously inducing the structure and weights of recurrent neural networks limit every task to an assumed class of architectures. Such a simplification is necessary since the interactions between network structure and function are not well understood. Evolutionary computations, which include genetic algorithms and evolutionary programming, are population-based search methods that have shown promise in many similarly complex tasks. This paper argues that genetic algorithms are inappropriate for network acquisition and describes an evolutionary program, called GNARL, that simultaneously acquires both the structure and weights for recurrent networks. GNARL's empirical acquisition method allows for the emergence of complex behaviors and topologies that are potentially excluded by the artificial architectural constraints imposed in standard network induction methods.
Several researchers have demonstrated how complex behavior can be learned through neuro-evolution (i.e. evolving neural networks with genetic algorithms). However, complex general behavior such as evading predators or avoiding obstacles, which is not tied to specific environments, turns out to be very difficult to evolve. Often the system discovers mechanical strategies (such as moving back and forth) that help the agent cope, but are not very effective, do not appear believable and would not generalize to new environments. The problem is that a general strategy is too difficult for the evolution system to discover directly. This paper proposes an approach where such complex general behavior is learned incrementally, by starting with simpler behavior and gradually making the task more challenging and general. The task transitions are implemented through successive stages of delta-coding (i.e. evolving modifications), which allows even converged populations to adapt to the new task. The...
This paper presents a method for the evolution of deterministic finite automata that combines genetic programming and cellular encoding. Programs are evolved that specify actions for the incremental growth of a deterministic finite automata from an initial single-state zygote. The results show that, given a test bed of positive and negative samples, the proposed method is successful at inducing automata to recognize several different languages. 1. Introduction The automatic creation of finite automata has long been a goal of the evolutionary computation community. Fogel et. al. [1966] was the first to propose the generation of deterministic finite automata (DFAs) by means of an evolutionary process, and the possibility of inferring languages from examples was initially established by Gold [1967]. Since then, much work has been done in the induction of DFAs for language recognition. Tomita [1982] showed that hill-climbing in the space of nine-state automata was both successful and supe...
In this paper we demonstrate that finite linear combinations of compositions of a fixed, univariate function and a set of affine functionals can uniformly approximate any continuous function ofn real variables with support in the unit hypercube; only mild conditions are imposed on the univariate function. Our results settle an open question about representability in the class of single hidden layer neural networks. In particular, we show that arbitrary decision regions can be arbitrarily well approximated by continuous feedforward neural networks with only a single internal, hidden layer and any continuous sigmoidal nonlinearity. The paper discusses approximation properties of other possible types of nonlinearities that might be implemented by artificial neural networks.
Learning and evolution ai-e two fundamental forms of adaptation. There has been a gl-eat interest in combining learning and evolution with artificial neural networks (ANN's) in recent years. This paper: I) reviews reviews ent combinations between ANN's and evolutionary algorithms (EA's), including using EA's to evolve ANN connection weights, architectures, learning rules, and input features; 2) discusses different search operators which have been used in various EA's; and 3) points out possible future research directions. it is shown, through a considerably large literature review, that combinations between ANN's and EA's can lead to significantly better intelligent systems than relying on ANN's or EA's alone.
In this paper we demonstrate that finite linear combinations of compositions of a fixed, univariate function and a set of
affine functionals can uniformly approximate any continuous function ofn real variables with support in the unit hypercube; only mild conditions are imposed on the univariate function. Our results
settle an open question about representability in the class of single hidden layer neural networks. In particular, we show
that arbitrary decision regions can be arbitrarily well approximated by continuous feedforward neural networks with only a
single internal, hidden layer and any continuous sigmoidal nonlinearity. The paper discusses approximation properties of other
possible types of nonlinearities that might be implemented by artificial neural networks.
Forma analysis is applied to the task of optimising the connectivity of a feed- forward neural network with a single layer of hidden units. This problem is reformulated as a multiset optimisation problem, and techniques are developed to allow principled genetic search over fixed- and variable-size sets and multisets. These techniques require a further generalisation of the notion of gene, which is presented. The result is a non-redundant representation of the neural network topology optimisation problem, together with recombination operators which have carefully designed and well-understood properties. The techniques developed have relevance to the application of genetic algorithms to constrained optimisation problems.
Understanding high-dimensional real world data usually requires learning the structure of the data space. The structure may contain high-dimensional clusters that are related in complex ways. Methods such as merge clustering and self-organizing maps are designed to aid the visualization and interpretation of such data. However, these methods often fail to capture critical structural properties of the input. Although self-organizing maps capture high-dimensional topology, they do not represent cluster boundaries or discontinuities. Merge clustering extracts clusters, but it does not capture local or global topology. This paper proposes an algorithm that combines the topology-preserving characteristics of self-organizing maps with a flexible, adaptive structure that learns the cluster boundaries in the data. 1 INTRODUCTION Real world data is often very high-dimensional, and often has a structure that is difficult both to recognize and describe. For instance, human blood can be tested fo...
Susumu Ohno's provocative book Evolution by Gene Duplication proposed that the creation of new proteins in nature (and hence new structures and new behaviors in living things) begins with a gene duplication and that gene duplication is "the major force of evolution." This paper describes six new architecture-altering operations for genetic programming that are patterned after the naturally-occurring chromosomal operations of gene duplication and gene deletion. When these new operations are included in a run of genetic programming, genetic programming can dynamically change, during the run, the architecture of a multi-part program consisting of a main program and a set of hierarchically-called subprograms. These on-the-fly architectural changes occur while genetic programming is concurrently evolving the work-performing steps of the main program and the hierarchically-called subprograms. The new operations can be interpreted as an automated way to change the representation of a problem while solving the problem. Equivalently, these operations can be viewed as an automated way to decompose a problem into an non-pre-specified number of subproblems of non-pre-specified dimensionality; solve the subproblems; and assemble the solutions of the subproblems into a solution of the overall problem. These operations can also be interpreted as providing an automated way to specialize and generalize.
This article describes a system for the evolution and coevolution of virtual creatures that compete in physically simulated three-dimensional worlds. Pairs of individuals enter one-on-one contests in which they contend to gain control of a common resource. The winners receive higher relative fitness scores allowing them to survive and reproduce. Realistic dynamics simulation including gravity, collisions, and friction, restricts the actions to physically plausible behaviors.
The morphology of these creatures and the neural systems for controlling their muscle forces are both genetically determined, and the morphology and behavior can adapt to each other as they evolve simultaneously. The genotypes are structured as directed graphs of nodes and connections, and they can efficiently but flexibly describe instructions for the development of creatures' bodies and control systems with repeating or recursive components. When simulated evolutions are performed with populations of competing creatures, interesting and diverse strategies and counterstrategies emerge.
All groups for which data exist go extinct at a rate that is constant for a given group. When this is recast in ecological form (the effective environment of any homogeneous, group of organisms deteriorates at a stochasti- cally constant rate), no definite exceptions exist although a few are possible. Extinction rates are similar within some very broad categories and vary regularly with size of area inhabited. A new unit of rates for discrete phenomena, the macarthur, is introduced. Laws are appropriate in evolutionary biology. Truth needs more than correct predictions. The Law of Extinction is evidence for ecological significance and comparability of taxa. A ncn- Markovian hypothesis to explain the law invokes mutually incompatible optima within an adaptive zone. A self-perpetuating fluctuation results which can be stated in terms of an unstudied aspect of zero-sum game theory. The hypothesis can be derived from a view that momentary fitness is the amount of control of resources, which remain constant in total amount. The hypothesis implies that long-term fitness has only two components and that events of mutualism are rare. The hypothesis largely explains the observed pattern of
molecular evolution.
An adaptation in one lineage (e.g. predators) may change the selection pressure on another lineage (e.g. prey), giving rise to a counter-adaptation. If this occurs reciprocally, an unstable runaway escalation or 'arms race' may result. We discuss various factors which might give one side an advantage in an arms race. For example, a lineage under strong selection may out-evolve a weakly selected one (' the life-dinner principle'). We then classify arms races in two independent ways. They may be symmetric or asymmetric, and they may be interspecific or intraspecific. Our example of an asymmetric interspecific arms race is that between brood parasites and their hosts. The arms race concept may help to reduce the mystery of why cuckoo hosts are so good at detecting cuckoo eggs, but so bad at detecting cuckoo nestlings. The evolutionary contest between queen and worker ants over relative parental investment is a good example of an intraspecific asymmetric arms race. Such cases raise special problems because the participants share the same gene pool. Interspecific symmetric arms races are unlikely to be important, because competitors tend to diverge rather than escalate competitive adaptations. Intraspecific symmetric arms races, exemplified by adaptations for male-male competition, may underlie Cope's Rule and even the extinction of lineages. Finally we consider ways in which arms races can end. One lineage may drive the other to extinction; one may reach an optimum, thereby preventing the other from doing so; a particularly interesting possibility, exemplified by flower-bee coevolution, is that both sides may reach a mutual local optimum; lastly, arms races may have no stable and but may cycle continuously. We do not wish necessarily to suggest that all, or even most, evolutionary change results from arms races, but we do suggest that the arms race concept may help to resolve three long-standing questions in evolutionary theory.
Molecular models have been used to demonstrate that a crossed strand-exchange between two homologous DNA molecules, a likely intermediate in genetic recombination, can be formed with all of the bases in the two double-helices remaining paired. There are two “outside” and two connecting, or “bridging”, strands in this structure. Since the positions of these two pairs of strands can be interchanged in the model by rotation, all four strands participating in the exchange must be regarded as equivalent. As a consequence, when pairing is terminated by strand-scissions, there is an equal probability of generating two kinds of DNA molecules: those with genes flanking the region of strand-exchange in parental configuration and those with flanking genes recombined.
Clusters of homeotic genes sculpt the morphology of animal body plans and body parts. Different body patterns may evolve through changes in homeotic gene number, regulation or function. Recent evidence suggests that homeotic gene clusters were duplicated early in vertebrate evolution, but the generation of arthropod and tetrapod diversity has largely involved regulatory changes in the expression of conserved arrays of homeotic genes and the evolution of interactions between homeotic proteins and the genes they regulate.
Phylogenetic analyses and sequence surveys of developmental regulator gene families indicate that two large-scale gene duplications, most likely genome duplications, occurred in ancestors of vertebrates. Relaxed constraints allowed duplicated and thus redundant genes to diverge in a two stage mechanism. Neutral changes dominated at first but then positively selected regulatory changes evolved the novel and increasingly complex vertebrate developmental program.
We consider "competitive coevolution," in which fitness is based on direct competition among individuals selected from two independently evolving populations of "hosts" and "parasites." Competitive coevolution can lead to an "arms race," in which the two populations reciprocally drive one another to increasing levels of performance and complexity. We use the games of Nim and 3-D Tic-Tac-Toe as test problems to explore three new techniques in competitive coevolution. "Competitive fitness sharing" changes the way fitness is measured; "shared sampling" provides a method for selecting a strong, diverse set of parasites; and the "hall of fame" encourages arms races by saving good individuals from prior generations. We provide several different motivations for these methods and mathematical insights into their use. Experimental comparisons are done, and a detailed analysis of these experiments is presented in terms of testing issues, diversity, extinction, arms race progress measurements, and drift.
An important question in neuroevolution is how to gain an advantage from evolving neural network topologies along with weights. We present a method, NeuroEvolution of Augmenting Topologies (NEAT), which outperforms the best fixed-topology method on a challenging benchmark reinforcement learning task. We claim that the increased efficiency is due to (1) employing a principled method of crossover of different topologies, (2) protecting structural innovation using speciation, and (3) incrementally growing from minimal structure. We test this claim through a series of ablation studies that demonstrate that each component is necessary to the system as a whole and to each other. What results is significantly faster learning. NEAT is also an important contribution to GAs because it shows how it is possible for evolution to both optimize and complexify solutions simultaneously, offering the possibility of evolving increasingly complex solutions over generations, and strengthening the analogy with biological evolution.
When learning a mixture model, we suffer from the local optima and model structure determination problems. In this paper, we present a method for simultaneously solving these problems based on the variational Bayesian (VB) framework. First, in the VB framework, we derive an objective function that can simultaneously optimize both model parameter distributions and model structure. Next, focusing on mixture models, we present a deterministic algorithm to approximately optimize the objective function by using the idea of the split and merge operations which we previously proposed within the maximum likelihood framework. Then, we apply the method to mixture of expers (MoE) models to experimentally show that the proposed method can find the optimal number of experts of a MoE while avoiding local maxima.
Neuroevolution, i.e. evolving artificial neural networks with genetic algorithms, has been highly effective in reinforcement learning tasks, particularly those with hidden state information. An important question in neuroevolution is how to gain an advantage from evolving neural network topologies along with weights. We present a method, NeuroEvolution of Augmenting Topologies (NEAT) that outperforms the best fixed-topology methods on a challenging benchmark reinforcement learning task. We claim that the increased efficiency is due to (1) employing a principled method of crossover of different topologies, (2) protecting structural innovation using speciation, and (3) incrementally growing from minimal structure. We test this claim through a series of ablation studies that demonstrate that each component is necessary to the system as a whole and to each other. What results is significantly faster learning. NEAT is also an important contribution to GAs because it shows how it is possible for evolution to both optimize and complexify solutions simultaneously, making it possible to evolve increasingly complex solutions over time, thereby strengthening the analogy with biological evolution
Learning and evolution are two fundamental forms of adaptation.
There has been a great interest in combining learning and evolution with
artificial neural networks (ANNs) in recent years. This paper: 1)
reviews different combinations between ANNs and evolutionary algorithms
(EAs), including using EAs to evolve ANN connection weights,
architectures, learning rules, and input features; 2) discusses
different search operators which have been used in various EAs; and 3)
points out possible future research directions. It is shown, through a
considerably large literature review, that combinations between ANNs and
EAs can lead to significantly better intelligent systems than relying on
ANNs or EAs alone