Article

Evolutionary Generative Adversarial Networks

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

Generative adversarial networks (GAN) have been effective for learning generative models for real-world data. However, existing GANs (GAN and its variants) tend to suffer from training problems such as instability and mode collapse. In this paper, we propose a novel GAN framework called evolutionary generative adversarial networks (E-GAN) for stable GAN training and improved generative performance. Unlike existing GANs, which employ a pre-defined adversarial objective function alternately training a generator and a discriminator, we utilize different adversarial training objectives as mutation operations and evolve a population of generators to adapt to the environment (i.e., the discriminator). We also utilize an evaluation mechanism to measure the quality and diversity of generated samples, such that only well-performing generator(s) are preserved and used for further training. In this way, E-GAN overcomes the limitations of an individual adversarial training objective and always preserves the best offspring, contributing to progress in and the success of GANs. Experiments on several datasets demonstrate that E-GAN achieves convincing generative performance and reduces the training problems inherent in existing GANs.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... All rights reserved. artificial intelligence (Wang et al. 2019;Garciarena, Santana, and Mendiburu 2018;Costa et al. 2019;Miikkulainen et al. 2019;Wu et al. 2019;Stanley and Miikkulainen 2002) as well as biology, sociology, and economics (Stewart and Plotkin 2014;Tilman, Plotkin, and Akçay 2020;Tilman, Watson, and Levin 2017;Bowles, Choi, and Hopfensitz 2003;Weitz et al. 2016), the rules of interaction can themselves adapt to the collective history of the agent behavior. For example, in adversarial learning and curriculum learning (Huang et al. 2011;Bengio et al. 2009), the difficulty of the game can increase over time by exactly focusing on the settings where the agent has performed the weakest. ...
... Doubly Evolutionary Behavior in AI and ML. Evolutionary game theory methods for training generative adversarial networks commonly exhibit time-evolving dynamic behavior and there is a pair of predominant doubly evolutionary process models (Costa et al. 2020;Wang et al. 2019;Garciarena, Santana, and Mendiburu 2018;Costa et al. 2019;Miikkulainen et al. 2019). In the first formulation, Wang et al. (2019) describe training the generator network, with parameters y, via a gradient-based algorithm composed of variation, evaluation, and selection. ...
... Evolutionary game theory methods for training generative adversarial networks commonly exhibit time-evolving dynamic behavior and there is a pair of predominant doubly evolutionary process models (Costa et al. 2020;Wang et al. 2019;Garciarena, Santana, and Mendiburu 2018;Costa et al. 2019;Miikkulainen et al. 2019). In the first formulation, Wang et al. (2019) describe training the generator network, with parameters y, via a gradient-based algorithm composed of variation, evaluation, and selection. The discriminator network, with parameters w updated via gradient-based learning, is modeled as the environment operating in a feedback loop with y. ...
Article
The predominant paradigm in evolutionary game theory and more generally online learning in games is based on a clear distinction between a population of dynamic agents that interact given a fixed, static game. In this paper, we move away from the artificial divide between dynamic agents and static games, to introduce and analyze a large class of competitive settings where both the agents and the games they play evolve strategically over time. We focus on arguably the most archetypal game-theoretic setting---zero-sum games (as well as network generalizations)---and the most studied evolutionary learning dynamic---replicator, the continuous-time analogue of multiplicative weights. Populations of agents compete against each other in a zero-sum competition that itself evolves adversarially to the current population mixture. Remarkably, despite the chaotic coevolution of agents and games, we prove that the system exhibits a number of regularities. First, the system has conservation laws of an information-theoretic flavor that couple the behavior of all agents and games. Secondly, the system is Poincare recurrent, with effectively all possible initializations of agents and games lying on recurrent orbits that come arbitrarily close to their initial conditions infinitely often. Thirdly, the time-average agent behavior and utility converge to the Nash equilibrium values of the time-average game. Finally, we provide a polynomial time algorithm to efficiently predict this time-average behavior for any such coevolving network game.
... Also, since the exploration capability of the AEs with respect to gradient-based meth-ods, there are many hybrid approaches which combine these techniques to achieve better results [16,17]. ...
... We can find out in the literature different strategies: For instance, we can apply the genetic operators between two backpropagation steps with the aim to increase the stability and reduce the local minima issue, like [23]. Otherwise, as in [16], we can use a population of candidates solutions to explore the flexible region and then use the backpropagation to optimize each instance; or else combine both strategies as in [17] to explore and reduce that problems even more. ...
... Therefore there are many hybrid approaches proposed, for instance, in [23] a genetic mutation operator between a backpropagation pass and the next one is proposed, that resulted in an improvement of the stability of the original algorithm [2] to train a generator capable to produces the MNIST [37] images. Another hybrid is the Evolutionary GAN [16], in which a population of generators is trained, more precisely the evolution is performed by means of different backpropagation passes with different loss functions applied to each individual of the population, and the best offspring are chosen for the next generation using a ranking selection. ...
Thesis
Neuroevolution is a branch of artificial intelligence that uses evolutionary algorithms to optimize neural networks. During the last thirty years, many evolutionary algorithms have been proposed, someones focused on the neural networks' weights optimization, and others on its structure optimization. The goal of this thesis is to propose new evolutionary algorithms for the optimization of the neural network able to achieve better results in contexts where the gradient descent does not achieve satisfying results. We started with classification, a classical machine learning task, in which we tested our algorithm with respect to the well-known backpropagation one on multiple real problems. Then, we moved on to test our algorithm on a harder task, in which a neural network has to be trained, utilizing a set of input and output pairs, to reproduce the behavior of an algorithm. The last task faced is regarding image generation, where we have proposed and tested an evolutionary algorithm able to mitigate some limitations of training algorithms already presented in the literature. For each of these tasks, we have recorded that the use of the evolutionary algorithms can achieve better results, or improve the training stability, in those problems with no-differentiable features. For classification, this is verified in those cases where the problem features are qualitative information or appertaining to discrete domains. Instead, since the algorithm learning task is not differentiable per se, this kind of approach resulted in a better quality solution and also faster training time, on all problems tested. While in the image generation the evolutionary technique improved the stability of the learning process achieving better results on synthetic and real data.
... Evolutionary computing techniques, which can automatically design hyperparameters and network architectures according to practical problems, have been receiving increasing attention from researchers in deep learning. Wang et al. introduced evolutionary computing technology into GANs [20]. In each iteration, a set of generators were obtained by using different objective functions as mutation operations, and a certain evaluation strategy was used to select the best offspring to enter the next iteration. ...
... Wang et al. designed an evolutionary generative adversarial network framework [20]. By applying different objective functions as mutation strategies, a certain number of generator individuals are generated, and the generators are continuously evolved through fitness evaluation and selection strategies. ...
... In traditional EGANs [20], the fitness evaluation function considers the diversity of samples, that is, they believe that there is a positive correlation between the gradient norm of D and the diversity of generated samples. However, lin et al.proved it is unscientific [24]. ...
Article
Full-text available
Generative adversarial networks have made remarkable achievements in generative tasks. However, instability and mode collapse are still frequent problems. We improve the framework of evolutionary generative adversarial networks (E-GANs), calling it phased evolutionary generative adversarial networks (PEGANs), and adopt a self-attention module to improve upon the disadvantages of convolutional operations. During the training process, the discriminator will play against multiple generators simultaneously, where each generator adopts a different objective function as a mutation operation. Every time after the specified number of training iterations, the generator individuals will be evaluated and the best performing generator offspring will be retained for the next round of evolution. Based on this, the generator can continuously adjust the training strategy during training, and the the self-attention module also enables the model to obtain the modeling ability of long-range dependencies. Experiments on two datasets showed that PEGANs improve the training stability and are competitive in generating high-quality samples.
... However, in a distributed control application, agents' actions may directly modify the strategic environment; for instance if a search-and-rescue UAV identifies a disaster victim, that victim may be removed from the list of other UAVs' objectives. Other applications that can be modeled by history-dependent games are in machine learning [9]- [11] and biology [12], [13]. ...
... The potential function can be verified by checking (9). Letting N i (1) denote the neighbors of agent i with play action 1. ...
Preprint
The theory of learning in games has extensively studied situations where agents respond dynamically to each other by optimizing a fixed utility function. However, in real situations, the strategic environment varies as a result of past agent choices. Unfortunately, the analysis techniques that enabled a rich characterization of the emergent behavior in static environment games fail to cope with dynamic environment games. To address this, we develop a general framework using probabilistic couplings to extend the analysis of static environment games to dynamic ones. Using this approach, we obtain sufficient conditions under which traditional characterizations of Nash equilibria with best response dynamics and stochastic stability with log-linear learning can be extended to dynamic environment games. As a case study, we pose a model of cyber threat intelligence sharing between firms and a simple dynamic game-theoretic model of social precautions in an epidemic, both of which feature dynamic environments. For both examples, we obtain conditions under which the emergent behavior is characterized in the dynamic game by performing the traditional analysis on a reference static environment game.
... Badem et al. [23] developed a hybrid method integrating limited-memory Broyden-Fletcher-Goldfarb-Shannon (L-BFGS) into artificial bee colony (ABC) to statistically gain better performance of autoencoders (AEs) in comparison to traditional L-BFGS and ABC variants as well. Wang et al. [24] devised an evolutionary generative adversial networks (GANs) by applying mutation operators and elitist mechanism into adversarial training process to improve GAN performance considering various image generation problems. An alternative way is to decompose the training problem into a multiobjective optimization one. ...
... Considering the movement from θ t+1 pL to θ t+1 F in the second phase, it is reminded that θ t L , θ t pL and θ t+1 pL are connected through the motion chains beginning from θ t F as presented in Fig. 1 and Eqs. (23) - (24). Therefore, the gradient ∇J(θ t+1 pL ) can be extracted and calibrated from ∇J(θ t L ) via chain rule. ...
Article
Full-text available
In this paper, we explore the advantages of heuristic mechanisms and devise a new optimization framework named Sequential Motion Optimization (SMO) to strengthen gradientbased methods. The key idea of SMO is inspired from a movement mechanism in a recent metaheuristic method called Balancing Composite Motion Optimization (BCMO). Specifically, SMO establishes a sequential motion chain of two gradientguided individuals including a leader and a follower to enhance the effectiveness of parameter updates in each iteration. A surrogate gradient model with low computation cost is theoretically established to estimate the gradient of the follower by that of the leader through chain rule during training process. Experimental results in terms of training quality on both fully-connected multilayer perceptrons (MLPs) and convolutional neural networks (CNNs) with respect to three popular benchmark datasets including MNIST, Fashion-MNIST and CIFAR-10 demonstrate the superior performance of the proposed framework in comparison with the vanilla stochastic gradient descent (SGD) implemented via back-propagation (BP) algorithm. Although this study only introduces the vanilla gradient descent (GD) as a main gradientguided factor in SMO for deep neural networks (DNNs) training application, it is greatly potential to combine with other gradientbased variants to improve its effectiveness and solve other largescale optimization problems in practice.
... Current few-shot part segmentation works mainly focus on utilizing and studying the impact of pre-trained features [4,34,41,45,53]. Rep-GAN [41] and DatasetGAN [53] are some early works on fewshot part segmentation, and they extract pixel-wise representations from a pre-trained GAN and use them as feature vectors for a segmentation network. ...
... distribution of real-world data (i.e., images) in an implicit or explicit way. In recent years, GANs achieve great success in image generation [2,3,17,22,23,28,30,37,45]. In general, for most visual generative models, the ultimate goal is to generate high-quality visual contents that follow the training data distribution. ...
Preprint
Few-shot part segmentation aims to separate different parts of an object given only a few annotated samples. Due to the challenge of limited data, existing works mainly focus on learning classifiers over pre-trained features, failing to learn task-specific features for part segmentation. In this paper, we propose to learn task-specific features in a "pre-training"-"fine-tuning" paradigm. We conduct prompt designing to reduce the gap between the pre-train task (i.e., image generation) and the downstream task (i.e., part segmentation), so that the GAN priors for generation can be leveraged for segmentation. This is achieved by projecting part segmentation maps into the RGB space and conducting interpolation between RGB segmentation maps and original images. Specifically, we design a fine-tuning strategy to progressively tune an image generator into a segmentation generator, where the supervision of the generator varying from images to segmentation maps by interpolation. Moreover, we propose a two-stream architecture, i.e., a segmentation stream to generate task-specific features, and an image stream to provide spatial constraints. The image stream can be regarded as a self-supervised auto-encoder, and this enables our model to benefit from large-scale support images. Overall, this work is an attempt to explore the internal relevance between generation tasks and perception tasks by prompt designing. Extensive experiments show that our model can achieve state-of-the-art performance on several part segmentation datasets.
... While evolutionary computation (EC) has been utilized before to help optimize and search for successful GAN architectures [28] and loss functions [9], we could not find an evolutionary system that can perform neural architecture search for a SimGAN, adjust custom loss functions, train and evaluate models across multiple objectives, and optimize hyperparameters simultaneously while being customizable to fit any data problem, including 1D data. ...
... Integrating evolutionary techniques into traditional GAN approaches is not a novel idea. Evolutionary GAN (E-GAN) [28] evolves a population of generators, where the mutation method selects among different loss functions to train the generators, which then compete against a single discriminator. COEGAN [3] utilizes neuroevolution to coevolve discriminator and generator architectures. ...
Preprint
Full-text available
Machine Learning models are used in a wide variety of domains. However, machine learning methods often require a large amount of data in order to be successful. This is especially troublesome in domains where collecting real-world data is difficult and/or expensive. Data simulators do exist for many of these domains, but they do not sufficiently reflect the real world data due to factors such as a lack of real-world noise. Recently generative adversarial networks (GANs) have been modified to refine simulated image data into data that better fits the real world distribution, using the SimGAN method. While evolutionary computing has been used for GAN evolution, there are currently no frameworks that can evolve a SimGAN. In this paper we (1) extend the SimGAN method to refine one-dimensional data, (2) modify Easy Cartesian Genetic Programming (ezCGP), an evolutionary computing framework, to create SimGANs that more accurately refine simulated data, and (3) create new feature-based quantitative metrics to evaluate refined data. We also use our framework to augment an electrocardiogram (ECG) dataset, a domain that suffers from the issues previously mentioned. In particular, while healthy ECGs can be simulated there are no current simulators of abnormal ECGs. We show that by using an evolved SimGAN to refine simulated healthy ECG data to mimic real-world abnormal ECGs, we can improve the accuracy of abnormal ECG classifiers.
... Chen et al. [22] combined LSTM with chaos theory to optimize the tone shift in music without deformation, reduce the amount of calculation, and optimize the training efficiency. Lehner et al. [23] combined LSTM with a restricted Boltzmann machine (RBM) [24]. These techniques and other probabilistic methods are combined with deep neural networks to help people better make music. ...
Article
Full-text available
With the rapid development of artificial intelligence, the application of this new technology to music generation has attracted more attention and achieved gratifying results. This study proposes a method for combining the transformer deep-learning model with generative adversarial networks (GANs) to explore a more competitive music generation algorithm. The idea of text generation in natural language processing (NLP) was used for reference, and a unique loss function was designed for the model. The training process solves the problem of a nondifferentiable gradient in generating music. Compared with the problem that LSTM cannot deal with long sequence music, the model based on transformer and GANs can extract the relationship in the notes of long sequence music samples and learn the rules of music composition well. At the same time, the optimized transformer and GANs model has obvious advantages in the complexity of the system and the accuracy of generating notes.
... Target detection and recognition algorithms based on deep learning can usually be classified according to different angles. Examples are region based image target detection and recognition algorithms, such as R-CNN (regions with convolutional neural network features), Fast-RCNN, and Faster-RCNN; regression-based image target detection and recognition algorithms, such as YOLO and SSD [5,6]; and search based image target detection and recognition algorithms, such as AttentionNet. Although the deep learning method improves the efficiency of image target detection and recognition to a certain extent, due to the high operation cost in the process of target detection and recognition and the low efficiency of image processing for large scenes, in addition, the existing methods have poor robustness; they have a high error rate in target recognition, and the deep learning network model has high dependence on parameters. ...
Article
Full-text available
Image target detection and recognition had been widely used in many fields. However, the existing methods had poor robustness; they not only had high error rate of target recognition but also had high dependence on parameters, so they were limited in application. Therefore, this paper proposed an image target detection and recognition method based on the improved R-CNN model, so as to detect and recognize the dynamic image target in real time. Based on the analysis of the existing theories of deep learning detection and recognition, this paper summarized the composition and working principle of the traditional image target detection and recognition system and compared the basic models of target detection and recognition, such as R-CNN network, Fast-RCNN network, and Faster-RCNN network. In order to improve the accuracy and real-time performance of the model in image target detection and recognition, this paper adopted the target feature matching module in the existing R-CNN network model, so as to obtain the feature map close to the same target through similarity calculation for the features extracted by the model. Therefore, an image target detection and recognition algorithm based on the improved R-CNN network model is proposed. Finally, the experimental results showed that the image target detection and recognition algorithm proposed in this paper can be better applied to image target detection and classification in complex environment and had higher detection efficiency and recognition accuracy than the existing models. The target detection and recognition algorithm proposed in this paper had certain reference value and guiding significance for further application research in related fields.
... From the optimization (algorithm) point of view, many approaches can be found in the neuroevolution literature, ranging from evolutionary algorithm (EA) [21], GA [22], harmony search [23] and mixed integer parallel efficient global optimization technique [24], to Bayesian optimization [7]. And also, from the point of view of the neural network architecture, e.g., recurrent neural network [25], convolutional neural network [26] and generative adversarial networks [27]. ...
Article
Algorithmic design in neural architecture search (NAS) has received a lot of attention, aiming to improve performance and reduce computational cost. Despite the great advances made, few authors have proposed to tailor initialization techniques for NAS. However, the literature shows that a good initial set of solutions facilitates finding the optima. Therefore, in this study, we propose a data-driven technique to initialize a population-based NAS algorithm. First, we perform a calibrated clustering analysis of the search space, and second, we extract the centroids and use them to initialize a NAS algorithm. We benchmark our proposed approach against random and Latin hypercube sampling initialization using three population-based algorithms, namely a genetic algorithm, an evolutionary algorithm, and aging evolution, on CIFAR-10. More specifically, we use NAS-Bench-101 to leverage the availability of NAS benchmarks. The results show that compared to random and Latin hypercube sampling, the proposed initialization technique enables achieving significant long-term improvements for two of the search baselines, and sometimes in various search scenarios (various training budget). Besides, we also investigate how an initial population gathered on the tabular benchmark can be used for improving search on another dataset, the So2Sat LCZ-42. Our results show similar improvements on the target dataset, despite a limited training budget. Moreover, we analyse the distributions of solutions obtained and find that that the population provided by the data-driven initialization technique enables retrieving local optima (maxima) of high fitness and similar configurations.
... Douzas and Bacao (2018) proposed an efficient algorithm for learning unbalanced data using a conditional GAN, which can generate better virtual data than existing data augmentation algorithms. Wang et al. (2019) proposed an improved data augmentation algorithm by applying evolutionary computation to solve existing GAN problems, such as model instability and mode collapse. Shin et al. (2020) proposed a hybrid deep learning algorithm based on GAN and transfer learning for video anomaly detection. ...
Article
Full-text available
Deep learning-based metamodeling can approximate complex engineering systems based on design data, but it has limitations in acquiring a large amount of data through experiments or simulations. When the design data for metamodeling are insufficient, data can be generated through generative models of deep learning. This study proposes a deep learning-based efficient metamodeling method called domain knowledge-integrated designable data augmentation (DDA) with transfer learning for engineering design. The DDA is a metamodel that applies an inverse generator to existing data augmentation algorithms. Virtual responses can be generated using a small number of actual responses to predict the performance of an engineering system and estimate the design variables that affect the generated virtual responses. Moreover, a rapid and accurate design can be achieved by applying transfer learning and domain knowledge-based learning to DDA. The proposed algorithm was applied to the design of a bumper considering vehicle crash safety. As a result, virtual responses that were approximately 95% similar to the actual responses were generated, and design solutions were derived. In addition, the validity of the proposed algorithm was verified by comparing it with existing metamodels. Graphical abstract
... The GAN model and the advancements have been found significantly important for learning the structure of deep generative models to generate images or videos similar to real-time data. However, the persist of instability and mode collapse issues with the generated results, the evolutionary GAN model (Wang et al. 2019b) addresses these issues by employing different adversarial training methods and mutation operations in both generator network and discriminator network. ...
Article
Full-text available
The recent advancements of unsupervised deep generative models have produced incredible results in image and video generation tasks. However, existing approaches still pose huge challenges in the high-quality video generation process. The generated videos consist of blurred effects and poor video quality. In this paper, we introduce a novel generative framework named dynamic generative adversarial networks (dynamic GAN) model for regulating the adversarial training and generating photo-realistic high-quality sign language videos. The proposed model uses skeletal poses information and person images as input and produces high-quality videos. In generator phase, the proposed model uses U-Net-like network to generate target frames from skeletal poses. Further, the generated samples are classified using the VGG-19 framework to identify its word class. The discriminator network classifies the real and fake samples as well as concatenates the resultant frames and generates the high-quality video output. Unlike, existing approaches the proposed novel framework produces photo-realistic video quality results without employing any animation or avatar approaches. To evaluate the model performance qualitatively and quantitatively, the proposed model has been evaluated using three benchmark datasets that yield plausible results. The datasets are RWTH-PHOENIX-Weather 2014T dataset, and our self-created dataset for Indian Sign Language (ISL-CSLTR), and the UCF-101 Action Recognition dataset. The proposed model achieves average 28.7167 PSNR score, 0.921 average SSIM score, 14 average FID score and 8.73 ± 0.23 average inception score which are relatively higher than existing approaches.
... There are many different cost functions for many different models including Bayesian formulations [19] or a third network [22], using a genetic approach [24] or even using Variational auto-encoders [25]. However, a recent work [12] has shown that the choice of a cost function does not matter compared to optimizing hyper-parameters. ...
... In a similar vein, in [22] it is proposed to use the f -divergence (a divergence in the spirit of the Kullback-Leibler divergence) as criterion for training GANs. Even genetic algorithms have been used to stabilize the training process, as in [27], where the authors applied genetic programming to optimize the use of different adversarial training objectives and evolved a population of generators to adapt to the discriminator, which acts as the hostile environment driving evolution. Nevertheless, despite of all these efforts, no master method is currently available and hence assuring a fast, or even effective, convergence of GANs is an open problem. ...
Preprint
Generative Adversarial Networks (GANs) are powerful Machine Learning models capable of generating fully synthetic samples of a desired phenomenon with a high resolution. Despite their success, the training process of a GAN is highly unstable and typically it is necessary to implement several accessory heuristics to the networks to reach an acceptable convergence of the model. In this paper, we introduce a novel method to analyze the convergence and stability in the training of Generative Adversarial Networks. For this purpose, we propose to decompose the objective function of the adversary min-max game defining a periodic GAN into its Fourier series. By studying the dynamics of the truncated Fourier series for the continuous Alternating Gradient Descend algorithm, we are able to approximate the real flow and to identify the main features of the convergence of the GAN. This approach is confirmed empirically by studying the training flow in a $2$-parametric GAN aiming to generate an unknown exponential distribution. As byproduct, we show that convergent orbits in GANs are small perturbations of periodic orbits so the Nash equillibria are spiral attractors. This theoretically justifies the slow and unstable training observed in GANs.
... Many data augmentation, regularization, architectures, and pre-training techniques [10] have been proposed to mitigate this issue. (i) Data Augmentation [12,24,13,25,26] is a striking method for mitigating overfitting of the discriminator and orthogonal to other ongoing researches on training, architecture, and regularization. Popular augmentation strategies, such as Adaptive Data Augmentation (ADA), are employed as the essential complements in most DE-GANs. ...
Preprint
Full-text available
Data-Efficient GANs (DE-GANs), which aim to learn generative models with a limited amount of training data, encounter several challenges for generating high-quality samples. Since data augmentation strategies have largely alleviated the training instability, how to further improve the generative performance of DE-GANs becomes a hotspot. Recently, contrastive learning has shown the great potential of increasing the synthesis quality of DE-GANs, yet related principles are not well explored. In this paper, we revisit and compare different contrastive learning strategies in DE-GANs, and identify (i) the current bottleneck of generative performance is the discontinuity of latent space; (ii) compared to other contrastive learning strategies, Instance-perturbation works towards latent space continuity, which brings the major improvement to DE-GANs. Based on these observations, we propose FakeCLR, which only applies contrastive learning on perturbed fake samples, and devises three related training techniques: Noise-related Latent Augmentation, Diversity-aware Queue, and Forgetting Factor of Queue. Our experimental results manifest the new state of the arts on both few-shot generation and limited-data generation. On multiple datasets, FakeCLR acquires more than 15% FID improvement compared to existing DE-GANs. Code is available at https://github.com/iceli1007/FakeCLR.
... In particular, A Survey on Learnable Evolutionary Algorithms for Scalable Multiobjective Optimization Songbai Liu E the agent (like a generator) based on the observation of state and reward to create an action, which will update the environment to a new state followed by determining whether to reward or punish this action-state pair (like a discriminator) [13], resulting in adapting the policy to predict the best trajectory of actions for gaining the maximum cumulative rewards. Because of these potential connections, MOEAs have been the alternative choices for augmenting generative models [14] and reinforcement learning [15]. After decades of development, MOEAs should have grown into sophisticated, innovative, and creative solvers for various MOPs, especially from the perspective of biological evolution [16]. ...
Preprint
Recent decades have witnessed remarkable advancements in multiobjective evolutionary algorithms (MOEAs) that have been adopted to solve various multiobjective optimization problems (MOPs). However, these progressively improved MOEAs have not necessarily been equipped with sophisticatedly scalable and learnable problem-solving strategies that are able to cope with new and grand challenges brought by the scaling-up MOPs with continuously increasing complexity or scale from diverse aspects, mainly including expensive function evaluations, many objectives, large-scale search space, time-varying environments, and multitask. Under different scenarios, it requires divergent thinking to design new powerful MOEAs for solving them effectively. In this context, research into learnable MOEAs that arm themselves with machine learning techniques for scaling-up MOPs has received extensive attention in the field of evolutionary computation. In this paper, we begin with a taxonomy of scalable MOPs and learnable MOEAs, followed by an analysis of the challenges that scaling up MOPs pose to traditional MOEAs. Then, we synthetically overview recent advances of learnable MOEAs in solving various scaling up MOPs, focusing primarily on three attractive and promising directions (i.e., learnable evolutionary discriminators for environmental selection, learnable evolutionary generators for reproduction, and learnable evolutionary transfer for sharing or reusing optimization experience between different problem domains). The insight into learnable MOEAs held throughout this paper is offered to the readers as a reference to the general track of the efforts in this field.
... After completing the super-supernet training, we usually search both base model and a group of scaling strategies by an evolution algorithm (EA). The original objective of the optimization is globally maximizing the weighted sum of the validation accuracies ACC of the M scaling stages [41]] as ...
Preprint
Full-text available
Recently, community has paid increasing attention on model scaling and contributed to developing a model family with a wide spectrum of scales. Current methods either simply resort to a one-shot NAS manner to construct a non-structural and non-scalable model family or rely on a manual yet fixed scaling strategy to scale an unnecessarily best base model. In this paper, we bridge both two components and propose ScaleNet to jointly search base model and scaling strategy so that the scaled large model can have more promising performance. Concretely, we design a super-supernet to embody models with different spectrum of sizes (e.g., FLOPs). Then, the scaling strategy can be learned interactively with the base model via a Markov chain-based evolution algorithm and generalized to develop even larger models. To obtain a decent super-supernet, we design a hierarchical sampling strategy to enhance its training sufficiency and alleviate the disturbance. Experimental results show our scaled networks enjoy significant performance superiority on various FLOPs, but with at least 2.53x reduction on search cost. Codes are available at https://github.com/luminolx/ScaleNet.
... Taking advantage of the coarse-tofine scheme for high-quality image generation, Laplacian GAN (LapGAN) [50] develops a sequential image generation framework by adopting the Laplacian pyramid coding [51] and conducting up-sampling and down-sampling for the generator and the discriminator respectively. By combining different objectives of GANs with the evolutionary strategy, Evolutionary GAN [52] not only alleviates the mode collapse issue, but further improved the generation performance. To explore the properties of unlabelled data, Information Maximizing GAN (InfoGAN) [53] introduces a regularization term, i.e., the mutual information, to the cost function, and improves the interpretability of input vectors for the generator. ...
Preprint
Full-text available
Quantum computers are next-generation devices that hold promise to perform calculations beyond the reach of classical computers. A leading method towards achieving this goal is through quantum machine learning, especially quantum generative learning. Due to the intrinsic probabilistic nature of quantum mechanics, it is reasonable to postulate that quantum generative learning models (QGLMs) may surpass their classical counterparts. As such, QGLMs are receiving growing attention from the quantum physics and computer science communities, where various QGLMs that can be efficiently implemented on near-term quantum machines with potential computational advantages are proposed. In this paper, we review the current progress of QGLMs from the perspective of machine learning. Particularly, we interpret these QGLMs, covering quantum circuit born machines, quantum generative adversarial networks, quantum Boltzmann machines, and quantum autoencoders, as the quantum extension of classical generative learning models. In this context, we explore their intrinsic relation and their fundamental differences. We further summarize the potential applications of QGLMs in both conventional machine learning tasks and quantum physics. Last, we discuss the challenges and further research directions for QGLMs.
... In recent years, a number of CNN-based generative models have been proposed, and achieved significant progress on (un)conditional image generation tasks. Till now, CNN-based generative models (e.g., GANs [25,79,80], VAEs [72]) are good at synthesizing high-resolution and high-fidelity object images, which include but are not limited to flowers, human/animal faces, and buildings [15,26,28]. However, generating complex real-world scenes which include multiple instances with variant layout and scale has still been a challenging task [5,9,12]. ...
Preprint
We present a method that achieves state-of-the-art results on challenging (few-shot) layout-to-image generation tasks by accurately modeling textures, structures and relationships contained in a complex scene. After compressing RGB images into patch tokens, we propose the Transformer with Focal Attention (TwFA) for exploring dependencies of object-to-object, object-to-patch and patch-to-patch. Compared to existing CNN-based and Transformer-based generation models that entangled modeling on pixel-level&patch-level and object-level&patch-level respectively, the proposed focal attention predicts the current patch token by only focusing on its highly-related tokens that specified by the spatial layout, thereby achieving disambiguation during training. Furthermore, the proposed TwFA largely increases the data efficiency during training, therefore we propose the first few-shot complex scene generation strategy based on the well-trained TwFA. Comprehensive experiments show the superiority of our method, which significantly increases both quantitative metrics and qualitative visual realism with respect to state-of-the-art CNN-based and transformer-based methods. Code is available at https://github.com/JohnDreamer/TwFA.
... e research of artificial neural network is an important branch in the field of artificial intelligence. Since Hinton put forward the deep neural network in 2006, the neural network has made great achievements in the field of information processing such as voice, image, and text [3][4][5]. Researchers in major universities and research institutions all over the world have vigorously carried out relevant research and proposed a variety of deep neural networks, which has greatly expanded the application of deep neural network [6,7]. Modern deep neural network model mainly includes three basic structures: sequence-to-sequence neural network, convolution neural network, and adversarial generation network [8,9]. ...
Article
Full-text available
In this paper, aiming at the application of online rapid sorting of waste textiles, a large number of effective high-content blending data are generated by using generative adversity network to deeply mine the combination relationship of blending spectra, and A BEGAN-RBF-SVM classification model is constructed by compensating the imbalance of negative samples in the data set. Various experiments show that the model can effectively extract the spectrum of pure textile samples. The classification model has high robustness and high speed, reaches the performance of similar products in the world, and has a broad application market.
... Optimization algorithms have witnessed prevailing success in different application areas [134,135,167,182]. Formally, they help to find a parameter vector x * in order to minimize an objective function f (x) : ...
Thesis
Optimization algorithms have seen unprecedented growth thanks to their successful applications in fields including engineering and health sciences. Similarly, machine learning has been popularly used in perceptual tasks by both academic and industrial researchers. They are both designed to find solutions for some specific tasks and it is not straightforward to apply an existing method to a new domain and still have superior results. Hence, experts have to construct specialized methods for each given task. The extra degree of freedom from the design space could make this process very time-consuming and has motivated a demand for automated search methods that can be adopted easily without any expert knowledge. In this thesis, we claim the mentioned contribution by porting existing methodsfrom machine learning to optimization domain and vice versa. The first part of this thesis suggests many lines of investigation with possibilities related to the development of more enhanced optimization algorithms using machine learning. The second part discusses a modeling scheme to optimize the performance of machine learning tools with metaheuristic algorithms.
... GEO is mainly influenced by Evolutionary Generative Adversarial Networks (EGAN) [22]. Like Local Generative Surrogates Optimization (L-GSO) [21], surrogate model-based optimizations are also related to GEO. ...
Preprint
Many scientific and technological problems are related to optimization. Among them, black-box optimization in high-dimensional space is particularly challenging. Recent neural network-based black-box optimization studies have shown noteworthy achievements. However, their capability in high-dimensional search space is still limited. This study investigates a novel black-box optimization method based on evolution strategy and generative neural network model. We designed the algorithm so that the evolutionary strategy and the generative neural network model work cooperatively with each other. This hybrid model enables reliable training of surrogate networks; it optimizes multi-objective, high-dimensional, and stochastic black-box functions. In this experiment, our method outperforms baseline optimization methods, including an NSGA-II and Bayesian optimization.
... A generative adversarial network (GAN) is a typical generative model [34]. It mainly consists of two parts, generator (G) and discriminator (D). ...
Article
Full-text available
Anomaly detection plays an important role in intelligent surveillance and has attracted increasing attention from researchers in recent years. It is generally regarded as discrimination that cannot be properly represented in most approaches. Despite its importance in probing the scarcity and indefinability of abnormal data during training, designing an effective network is exceptionally complex due to the diversity of the motion information, difficulty of parsing prediction errors, robustness and so on. To improve the ability to extract subtle features between normal and abnormal frames, and to improve the robustness to noise, a generative assistant discriminative network for anomaly detection is proposed. This method detects anomalies by predicting future frames in combination of adversary and cooperation, which mainly consists of the generator, discriminator and assistor. The generator predicts the future frames, while the discriminator distinguishes the predicted future frames from actual future frames. Moreover, by means of noise, the assistor is able to learn from pseudo abnormal future frames and predicted future frames. This helps the generator strengthen the ability to extract the discriminative features between normal and abnormal events. The motion information is used for integration into the predicted future frames. Extensive experiments are conducted on the UCSD Ped1, Ped2, CUHK Avenue and ShanghaiTech datasets. A comparison with the state-of-the-art methods shows the effectiveness and advantages of our method for anomaly detection.
... Moreover, the implicit density model based methods can be easily generalized to scenario generation tasks of various renewable energy sources and loads in different regions by fine-tuning parameters and structures [11]. The popular implicit density model based methods mainly include generative adversarial networks (GANs) [12], hidden Markov models (HMMs) [13], non-linear independent component estimations (NICEs) [14], and variational auto-encoders (VAEs) [15]. Specifically, HMMs have sound theoretical bases and simple structures, but they have difficulty in capturing the spatio-temporal characteristics of timeseries curves, since the assumption of independence of their output values leads to the lack of context information. ...
Article
Scenario generations for renewable energy sources and loads play an important role in the stable operation and risk assessment of integrated energy systems. This paper proposes a deep generative network based method to model time-series curves, e.g., power generation curves and load curves, of renewable energy sources and loads based on implicit maximum likelihood estimations (IMLEs), which can generate realistic scenarios with similar patterns as real ones. After training the model, any number of new scenarios can be obtained by simply inputting Gaussian noises into the data generator of IMLEs. The proposed approach does not require any model assumptions or prior knowledge of the form in the likelihood function being made during the training process, which leads to stronger applicability than explicit density model based methods. The extensive experiments show that the IMLEs accurately capture the complex shapes, frequency-domain characteristics, probability distributions, and correlations of renewable energy sources and loads. Moreover, the proposed approach can be easily generalized to scenario generation tasks of various renewable energy sources and loads by fine-tuning parameters and structures.
Book
Full-text available
The book’s core argument is that an artificial intelligence that could equal or exceed human intelligence—sometimes called artificial general intelligence (AGI)—is for mathematical reasons impossible. It offers two specific reasons for this claim: Human intelligence is a capability of a complex dynamic system—the human brain and central nervous system. Systems of this sort cannot be modelled mathematically in a way that allows them to operate inside a computer. In supporting their claim, the authors, Jobst Landgrebe and Barry Smith, marshal evidence from mathematics, physics, computer science, philosophy, linguistics, and biology, setting up their book around three central questions: What are the essential marks of human intelligence? What is it that researchers try to do when they attempt to achieve "artificial intelligence" (AI)? And why, after more than 50 years, are our most common interactions with AI, for example with our bank’s computers, still so unsatisfactory?
Article
While Generative Adversarial Networks (GANs) have brought immense success in various content-generation tasks, they still face enormous challenges in generating high-quality visually realistic images because of the model collapse or instability during GAN training. One common accepted explanation for the model collapse and instability is that the learning signal provided by the discriminator to the generator become inadequate when the discriminator overconcentrates on the most discriminative difference between real and synthetic images and ignores the less discriminative parts. To this end, we propose a lightweight ensemble discriminator to evaluate the generator from multi-perspective. Borrowing the insights from ensemble learning, several auxiliary discriminators are embedded into one deep model. A novel ensemble loss function is designed to promote the complementariness within the ensemble and train the whole framework in an end-to-end manner. Extensive experiments on datasets of varying resolutions and data sizes prove significant performance improvements over the state-of-the-art GANs. The proposed method can be easily embedded into various GAN frameworks and combined with different loss functions.
Chapter
The rapid development in the internet of things (IoT) produced a heavenland for attackers due to neglecting the security aspect in manufacturing. Intrusion Detection System has shown a great promise in anomaly detection traffic in the network. However, this lastest had problems because of imbalance data and the minority classes problem. This paper presents a new security model using Machine Learning. In particular, this model is based on a Multi Conditional-Task Generative Adversarial network (MCTGAN) to address the problem of class imbalance in anomaly detection System. The proposed approach use is illustrated by a case study: A Smart Healthcare System-based scenario.KeywordsAnomaly detectionGenerative Adversarial Network (GAN)Intrusion detection systemInternet of Things (IoT)Machine learning
Article
Large-scale introduction of new energy could effectively alleviate energy shortage and environmental pollution. However, the uncertainty of wind and solar energy brings serious random disturbance problems to microgrids. This paper proposes lightweight actor-critic generative adversarial networks based on ensemble empirical mode decomposition and evolutionary strategy for increasing the robustness and adaptability of microgrids. Firstly, to improve the training speed and stability of generative adversarial networks, the complex power data is properly decomposed into more regular and simpler sub-data by the ensemble empirical mode decomposition; the generative adversarial networks are optimized by the evolutionary strategy with a set of different loss functions. Secondly, fully connecting the generative adversarial networks with the actor-critic framework, the lightweight actor-critic generative adversarial networks can realize dynamic learning in the random environment and store the sample for online training by the empirical replay mechanism. Thirdly, the multi-path lightweight method is proposed to reduce the consumption of time and storage resources of lightweight actor-critic generative adversarial networks. Eventually, the lightweight actor-critic generative adversarial networks are compared with comparison algorithms in two-area and real-life four-area systems. Case study results reveal that lightweight actor-critic generative adversarial networks have better dynamic performance, online learning capabilities, and higher control performance with lower economic costs.
Article
Recently, domain adaptation has been used to solve the fault diagnosis problem in rolling bearings. However, most of the existing methods only align the distribution of domains, and ignore the fine-grained information of the same fault categories in different domains, which leads to the degradation of diagnostic performance. To address such domain difference issues, this paper proposes a novel coarse-to-fine bi-level adversarial domain adaptation approach (C2FADA) for bearings fault diagnosis. Firstly, a sparse auto-encoder (SAE) is used to extract features from raw data (containing both the source and target domains), and a Kullback-Leibler (KL) divergence term is then introduced to measure the discrepancy between the features from the source domain and the target domain. Secondly, a bi-level adversarial module is established to gradually align different domains at the domain level (with a coarse-grained model) and the class level (with a fine-grained approach) to tackle the domain shift issue, and enable the classifier to learn the domain invariant representation features. Thirdly, a spectral norm regularization constraint term is introduced to improve the stability of adversarial training process by mitigating the effect of adversarial perturbations. results show that the classification performance of the proposed C2FADA method is better than the compared existing peer methods.
Article
Generative Adversarial Networks (GANs) have been widely applied in different scenarios thanks to the development of deep neural networks. The original GAN was proposed based on the non-parametric assumption of the infinite capacity of networks. However, it is still unknown whether GANs can fit the target distribution without any prior information. Due to the overconfident assumption, many issues remain unaddressed in GANs’ training, such as non-convergence, mode collapses, gradient vanishing. Regularization and normalization are common methods of introducing prior information to stabilize training and improve discrimination. Although a handful number of regularization and normalization methods have been proposed for GANs, to the best of our knowledge, there exists no comprehensive survey which primarily focuses on objectives and development of these methods, apart from some in-comprehensive and limited scope studies. In this work, we conduct a comprehensive survey on the regularization and normalization techniques from different perspectives of GANs training. First, we systematically describe different perspectives of GANs training and thus obtain the different objectives of regularization and normalization. Based on these objectives, we propose a new taxonomy. Furthermore, we compare the performance of the mainstream methods on different datasets and investigate the applications of regularization and normalization techniques that have been frequently employed in state-of-the-art GANs. Finally, we highlight potential future directions of research in this domain. Code and studies related to the regularization and normalization of GANs in this work is summarized on https://github.com/iceli1007/GANs-Regularization-Review.
Article
The emergence and dissemination of hotspots in social networks mainly depend on the participation of group users. In this paper, considering the sparsity and complexity of effective data, we propose a group behavior dissemination model on the basis of data enhancement and data representation. First, given the inaccurate prediction results caused by the sparsity of valid data and the advantages of Generative Adversarial Networks (GAN) in learning data distribution and enhancing data, GAN is introduced to generate homomorphic data. We found that the accuracy improved by at least 6%. Second, with the diversity and complexity of the hotspot feature space and the ability of representation learning to mine hidden features of the hotspot, we designed a new method, called HP2vec, convert the feature space to a low rank and dense vector. Finally, considering the dynamic time limit of hotspot spreading, we create time slices to discretize the life of a hotspot and propose a dynamic dissemination method based on CNN. The experimental section shows that the accuracy of this method is as high as 86% in Weibo dataset and as high as 93% in twitter dataset. The model alleviates data sparseness and effectively predicts the group behavior in hotspots.
Article
Full-text available
The learning process and hyper-parameter optimization of artificial neural networks (ANNs) and deep learning (DL) architectures is considered one of the most challenging machine learning problems. Several past studies have used gradient-based back propagation methods to train DL architectures. However, gradient-based methods have major drawbacks such as stucking at local minimums in multi-objective cost functions, expensive execution time due to calculating gradient information with thousands of iterations and needing the cost functions to be continuous. Since training the ANNs and DLs is an NP-hard optimization problem, their structure and parameters optimization using the meta-heuristic (MH) algorithms has been considerably raised. MH algorithms can accurately formulate the optimal estimation of DL components (such as hyper-parameter, weights, number of layers, number of neurons, learning rate, etc.). This paper provides a comprehensive review of the optimization of ANNs and DLs using MH algorithms. In this paper, we have reviewed the latest developments in the use of MH algorithms in the DL and ANN methods, presented their disadvantages and advantages, and pointed out some research directions to fill the gaps between MHs and DL methods. Moreover, it has been explained that the evolutionary hybrid architecture still has limited applicability in the literature. Also, this paper classifies the latest MH algorithms in the literature to demonstrate their effectiveness in DL and ANN training for various applications. Most researchers tend to extend novel hybrid algorithms by combining MHs to optimize the hyper-parameters of DLs and ANNs. The development of hybrid MHs helps improving algorithms performance and capable of solving complex optimization problems. In general, the optimal performance of the MHs should be able to achieve a suitable trade-off between exploration and exploitation features. Hence, this paper tries to summarize various MH algorithms in terms of the convergence trend, exploration, exploitation, and the ability to avoid local minima. The integration of MH with DLs is expected to accelerate the training process in the coming few years. However, relevant publications in this way are still rare.
Chapter
Data-Efficient GANs (DE-GANs), which aim to learn generative models with a limited amount of training data, encounter several challenges for generating high-quality samples. Since data augmentation strategies have largely alleviated the training instability, how to further improve the generative performance of DE-GANs becomes a hotspot. Recently, contrastive learning has shown the great potential of increasing the synthesis quality of DE-GANs, yet related principles are not well explored. In this paper, we revisit and compare different contrastive learning strategies in DE-GANs, and identify (i) the current bottleneck of generative performance is the discontinuity of latent space; (ii) compared to other contrastive learning strategies, Instance-perturbation works towards latent space continuity, which brings the major improvement to DE-GANs. Based on these observations, we propose FakeCLR, which only applies contrastive learning on perturbed fake samples, and devises three related training techniques: Noise-related Latent Augmentation, Diversity-aware Queue, and Forgetting Factor of Queue. Our experimental results manifest the new state of the arts on both few-shot generation and limited-data generation. On multiple datasets, FakeCLR acquires more than 15% FID improvement compared to existing DE-GANs. Code is available at https://github.com/iceli1007/FakeCLR.
Chapter
Recently, community has paid increasing attention on model scaling and contributed to developing a model family with a wide spectrum of scales. Current methods either simply resort to a one-shot NAS manner to construct a non-structural and non-scalable model family or rely on a manual yet fixed scaling strategy to scale an unnecessarily best base model. In this paper, we bridge both two components and propose ScaleNet to jointly search base model and scaling strategy so that the scaled large model can have more promising performance. Concretely, we design a super-supernet to embody models with different spectrum of sizes (e.g., FLOPs). Then, the scaling strategy can be learned interactively with the base model via a Markov chain-based evolution algorithm and generalized to develop even larger models. To obtain a decent super-supernet, we design a hierarchical sampling strategy to enhance its training sufficiency and alleviate the disturbance. Experimental results show our scaled networks enjoy significant performance superiority on various FLOPs, but with at least \(2.53\times \) reduction on search cost. Codes are available at https://github.com/luminolx/ScaleNet.KeywordsNeural architecture search (NAS)Model scalingHierarchical sampling strategyMarkov chain-based evolution algorithm
Chapter
Vision transformers (ViTs) inherited the success of NLP but their structures have not been sufficiently investigated and optimized for visual tasks. One of the simplest solutions is to directly search the optimal one via the widely used neural architecture search (NAS) in CNNs. However, we empirically find this straightforward adaptation would encounter catastrophic failures and be frustratingly unstable for the training of superformer. In this paper, we argue that since ViTs mainly operate on token embeddings with little inductive bias, imbalance of channels for different architectures would worsen the weight-sharing assumption and cause the training instability as a result. Therefore, we develop a new cyclic weight-sharing mechanism for token embeddings of the ViTs, which enables each channel could more evenly contribute to all candidate architectures. Besides, we also propose identity shifting to alleviate the many-to-one issue in superformer and leverage weak augmentation and regularization techniques for more steady training empirically. Based on these, our proposed method, ViTAS, has achieved significant superiority in both DeiT- and Twins-based ViTs. For example, with only 1.4G FLOPs budget, our searched architecture achieves \(3.3\%\) higher accuracy than the baseline DeiT on ImageNet-1k dataset. With 3.0G FLOPs, our results achieve \(82.0\%\) accuracy on ImageNet-1k, and \(45.9\%\) mAP on COCO2017, which is \(2.4\%\) superior than other ViTs.KeywordsVision transformer (ViT)nerual architecture search (NAS)Cyclic weight sharing mechanismIdentity shiftingWeak augmentation
Chapter
Generative adversarial networks (GANs) have proven successful in image generation tasks. However, GAN training is inherently unstable. Although many works try to stabilize it by manually modifying GAN architecture, it requires much expertise. Neural architecture search (NAS) has become an attractive solution to search GANs automatically. The early NAS-GANs search only generators to reduce search complexity but lead to a sub-optimal GAN. Some recent works try to search both generator (G) and discriminator (D), but they suffer from the instability of GAN training. To alleviate the instability, we propose an efficient two-stage evolutionary algorithm-based NAS framework to search GANs, namely EAGAN. We decouple the search of G and D into two stages, where stage-1 searches G with a fixed D and adopts the many-to-one training strategy, and stage-2 searches D with the optimal G found in stage-1 and adopts the one-to-one training and weight-resetting strategies to enhance the stability of GAN training. Both stages use the non-dominated sorting method to produce Pareto-front architectures under multiple objectives (e.g., model size, Inception Score (IS), and Fréchet Inception Distance (FID)). EAGAN is applied to the unconditional image generation task and can efficiently finish the search on the CIFAR-10 dataset in 1.2 GPU days. Our searched GANs achieve competitive results (IS = 8.81 ± 0.10, FID = 9.91) on the CIFAR-10 dataset and surpass prior NAS-GANs on the STL-10 dataset (IS = 10.44 ± 0.087, FID = 22.18). Source code: https://github.com/marsggbo/EAGAN.
Chapter
Machine learning and deep learning have undoubtedly contributed to tremendous achievements in Artificial Intelligence (AI) in recent years, and more are likely to follow. They have demonstrated extraordinary superiority in various real-world applications like computer vision, medical diagnostic systems, agriculture, robotics, and many more. It enables automating the computer-aided system and drastically reducing the human workload where correct prediction with accurate precision is needed. On the other side, as technology advances, a vast amount of data is generated, raising the problem complexity and computational challenges of real-world applications. Furthermore, machine learning, deep learning, and the majority of real-world applications have complex optimization problems within themselves that must be adequately addressed for better and more accurate analysis. Nonetheless, we believe that swarm intelligence-based approaches to deep learning have traditionally been understudied and may ultimately deliver similar advances in AI capabilities - either building on those provided by deep learning or offering whole new ones. Swarm intelligence approaches are frequently employed to solve a wide range of optimization issues. Nowadays, swarm intelligence-based methods are attracting a lot of attention from the research communities of different domains because previous research in complex optimization has shown that behavioral patterns and phenomena observed in nature have the ability to facilitate the foundation for many optimization algorithms and solve problems efficiently. Swarm intelligence, machine learning, and deep learning, on the other hand, each has its own set of advantages and disadvantages. Recently, research communities have discovered an interest in integrating these concepts in order to overcome the limitations of each domain and give rise to a new paradigm known as evolutionary machine learning or evolutionary deep learning. In the case of machine learning and deep learning, the “curse of dimensionality,” non-convex optimization, automatic parameter optimization, and optimal architecture are just a few of the issues that can be efficiently addressed with swarm intelligence, whereas in the case of swarm intelligence, slow convergence, local optima stagnation, and extensive computation cost can be addressed with the machine learning and deep learning community. Therefore, a robust and self-efficient model can be developed by integrating these concepts to solve the complex problem associated with real-world applications. This hybrid approach benefits the majority of research domains. Thus, this chapter will primarily present the ideas, challenges, and recent trends of an integrative approach of swarm intelligence with deep learning, which is currently in high demand for addressing industrial problems.KeywordsSwarm intelligenceDeep learningNeuroevolutionHyperparameter optimization
Article
Generative Adversarial Network (GAN) is a novel class of deep generative models that has recently gained significant attention. However, the original GAN with one generator can easily get trapped into the mode collapsing problem, which could cause the generator only to produce similar images. This paper proposed a combination of GAN and an evolutionary algorithm to overcome the mode collapsing problem. In our approach, multiple generator networks are trained with the evolutionary strategy (ES), an evolution algorithm. The discriminator network distinguishes if the image comes from the real dataset or not. An additional classifier network is implemented to distinguish different generators. The mutations in the evolutionary strategy and the additional classifier network keep the diversity among generators. We term our approach the Evolution-GAN. In this paper, we conduct experiments on 2D synthetic data to verify that the Evolution-GAN overcomes the mode collapsing problem. Furthermore, experiments on MNIST datasets are implemented to compare the performance of Evolution-GAN, the original GAN, and Deep Convolutional GAN(DCGAN) and Evolutionary GAN.
Article
The accuracy and stability of wind power forecasting are very important for the operation of wind farms. However, for the newly built wind farms without sufficient historical data, it is difficult to make a more accurate prediction. Therefore, it is of great significance to explore a method to improve the wind power prediction accuracy with no sufficient historical data available. In this paper, a novel prediction model is proposed to address the few-shot learning problem of wind power prediction in new-built wind farms based on secondary evolutionary generative adversarial networks (SEGAN) and dual-dimension attention mechanism (DDAM) assisted bidirectional gate recurrent unit (BiGRU). The SEGAN first introduces the secondary evolutionary learning paradigm into learning GAN, aiming to learn the marginal distribution of real data and generate high-quality realistic data to augment the training dataset. In the prediction stage, the DDAM is attempted to obtain a new input matrix with global weight allocation and improve the sensitivity of the BiGRU model to the key information of the input data. The proposed SEGAN-DDAM-BiGRU model is validated on the data from the Galicia Wind Farm in Sotavento and the experimental results show that the proposed model is applicative for short-term prediction of new-built wind farms.
Article
Full-text available
A high resolution dataset is one of the prerequisites for tea chrysanthemum detection with deep learning algorithms. This is crucial for further developing a selective chrysanthemum harvesting robot. However, generating high resolution datasets of the tea chrysanthemum with complex unstructured environments is a challenge. In this context, we propose a novel tea chrysanthemum – generative adversarial network (TC-GAN) that attempts to deal with this challenge. First, we designed a non-linear mapping network for untangling the features of the underlying code. Then, a customized regularization method was used to provide fine-grained control over the image details. Finally, a gradient diversion design with multi-scale feature extraction capability was adopted to optimize the training process. The proposed TC-GAN was compared with 12 state-of-the-art generative adversarial networks, showing that an optimal average precision (AP) of 90.09% was achieved with the generated images (512 × 512) on the developed TC-YOLO object detection model under the NVIDIA Tesla P100 GPU environment. Moreover, the detection model was deployed into the embedded NVIDIA Jetson TX2 platform with 0.1 s inference time, and this edge computing device could be further developed into a perception system for selective chrysanthemum picking robots in the future.
Conference Paper
Full-text available
Compressing convolutional neural networks (CNNs) is essential for transferring the success of CNNs to a wide variety of applications to mobile devices. In contrast to directly recognizing subtle weights or filters as redundant in a given CNN, this paper presents an evolutionary method to automatically eliminate redundant convolution filters. We represent each compressed network as a binary individual of specific fitness. Then, the population is upgraded at each evolutionary iteration using genetic operations. As a result, an extremely compact CNN is generated using the fittest individual, which has the original network structure and can be directly deployed in any off-the-shelf deep learning libraries. In this approach, either large or small convolution filters can be redundant, and filters in the compressed network are more distinct. In addition, since the number of filters in each convolutional layer is reduced, the number of filter channels and the size of feature maps are also decreased, naturally improving both the compression and speed-up ratios. Experiments on benchmark deep CNN models suggest the superiority of the proposed algorithm over the state-of-the-art compression methods, e.g. combined with the parameter refining approach, we can reduce the storage requirement and the floating-point multiplications of ResNet-50 by a factor of 14.64x and 5.19x, respectively, without affecting its accuracy.
Conference Paper
Full-text available
Generative Adversarial Networks (GANs) excel at creating realistic images with complex models for which maximum likelihood is infeasible. However, the convergence of GAN training has still not been proved. We propose a two time-scale update rule (TTUR) for training GANs with stochastic gradient descent on arbitrary GAN loss functions. TTUR has an individual learning rate for both the discriminator and the generator. Using the theory of stochastic approximation, we prove that the TTUR converges under mild assumptions to a stationary local Nash equilibrium. The convergence carries over to the popular Adam optimization, for which we prove that it follows the dynamics of a heavy ball with friction and thus prefers flat minima in the objective landscape. For the evaluation of the performance of GANs at image generation, we introduce the `Fréchet Inception Distance'' (FID) which captures the similarity of generated images to real ones better than the Inception Score. In experiments, TTUR improves learning for DCGANs and Improved Wasserstein GANs (WGAN-GP) outperforming conventional GAN training on CelebA, CIFAR-10, SVHN, LSUN Bedrooms, and the One Billion Word Benchmark. https://papers.nips.cc/paper/7240-gans-trained-by-a-two-time-scale-update-rule-converge-to-a-local-nash-equilibrium
Article
Full-text available
A Triangle Generative Adversarial Network ($\Delta$-GAN) is developed for semi-supervised cross-domain joint distribution matching, where the training data consists of samples from each domain, and supervision of domain correspondence is provided by only a few paired samples. $\Delta$-GAN consists of four neural networks, two generators and two discriminators. The generators are designed to learn the two-way conditional distributions between the two domains, while the discriminators implicitly define a ternary discriminative function, which is trained to distinguish real data pairs and two kinds of fake data pairs. The generators and discriminators are trained together using adversarial learning. Under mild assumptions, in theory the joint distributions characterized by the two generators concentrate to the data distribution. In experiments, three different kinds of domain pairs are considered, image-label, image-image and image-attribute pairs. Experiments on semi-supervised image classification, image-to-image translation and attribute-based image generation demonstrate the superiority of the proposed approach.
Article
Full-text available
We propose in this paper a novel approach to tackle the problem of mode collapse encountered in generative adversarial network (GAN). Our idea is intuitive but proven to be very effective, especially in addressing some key limitations of GAN. In essence, it combines the Kullback-Leibler (KL) and reverse KL divergences into a unified objective function, thus it exploits the complementary statistical properties from these divergences to effectively diversify the estimated density in capturing multi-modes. We term our method dual discriminator generative adversarial nets (D2GAN) which, unlike GAN, has two discriminators; and together with a generator, it also has the analogy of a minimax game, wherein a discriminator rewards high scores for samples from data distribution whilst another discriminator, conversely, favoring data from the generator, and the generator produces data to fool both two discriminators. We develop theoretical analysis to show that, given the maximal discriminators, optimizing the generator of D2GAN reduces to minimizing both KL and reverse KL divergences between data distribution and the distribution induced from the data generated by the generator, hence effectively avoiding the mode collapsing problem. We conduct extensive experiments on synthetic and real-world large-scale datasets (MNIST, CIFAR-10, STL-10, ImageNet), where we have made our best effort to compare our D2GAN with the latest state-of-the-art GAN's variants in comprehensive qualitative and quantitative evaluations. The experimental results demonstrate the competitive and superior performance of our approach in generating good quality and diverse samples over baselines, and the capability of our method to scale up to ImageNet database.
Conference Paper
Full-text available
One of the challenges in the study of generative adversarial networks is the instability of its training. In this paper, we propose a novel weight normalization technique called spectral normalization to stabilize the training of the discriminator. Our new normalization technique is computationally light and easy to incorporate into existing implementations. We tested the efficacy of spectral normalization on CIFAR10, STL-10, and ILSVRC2012 dataset, and we experimentally confirmed that spectrally normalized GANs (SN-GANs) is capable of generating images of better or equal quality relative to the previous training stabilization techniques.
Article
Full-text available
Support Vector Machines are among the most powerful learning algorithms for classification tasks. However, these algorithms require a high computational cost during the training phase, which can limit their application on large-scale datasets. Moreover, it is known that their effectiveness highly depends on the hyper-parameters used to train the model. With the intention of dealing with these, this paper introduces an Evolutionary Multi-Objective Model and Instance Selection approach for support vector machines with Pareto-based Ensemble, whose goals are, precisely, to optimize the size of the training set and the classification performance attained by the selection of the instances, which can be done using either a wrapper or a filter approach. Due to the nature of multi-objective evolutionary algorithms, several Pareto optimal solutions can be found.We study several ways of using such information to perform a classification task. To accomplish this, our proposal performs a processing over the Pareto solutions in order to combine them into a single ensemble. This is done in five different ways, which are based on a global Pareto ensemble, error reduction, a complementary error reduction, maximized margin distance and boosting. Through a comprehensive experimental study we evaluate the suitability of the proposed approach and the Pareto processing, and we show its advantages over a single-objective formulation, traditional instance selection techniques and learning algorithms.
Article
Full-text available
Divide-and-Conquer (DC) is conceptually well suited to deal with high-dimensional optimization problems by decomposing the original problem into multiple low-dimensional sub-problems, and tackling them separately. Nevertheless, the dimensionality mismatch between the original problem and subproblems makes it non-trivial to precisely assess the quality of a candidate solution to a sub-problem, which has been a major hurdle for applying the idea of DC to non-separable highdimensional optimization problems. In this paper, we suggest that searching a good solution to a sub-problem can be viewed as a computationally expensive problem and can be addressed with the aid of meta-models. As a result, a novel approach, namely Self-Evaluation Evolution (SEE) is proposed. Empirical studies have shown the advantages of SEE over 4 representative compared algorithms increase with the problem size on the CEC2010 large scale global optimization benchmark. The weakness of SEE is also analysed in the empirical studies.
Article
Full-text available
The goal of this paper is not to introduce a single algorithm or method, but to make theoretical steps towards fully understanding the training dynamics of generative adversarial networks. In order to substantiate our theoretical analysis, we perform targeted experiments to verify our assumptions, illustrate our claims, and quantify the phenomena. This paper is divided into three sections. The first section introduces the problem at hand. The second section is dedicated to studying and proving rigorously the problems including instability and saturation that arize when training generative adversarial networks. The third section examines a practical and theoretically grounded direction towards solving these problems, while introducing new tools to study them.
Article
Full-text available
Generative Adversarial Networks (GAN) (Goodfellow et al., 2014) are an effective method for training generative models of complex data such as natural images. However, they are notoriously hard to train and can suffer from the problem of missing modes where the model is not able to produce examples in certain regions of the space. We propose an iterative procedure, called AdaGAN, where at every step we add a new component into a mixture model by running a GAN algorithm on a reweighted sample. This is inspired by boosting algorithms, where many potentially weak individual predictors are greedily aggregated to form a strong composite predictor. We prove that such an incremental procedure leads to convergence to the true distribution in a finite number of steps if each step is optimal, and convergence at an exponential rate otherwise. We also illustrate experimentally that this procedure addresses the problem of missing modes.
Article
Full-text available
Generative adversarial networks (GANs) are a framework for producing a generative model by way of a two-player minimax game. In this paper, we propose the \emph{Generative Multi-Adversarial Network} (GMAN), a framework that extends GANs to multiple discriminators. In previous work, the successful training of GANs requires modifying the minimax objective to accelerate training early on. In contrast, GMAN can be reliably trained with the original, untampered objective. We explore a number of design perspectives with the discriminator role ranging from formidable adversary to forgiving teacher. Image generation tasks comparing the proposed framework to standard GANs demonstrate GMAN produces higher quality samples in a fraction of the iterations when measured by a pairwise GAM-type metric.
Article
With benefits of low storage cost and fast query speed, cross-modal hashing has received considerable attention recently. However, almost all existing methods on cross-modal hashing cannot obtain powerful hash codes due to directly utilizing hand-crafted features or ignoring heterogeneous correlations across different modalities, which will greatly degrade the retrieval performance. In this paper, we propose a novel deep cross-modal hashing method to generate compact hash codes through an end-to-end deep learning architecture, which can effectively capture the intrinsic relationships between various modalities. Our architecture integrates different types of pairwise constraints to encourage the similarities of the hash codes from an intra-modal view and an inter-modal view, respectively. Moreover, additional decorrelation constraints are introduced to this architecture, thus enhancing the discriminative ability of each hash bit. Extensive experiments show that our proposed method yields state-of-the-art results on two cross-modal retrieval datasets.
Article
Due to its strong representation learning ability and its facilitation of joint learning for representation and hash codes, deep learning-to-hash has achieved promising results and is becoming increasingly popular for the large-scale approximate nearest neighbor search. However, recent studies highlight the vulnerability of deep image classifiers to adversarial examples; this also introduces profound security concerns for deep retrieval systems. Accordingly, in order to study the robustness of modern deep hashing models to adversarial perturbations, we propose hash adversary generation (HAG), a novel method of crafting adversarial examples for Hamming space search. The main goal of HAG is to generate imperceptibly perturbed examples as queries, whose nearest neighbors from a targeted hashing model are semantically irrelevant to the original queries. Extensive experiments prove that HAG can successfully craft adversarial examples with small perturbations to mislead targeted hashing models. The transferability of these perturbations under a variety of settings is also verified. Moreover, by combining heterogeneous perturbations, we further provide a simple yet effective method of constructing adversarial examples for black-box attacks.
Chapter
The recent years have witnessed significant growth in constructing robust generative models to capture informative distributions of natural data. However, it is difficult to fully exploit the distribution of complex data, like images and videos, due to the high dimensionality of ambient space. Sequentially, how to effectively guide the training of generative models is a crucial issue. In this paper, we present a subspace-based generative adversarial network (Sub-GAN) which simultaneously disentangles multiple latent subspaces and generates diverse samples correspondingly. Since the high-dimensional natural data usually lies on a union of low-dimensional subspaces which contain semantically extensive structure, Sub-GAN incorporates a novel clusterer that can interact with the generator and discriminator via subspace information. Unlike the traditional generative models, the proposed Sub-GAN can control the diversity of the generated samples via the multiplicity of the learned subspaces. Moreover, the Sub-GAN follows an unsupervised fashion to explore not only the visual classes but the latent continuous attributes. We demonstrate that our model can discover meaningful visual attributes which is hard to be annotated via strong supervision, e.g., the writing style of digits, thus avoid the mode collapse problem. Extensive experimental results show the competitive performance of the proposed method for both generating diverse images with satisfied quality and discovering discriminative latent subspaces.
Article
Style transfer describes the rendering of an image's semantic content as different artistic styles. Recently, generative adversarial networks (GANs) have emerged as an effective approach in style transfer by adversarially training the generator to synthesize convincing counterfeits. However, traditional GAN suffers from the mode collapse issue, resulting in unstable training and making style transfer quality difficult to guarantee. In addition, the GAN generator is only compatible with one style, so a series of GANs must be trained to provide users with choices to transfer more than one kind of style. In this paper, we focus on tackling these challenges and limitations to improve style transfer. We propose adversarial gated networks (Gated-GAN) to transfer multiple styles in a single model. The generative networks have three modules: an encoder, a gated transformer, and a decoder. Different styles can be achieved by passing input images through different branches of the gated transformer. To stabilize training, the encoder and decoder are combined as an auto-encoder to reconstruct the input images. The discriminative networks are used to distinguish whether the input image is a stylized or genuine image. An auxiliary classifier is used to recognize the style categories of transferred images, thereby helping the generative networks generate images in multiple styles. In addition, Gated-GAN makes it possible to explore a new style by investigating styles learned from artists or genres. Our extensive experiments demonstrate the stability and effectiveness of the proposed model for multi-style transfer.
Conference Paper
Generative Adversarial Networks (GANs) have become one of the dominant methods for fitting generative models to complicated real-life data, and even found unusual uses such as designing good cryptographic primitives. In this talk, we will first introduce the ba- sics of GANs and then discuss the fundamental statistical question about GANs — assuming the training can succeed with polynomial samples, can we have any statistical guarantees for the estimated distributions? In the work with Arora, Ge, Liang, and Zhang, we suggested a dilemma: powerful discriminators cause overfitting, whereas weak discriminators cannot detect mode collapse. Such a conundrum may be solved or alleviated by designing discrimina- tor class with strong distinguishing power against the particular generator class (instead of against all possible generators.)
Article
Neural text generation models are often autoregressive language models or seq2seq models. These models generate text by sampling words sequentially, with each word conditioned on the previous word, and are state-of-the-art for several machine translation and summarization benchmarks. These benchmarks are often defined by validation perplexity even though this is not a direct measure of the quality of the generated text. Additionally, these models are typically trained via maxi- mum likelihood and teacher forcing. These methods are well-suited to optimizing perplexity but can result in poor sample quality since generating text requires conditioning on sequences of words that may have never been observed at training time. We propose to improve sample quality using Generative Adversarial Networks (GANs), which explicitly train the generator to produce high quality samples and have shown a lot of success in image generation. GANs were originally designed to output differentiable values, so discrete language generation is challenging for them. We claim that validation perplexity alone is not indicative of the quality of text generated by a model. We introduce an actor-critic conditional GAN that fills in missing text conditioned on the surrounding context. We show qualitatively and quantitatively, evidence that this produces more realistic conditional and unconditional text samples compared to a maximum likelihood trained model.
Article
The use of multiple features has been shown to be an effective strategy for visual tracking because of their complementary contributions to appearance modeling. The key problem is how to learn a fused representation from multiple features for appearance modeling. Different features extracted from the same object should share some commonalities in their representations while each feature should also have some feature-specific representation patterns which reflect its complementarity in appearance modeling. Different from existing multi-feature sparse trackers which only consider the commonalities among the sparsity patterns of multiple features, this paper proposes a novel multiple sparse representation framework for visual tracking which jointly exploits the shared and feature-specific properties of different features by decomposing multiple sparsity patterns. Moreover, we introduce a novel online multiple metric learning to efficiently and adaptively incorporate the appearance proximity constraint, which ensures that the learned commonalities of multiple features are more representative. Experimental results on tracking benchmark videos and other challenging videos demonstrate the effectiveness of the proposed tracker.
Article
We describe a new training methodology for generative adversarial networks. The key idea is to grow both the generator and discriminator progressively: starting from a low resolution, we add new layers that model increasingly fine details as training progresses. This both speeds the training up and greatly stabilizes it, allowing us to produce images of unprecedented quality, e.g., CelebA images at 1024^2. We also propose a simple way to increase the variation in generated images, and achieve a record inception score of 8.80 in unsupervised CIFAR10. Additionally, we describe several implementation details that are important for discouraging unhealthy competition between the generator and discriminator. Finally, we suggest a new metric for evaluating GAN results, both in terms of image quality and variation. As an additional contribution, we construct a higher-quality version of the CelebA dataset.
Article
In the last 20 years, Evolutionary Algorithms (EAs) have shown to be an effective method to solve Multi-objective Optimization Problems (MOPs). Due to their population-based nature, Multi-objective Evolutionary Algorithms (MOEAs) are able to generate a set of trade-off solutions (called nondominated solutions) in a single algorithmic execution instead of having to perform a series of independent executions, as normally done with mathematical programming techniques. Additionally, MOEAs can be successfully applied to problems with difficult features such as multifrontality, discontinuity and disjoint feasible regions, among others. On the other hand, Coevolutionary algorithms (CAs) are extensions of traditional evolutionary algorithms (EAs) which have become subject of numerous studies in the last few years, particularly for dealing with large scale global optimization problems. CAs have also been applied to the solution of MOPs, motivating the development of new algorithmic and analytical formulations that have advanced the state of the art in coevolutionary algorithms research, while simultaneously opening a new research path within MOEAs. This paper presents a critical review of the most representative Coevolutionary MOEAs (CMOEAs) that have been reported in the specialized literature. This survey includes a taxonomy of approaches together with a brief description of their main features. In the final part of the paper, we also identify what we believe to be promising areas of future research in the field of CMOEAs.
Article
Generative adversarial networks (GANs) are a family of generative models that do not minimize a single training criterion. Unlike other generative models, the data distribution is learned via a game between a generator (the generative model) and a discriminator (a teacher providing training signal) that each minimize their own cost. GANs are designed to reach a Nash equilibrium at which each player cannot reduce their cost without changing the other players' parameters. One useful approach for the theory of GANs is to show that a divergence between the training distribution and the model distribution obtains its minimum value at equilibrium. Several recent research directions have been motivated by the idea that this divergence is the primary guide for the learning process and that every step of learning should decrease the divergence. We show that this view is overly restrictive. During GAN training, the discriminator provides learning signal in situations where the gradients of the divergences between distributions would not be useful. We provide empirical counterexamples to the view of GAN training as divergence minimization. Specifically, we demonstrate that GANs are able to learn distributions in situations where the divergence minimization point of view predicts they would fail. We also show that gradient penalties motivated from the divergence minimization perspective are equally helpful when applied in other contexts in which the divergence minimization perspective does not predict they would be helpful. This contributes to a growing body of evidence that GAN training may be more usefully viewed as approaching Nash equilibria via trajectories that do not necessarily minimize a specific divergence at each step.
Article
We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models: a generative model G that captures the data distribution, and a discriminative model D that estimates the probability that a sample came from the training data rather than G. The training procedure for G is to maximize the probability of D making a mistake. This framework corresponds to a minimax two-player game. In the space of arbitrary functions G and D, a unique solution exists, with G recovering the training data distribution and D equal to 1/2 everywhere. In the case where G and D are defined by multilayer perceptrons, the entire system can be trained with backpropagation. There is no need for any Markov chains or unrolled approximate inference networks during either training or generation of samples. Experiments demonstrate the potential of the framework through qualitative and quantitative evaluation of the generated samples.
Conference Paper
We introduce the "Energy-based Generative Adversarial Network" (EBGAN) model which views the discriminator in GAN framework as an energy function that associates low energies with the regions near the data manifold and higher energies everywhere else. Similar to the probabilistic GANs, a generator is trained to produce contrastive samples with minimal energies, while the energy function is trained to assign high energies to those generated samples. Viewing the discriminator as an energy function allows to use a wide variety of architectures and loss functionals in addition to the usual binary discriminant network. Among them, an instantiation of EBGANs is to use an auto-encoder architecture alongside the energy being the reconstruction error. We show that this form of EBGAN exhibits more stable behavior than regular GANs during training. We also show that a single-scale architecture can be trained to generate high-resolution images.
Conference Paper
In recent years, supervised learning with convolutional networks (CNNs) has seen huge adoption in computer vision applications. Comparatively, unsupervised learning with CNNs has received less attention. In this work we hope to help bridge the gap between the success of CNNs for supervised learning and unsupervised learning. We introduce a class of CNNs called deep convolutional generative adversarial networks (DCGANs), that have certain architectural constraints, and demonstrate that they are a strong candidate for unsupervised learning. Training on various image datasets, we show convincing evidence that our deep convolutional adversarial pair learns a hierarchy of representations from object parts to scenes in both the generator and discriminator. Additionally, we use the learned features for novel tasks - demonstrating their applicability as general image representations.
Article
The problem of maximizing monotone k -submodular functions under a size constraint arises in many applications, and it is NP-hard. In this paper, we propose a new approach which employs a multiobjective evolutionary algorithm to maximize the given objective and minimize the size simultaneously. For general cases, we prove that the proposed method can obtain the asymptotically tight approximation guarantee, which was also achieved by the greedy algorithm. Moreover, we further give instances where the proposed approach performs better than the greedy algorithm on applications of influence maximization, information coverage maximization, and sensor placement. Experimental results on real-world data sets exhibit the superior performance of the proposed approach.
Article
Combinatorial testing can test software that has various configurations for multiple parameters efficiently. This method is based on a set of test cases that guarantee a certain level of interaction among parameters. Mixed covering array can be used to represent a test-suite. Each row of the array corresponds to a test case. In general, a smaller size of mixed covering array does not necessarily imply less testing time. There are certain combinations of parameter values which would take much longer time than other cases. Based on this observation, it is more valuable to construct mixed covering arrays that are better in terms of testing effort characterization other than size. We present a method to find cost-aware mixed covering arrays. The method contains two steps. First, simulated annealing algorithm is used to get a mixed covering array with a small size. Then we propose a novel nested differential evolution algorithm to improve the solution with its testing effort. The experimental results indicate that our method succeeds in constructing cost-aware mixed covering arrays for real-world applications. The testing effort is significantly reduced compared with representative state-of-the-art algorithms.
Conference Paper
In this paper, we propose a principled Tag Disentangled Generative Adversarial Networks (TD-GAN) for re-rendering new images for the object of interest from a single image of it by specifying multiple scene properties (such as viewpoint, illumination, expression, etc.). The whole framework consists of a disentangling network, a generative network, a tag mapping net, and a discriminative network, which are trained jointly based on a given set of images that are completely/partially tagged (i.e., supervised/semi-supervised setting). Given an input image, the disentangling network extracts disentangled and interpretable representations, which are then used to generate images by the generative network. In order to boost the quality of disentangled representations, the tag mapping net is integrated to explore the consistency between the image and its tags. Furthermore, the discriminative network is introduced to implement the adversarial training strategy for generating more realistic images. Experiments on two challenging datasets demonstrate the state-of-the-art performance of the proposed framework in the problem of interest.
Conference Paper
Hashing has been a widely-adopted technique for nearest neighbor search in large-scale image retrieval tasks. Recent research has shown that leveraging supervised information can lead to high quality hashing. However, the cost of annotating data is often an obstacle when applying supervised hashing to a new domain. Moreover, the results can suffer from the robustness problem as the data at training and test stage may come from different distributions. This paper studies the exploration of generating synthetic data through semi-supervised generative adversarial networks (GANs), which leverages largely unlabeled and limited labeled training data to produce highly compelling data with intrinsic invariance and global coherence, for better understanding statistical structures of natural data. We demonstrate that the above two limitations can be well mitigated by applying the synthetic data for hashing. Specifically, a novel deep semantic hashing with GANs (DSH-GANs) is presented, which mainly consists of four components: a deep convolution neural networks (CNN) for learning image representations, an adversary stream to distinguish synthetic images from real ones, a hash stream for encoding image representations to hash codes and a classification stream. The whole architecture is trained end-to-end by jointly optimizing three losses, i.e., adversarial loss to correct label of synthetic or real for each sample, triplet ranking loss to preserve the relative similarity ordering in the input real-synthetic triplets and classification loss to classify each sample accurately. Extensive experiments conducted on both CIFAR-10 and NUS-WIDE image benchmarks validate the capability of exploiting synthetic images for hashing. Our framework also achieves superior results when compared to state-of-the-art deep hash models.
Article
An effective allocation of search effort is important in multi-objective optimization, particularly in many-objective optimization problems. This paper presents a new adaptive search effort allocation strategy for MOEA/D-M2M, a recent MOEA/D algorithm for challenging Many-Objective Optimization Problems (MaOPs). This proposed method adaptively adjusts the subregions of its subproblems by detecting the importance of different objectives in an adaptive manner. More specifically, it periodically resets the subregion setting based on the distribution of the current solutions in the objective space such that the search effort is not wasted on unpromising regions. The basic idea is that the current population can be regarded as an approximation to the Pareto front (PF) and thus one can implicitly estimate the shape of the PF and such estimation can be used for adjusting the search focus. The performance of proposed algorithm has been verified by comparing it with eight representative and competitive algorithms on a set of degenerated many-objective optimization problems with disconnected and connected PFs. Performances of the proposed algorithm on a number of non-degenerated test instances with connected and disconnected PFs are also studied.
Article
Despite their growing prominence, optimization in generative adversarial networks (GANs) is still a poorly-understood topic. In this paper, we analyze the "gradient descent" form of GAN optimization (i.e., the natural setting where we simultaneously take small gradient steps in both generator and discriminator paramete