CDE-GAN: Cooperative Dual Evolution Based Generative Adversarial Network

Abstract

Generative adversarial networks (GANs) have been a popular deep generative model for real-word applications. Despite many recent efforts on GANs have been contributed, however, mode collapse and instability of GANs are still open problems caused by their adversarial optimization difficulties. In this paper, motivated by the cooperative co-evolutionary algorithm, we propose a Cooperative Dual Evolution based Generative Adversarial Network (CDE-GAN) to circumvent these drawbacks. In essence, CDE-GAN incorporates dual evolution with respect to generator(s) and discriminators into a unified evolutionary adversarial framework, thus it exploits the complementary properties and injects dual mutation diversity into training to steadily diversify the estimated density in capturing multi-modes, and to improve generative performance. Specifically, CDE-GAN decomposes the complex adversarial optimization problem into two subproblems (generation and discrimination), and each subproblem is solved with a separated subpopulation (E-Generators and E-Discriminators), evolved by an individual evolutionary algorithm. Additionally, to keep the balance between E-Generators and E-Discriminators, we proposed a Soft Mechanism to cooperate them to conduct effective adversarial training. Extensive experiments on one synthetic dataset and three real-world benchmark image datasets, demonstrate that the proposed CDE-GAN achieves the competitive and superior performance in generating good quality and diverse samples over baselines. The code and more generated results are available at our project homepage
(Cite as: arXiv:2008.09388)

Full text

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 1
CDE-GAN: Cooperative Dual Evolution Based
Generative Adversarial Network
Shiming Chen, Wenjie Wang, Beihao Xia, Xinge You, Senior Member, IEEE, Qinmu Peng, Zehong Cao,
and Weiping Ding, Senior Member, IEEE
Abstract—Generative adversarial networks (GANs) have been
a popular deep generative model for real-world applications.
Despite many recent efforts on GANs that have been contributed,
mode collapse and instability of GANs are still open problems
caused by their adversarial optimization difficulties. In this
paper, motivated by the cooperative co-evolutionary algorithm,
we propose a Cooperative Dual Evolution based Generative Ad-
versarial Network (CDE-GAN) to circumvent these drawbacks.
In essence, CDE-GAN incorporates dual evolution with respect
to the generator(s) and discriminators into a unified evolu-
tionary adversarial framework to conduct effective adversarial
multi-objective optimization. Thus it exploits the complementary
properties and injects dual mutation diversity into training to
steadily diversify the estimated density in capturing multi-modes
and improve generative performance. Specifically, CDE-GAN
decomposes the complex adversarial optimization problem into
two subproblems (generation and discrimination), and each sub-
problem is solved with a separated subpopulation (E-Generators
and E-Discriminators), evolved by its own evolutionary algorithm.
Additionally, we further propose a Soft Mechanism to balance the
trade-off between E-Generators and E-Discriminators to conduct
steady training for CDE-GAN. Extensive experiments on one
synthetic dataset and three real-world benchmark image datasets
demonstrate that the proposed CDE-GAN achieves a competitive
and superior performance in generating good quality and diverse
samples over baselines. The code and more generated results are
available at our project homepage https://shiming-chen.github.io/
CDE-GAN-website/CDE-GAN.html.
Index Terms—Generative adversarial networks (GANs), evolu-
tionary computation (EC), cooperative co-evolutionary algorithm,
cooperative dual evolution, multi-objective optimization.
I. INTRODUCTION AND MOTIVAT IO N
G
ENERATIVE adversarial networks (GANs) [1] are new
popular methods for generative modeling, using game-
theoretic training schemes to implicitly learn a given probability
This work was supported in part by the National Natural Science Foundation
of China (61571205, 61772220 and 61976120), the Key Program for
International S&T Cooperation Projects of China (2016YFE0121200), the
Special Projects for Technology Innovation of Hubei Province (2018ACA135),
the Key Science and Technology Innovation Program of Hubei
Province (2017AAA017), the Natural Science Foundation of Jiangsu
Province BK (20191445), the Natural Science Foundation of Hubei
Province (2018CFB691), fund from Science, Technology and Innovation
Commission of Shenzhen Municipality (JCYJ20180305180804836 and
JSGG20180507182030600).(Corresponding author: Xinge You.)
S. Chen, W. Wang, Beihao Xia, X. You and Q. Peng are with the Department
of Electronic Information and Communication, Huazhong University of Science
and Technology, Wuhan 430074, China. (e-mail:gchenshiming@gmail.com;
{shimingchen, wangwj54, xbh_hust, youxg, pengqinmu}@hust.edu.cn.
Z. Cao is is with the Discipline of ICT, University of Tasmania, TAS 7001,
Australia (e-mail:zehong.cao@utas.edu.au or e-mail:zehong.cao@utas.edu.au).
W. Ding is with the School of Information Science and Technology, Nantong
University, Nantong, China (ding.wp@ntu.edu.cn).
density. With the potential power of capturing high dimen-
sional probability distributions, GANs have been successfully
deployed for various synthesis tasks, e.g., image generation
[2], video prediction [3], [4], text synthesis [5], [6].
A GAN consists of a framework describing the interaction
between two different models, i.e., generator and discrimina-
tor, which are used to solve a min-max game optimization
problem. During learning, the generator tries to learn real
data distribution by generating realistic-looking samples that
can fool the discriminator, while the discriminator attempts to
differentiate between samples from the data distribution and
the ones produced by the generator. Although the pioneered
GAN provided some analysis on the convergence properties
of the approach [1], [7], it assumed that updates occurred in
pure function space, allowed arbitrarily powerful generator and
discriminator networks, and modeled the resulting optimization
objective as a convex-concave gam. Therefore it is yielding well-
defined global convergence properties. Besides, this analysis
assumed that the discriminator network is fully optimized
between generator updates. However, these assumptions do not
mirror the practice of GAN optimization. In pratice, there exist
many well-documented failure GAN models caused by mode
collapse1or instability2.
Therefore, many recent efforts on GANs have contributed to
overcome these optimization difficulties by developing various
adversarial training approaches, i.e., modifying the optimization
objective, training additional discriminators, training multiple
generators, and using evolutionary computation. The first
method is a typical way to control the optimization gradient of
discriminator or generator parameters, which would help GANs
to steadily arrive at the equilibrium points of the optimization
procedure under proper conditions [11]–[15]. Compared to the
traditional GANs performed single-objective optimization, the
second method revisits the multiple-discriminator setting by
framing the simultaneous optimization of different discriminator
models as a multi-objective optimization problem. Thus it
would overcome the problem involving the lack of informative
gradient (i.e., the stable gradient is neither vanishing nor
exploding for the generator throughout training [9], [14].) signal
provided by discriminator [9], [16]–[19]. The third method
simultaneously trains multiple generators with the object that
mixture of their induced distributions would approximate the
1
the generator can only learn some limited patterns from the large-scale
given datasets, or assigns all of its probability mass to a small region in the
space [8]
2
the discriminator can easily distinguish between real and fake samples
during the training phase [9]–[11]

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 2
data distribution, and thus mode collapse problem can be
alleviated [8], [20]–[22]. However, the aforementioned GAN
methods are limited by the specified adversarial optimization
strategy. The last method introduced evolutionary computa-
tion for the optimization of GANs to improve generative
performance [6], [23]–[26]. In fact, the existing evolutionary
GANs evolve a population of generators (e.g., E-GAN [24],
CatGAN [6]) or GANs (Lipizzar [23], Mustangs [26]) to play
the adversarial game, which will result that GANs evolve in
a static environment
3
. In essential, training GANs is a large-
scale optimization problem, which is challenging [19], [24].
This is why the most existing GAN methods usually fall in
instability and mode collapse. To overcome these challenges,
we propose a novel insight for GANs, in which a cooperative
dual evolution paradigm is proposed to conduct adversarial
multi-objective optimization for GANs.
Recently, evolutionary computation (EC) has been used to
solve many deep learning challenges, e.g., optimizing deep
learning hyper-parameters [27]–[31] and designing network
architecture [32], [33]. Meanwhile, researchers also attempt to
apply EC on GANs to improve the robust of GANs for miti-
gating degenerate GAN dynamics [6], [23]–[26]. Among them,
Lipizzaner [23] and Mustangs [26] use a spatial distributed
competitive co-evolutionary algorithm to provide diversity in
the genome space for GANs. In [24], Wang introduced E-GAN
that injects diversity into the training with the evolutionary
population. Accordingly, we attempt to train GANs using a
cooperative dual evolutionary paradigm, which is effective for
large-scale optimization task and diversity learning [34]–[41].
In light of the above observation, we propose a cooper-
ative dual evolution based generative adversarial network,
called CDE-GAN, to train the model steadily and improve
generation performance effectively. In essence, CDE-GAN
incorporates dual evolution with respect to the generator(s) and
discriminators into a unified evolutionary adversarial framework
to conduct effective adversarial multi-objective optimization.
Thus, it exploits the complementary properties and injects
dual mutation diversity into training to steadily diversify the
estimated density in capturing multi-modes and to improve
the generative performance. Our strategy towards achieving
this goal is to decompose the complex adversarial optimization
problem into two subproblems (generation and discrimination),
and each subproblem is solved with a separated subpopu-
lation (E-Generators and E-Discriminators), evolved by its
own evolutionary algorithm, including individual variations
(mutations), evaluation (fitness function), and selection. Unlike
the existing EC based GANs [23], [24], [26], CDE-GAN
simultaneously evolves a population of generators and an array
of discriminators by operating their various objective functions
(mutations) that are interpretable and complementary. During
training E-Generators, acting as parents, generators of CDE-
GAN undergo different mutations to produce offspring to adapt
to the dynamic environment (E-Discriminators). Meanwhile,
we term the quality and diversity of samples generated
by the evolved offspring as a fitness score for evaluating
3
when evolving the generators of GAN, its discriminator acts as the
environment, e.g., [24]; and vice versa.
the offspring’s performance. According to the fitness score,
poorly performing offspring are removed, and the remaining
well-performing offspring are preserved and used for next-
generation training (i.e., evolution). Given optimal generators,
the similar mechanism holds for E-Discriminators with its own
evolutionary algorithm, and thus discriminators provide more
informative gradient to generators for distribution diversity
learning
4
. To keep the balance between E-Generators and E-
Discriminators, we proposed a Soft Mechanism to bridge them
to conduct effective and stable adversarial training. In this way,
CDE-GAN possesses three key benefits: 1) the cooperative dual
evolution (E-Generators and E-Discriminators) injects diversity
into training so that CDE-GAN can cover different data modes,
which significantly mitigates mode collapse of GANs; 2)
CDE-GAN terms adversarial training as an adversarial multi-
objective optimization problem, the multiple discriminators
provide informative feedback gradient to generators for stabi-
lizing the training process; 3) the complementary mutations in
E-Generators and E-Discriminators will help model place fair
distribution of probability mass across the modes of the data
generating distribution.
To summarize, this study makes the following salient
contributions:
•
We propose a novel GAN method, termed cooperative
dual evolution based generative adversarial network (CDE-
GAN), to circumvent adversarial optimization difficulties
of GANs, i.e., mode collapse and instability. To achieve
this goal, CDE-GAN incorporates dual evolution with
respect to the generator(s) and discriminators into a unified
evolutionary adversarial framework to conduct effective
adversarial multi-objective optimization. Thus it exploits
the complementary properties and injects dual mutation
diversity into training to steadily diversify the estimated
density in capturing multi-modes and improve generative
performance.
•
We design E-Generators and E-Discriminators, which
evolve respectively by their own evolutionary algorithms,
to solve the two subproblems (i.e., generation and dis-
crimination) of CDE-GAN. To keep the balance between
E-Generators and E-Discriminators, we further propose a
Soft Mechanism to cooperate them to conduct effective
adversarial training.
•
We carry out extensive experiments on several benchmark
datasets to demonstrate that our proposed method achieved
obvious advantages over the existing methods, which
proves the superiority and great potentials of CDE-GAN.
The remainder of this paper is organized as follows. Section
II gives related works in the field of generative adversarial
networks. The proposed CDE-GAN is illustrated in Section
III. The performance and evaluation are provided in Section
IV. Section Vpresents the discussion. Section VI provides a
summary and the outlook for future research.
4
Distribution diversity learning denotes that generator of GAN can learn the
diversity distribution that covers different data modes of true data distribution
[7], [20].

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 3
II. RE LATE D WOR K
In the following, we provide a review of GANs that
developed various adversarial training approaches to overcome
the optimization difficulties based on different methods.
A. Modifying Training Objective for GANs
Modifying training objective of GANs is a typical way
to improve and stabilize the optimization of GANs. Radford
et al. [42] introduced DCGAN to improve training stability.
In [11], Arjovsky not only proposed Wasserstein-GAN to
minimize a reasonable and efficient approximation of the
Earth Mover (EM) distance for promoting the stability of
training, he but also theoretically showed the corresponding
optimization problem. Meanwhile, Metz et al. [12] introduced a
method to stabilize GANs and increase diversity by defining the
generator objective with respect to an unrolled optimization of
the discriminator. Based on the idea that gradient signals from
Denoising AutoEncoder (DAE) can guide the generator towards
producing samples whose activations are close to the manifold
of real data activations, Denoising Feature Matching (DFM)
is proposed to improve GANs training [13]. SN-GAN called
spectral normalization to stabilize the discriminator’s training
and achieved promising generative performance [15]. Although
some of these methods are practically and theoretically well-
founded, convergence remains elusive in practice.
B. Multi-discriminator for GANs
Unlike the traditional GANs performed single-objective
optimization, some works attempt to revisit the multiple
discriminators setting by framing the simultaneous optimization
of different discriminator models as a multi-objective optimiza-
tion problem. Thus it will overcome the problem of lacking
informative gradient signal provided by the discriminators.
Nguyen et al. [16] proposed D2GAN, combing the KL and
reverse KL divergences into a unified objective function, to
exploits the complementary statistical properties from these
divergences to diversify the estimated density in capturing
multi-modes effectively. Durugkar et al. [17] simultaneously
introduced multiple discriminators into the GAN framework
to weaken the discriminators of GMAN, which provides
informative feedback to generator and better guides generator
towards amassing distribution in approximately true data region.
In [9], Neyshabur proposed an array of discriminators, each of
which looks at a different random low-dimensional projection
of the data, to play the adversarial game with a single generator.
Thus the individual discriminators are failed to reject generated
samples perfectly. In [18], Doan argued that less expressive
discriminators are smoother and have a general coarse-grained
view of the mode’s map, which enforces the generator to
cover a wide region of the true data distribution. Albuquerque
et al. [19] framed the training of multi-discriminator based
GANs as a multi-objective optimization problem and analyzed
its effectiveness. Although multi-discriminator based GANs
perform promising results, they neglect to mine the prominence
of the generator further.
C. Multi-generator for GANs
Multi-generator based GANs simultaneously trained multiple
generators with the object that a mixture of their induced
distributions would approximate the data distribution. Thus,
the mode collapse problem can be alleviated. Motivated by the
boosting method, Tolstikhin et al. [20] trained a mixture of
generators by sequentially training and adding new generators
to the mixture. Arora et al. [8] introduced a MIX+GAN
framework to optimize the minimax game with the reward
function being the weighted average reward function between
any pair of generator and discriminator. In [22], Ghost proposed
MAD-GAN that trains a set of generators using a multi-class
discriminator, which predicts which generator produces the
sample and detecting whether the sample is fake. Additionally,
MGAN [21] was developed to overcome the mode collapsing
problem, and its theoretical analysis was provided. Indeed,
multi-generator based GANs break the balance of generator
and discriminator. Thus, the additional supervised information
is typically used to steady adversarial training of GANs.
D. Evolutionary Computation for GANs
In fact, the aforementioned GAN methods are limited by
the specified adversarial optimization strategy, which heavily
limits optimization performance during training. Since EC
has been successfully applied to solve many deep learning
challenges [27]–[31], some researchers attempt to overcome
different training problems of GANs using the EC technique.
In [23], Schmiedlechner proposed Lipizzaner, which provides
population diversity by training a two-dimensional grid (each
cell contains a pair of generator-discriminator) of GANs
with a distributed evolutionary algorithm. Wang et al. [24]
introduced E-GAN to inject mutation diversity into adversarial
optimization of GANs by training the generator with three
independent objective functions then selecting the resulting
best performing generator for the next batch. Based on E-GAN,
Liu et al. developed CatGAN with hierarchical evolutionary
learning for category text generation. In [26], Toutouh proposed
Mustangs, hybridizing E-GAN and Lipizzaner, to combine
mutation and population approaches to diversity improvement
of GANs. Costa et al. [25] developed COEGAN, using neuro-
evolution and coevolution in the GAN training, to provide
a more stable training method and the automatic design of
neural network architectures. However, the aforementioned
evolutionary GANs evolve a population of generators or GANs
to play the adversarial game, which will result that a GAN
evolves in a static environment. Thus the evolutionary dynamic
of GANs is limited, and the balance between generator and
discriminator of GANs is unstable. To this end, we propose
a cooperative dual evolution based GAN to conduct effective
adversarial multi-objective optimization. It exploits the com-
plementary properties and injects dual mutation diversity into
training to steadily diversify the estimated density in capturing
multi-modes and to improve the generative performance.
III. PROP OS ED ME TH OD
Motivated by the success of cooperative co-evolutionary
algorithm in large-scale optimization and diversity learning

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 4
Fig. 1: The pipeline of CDE-GAN. In brief, CDE-GAN decomposes the complex adversarial optimization problem into two
subproblems (generation and discrimination), and each subproblem is solved with a separated subpopulation (i.e., E-Generators
and E-Discriminators), evolved by its own evolutionary algorithm. The best offspring of E-Generators and E-Discriminator are
served as new parents to produce the next generation’s offspring during training. Furthermore, a Soft Mechanism is proposed to
cooperate E-Generators and E-Discriminators to conduct effective adversarial training.
[34]–[41], in this paper, we propose a Cooperative Dual
Evolution based Generative Adversarial Network (CDE-GAN)
to circumvent drawbacks (i.e., instability and mode collapse) of
GANs. In essence, CDE-GAN incorporates dual evolution with
respect to generator and discriminator into a unified evolution-
ary adversarial framework to conduct effective adversarial multi-
objective optimization. Thus it exploits the complementary
properties and injects dual mutation diversity into training
to steadily diversify the estimated density in capturing multi-
modes and improve the generative performance. As shown
in Figure 1, CDE-GAN decomposes the complex adversarial
optimization problem into two subproblems (generation and
discrimination), and each subproblem is solved with a sepa-
rated subpopulation (i.e., E-Generators and E-Discriminators),
evolved by its own evolutionary algorithm during training.
Specifically, we first propose a Soft Mechanism to keep
the balance between E-Generators and E-Discriminators and
cooperate them to conduct effective adversarial training. Then,
we introduce E-Generators and E-Discriminators, including
their own Variations,Evaluation and Selection.
A. Revisiting GANs
A GAN [1] consists of a framework describing the inter-
action between two different models, i.e., generator (
G
) and
discriminator (
D
), which are used to solve a min-max game
optimization problem. Taking noisy sample
z∼pz
as input,
G
tries to learn real data distribution
pdata(x)
by generating
realistic looking samples
x∼pG(x)
that are able to fool
D
,
while
D
attempts to differentiate between samples from the
data distribution and the ones produced by
G
. Mathematically,
GAN [1] is formulated as
min
Gmax
D
Ex∼pdata [log D(x)] + Ez∼pz[log(1 −D(G(z)))]
(1)
Most existing GANs conduct a similar adversarial procedure
with different adversarial objectives.
B. Soft Mechanism
For the sake of keeping the balance between E-Generators
and E-Discriminators, we proposed a Soft Mechanism to bridge
them to conduct effective adversarial training. In practice, the
generator’s learning will be impeded when it trains over a far
superior discriminator. That is, the generator is unlikely to
generate any samples considered "realistic" according to the
discriminator’s standards, and thus the generator will receive
uniformly negative feedback [17]. This is problematic for GANs
because the information contained in the gradient derived
from negative feedback only dictates where to drive down
the generated distribution
pG(x)
, not specifically where to
increase
pG(x)
. Furthermore, it inevitably increases
pG(x)
in
other regions of true data distribution
pdata(x)
to maintain
RXpG(x)=1
, which may or may not contain samples from
the true dataset (whack-a-mole dilemma). To this end, the
generator and discriminator can not be well balanced for
effective adversarial training. In fact, the degenerate results of
GANs can be avoided by employing learner (discriminator)
with limited capacity and corrupting data samples with noise
[9], [17], [18]. To this end, a generator is more likely to
meet positive feedback against a more lenient discriminator,
which may better guide a generator towards amassing
pG(x)
in approximately correct regions of pdata(x).
Inspired by this observation, we use a Soft Mechanism of
the classical Pythagorean method parameterized by
δ
to soften
the maximization of discriminators. Thus discriminators of
CDE-GAN will be well weaken, which avoids the problem
of whack-a-mole dilemma during training and enables us to
obtain a diverse set of seemingly tenable hypotheses for the
true data distribution
pdata(x)
. Furthermore, using softmax has
a well-known benefit of being differentiable. Specifically, given

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 5
the optimal E-Discriminators
D∗
φ
, the generator(s)
Gθj
trains
against a softmax weighted arithmetic average of
I
different
discriminators. It can be formulated as
LGθj(δ) =
I
X
i=1
wili,(2)
where
wi=eδli
PI
t=1 eδlt
,
li=MG(Dφi)
,
MG∈
{Mminimax
G,Mheuristic
G,Mls
G}
is the variation (mutation) of
the evolutionary algorithm in E-Generators (see Section
III-C
1
for more details), and
δ≥0
. When
δ→ ∞
,
Gθj
trains against
only a single weak discriminator; when
δ= 0
,
Gθj
trains
against an ensemble with equal weights. These two scenes are
not desired. Here, we set δ= 1 for our all experiments.
C. E-Generators
In fact, the most existing GAN methods (e.g., modifying
training objective based GANs, multi-discriminator based
GANs, and multi-generator based GANs) are limited by the
specified adversarial optimization strategy, which heavily
affects optimization performance during training. To this end,
we build E-Generators that evolves a set of generators (parents)
{Gθ1, Gθ2,· · · , GθJ}
with number of
J
in a given dynamic
environment (E-Discriminators) based on an evolutionary
algorithm to produce a set of new generators (offspring)
{Gθ1,1, Gθ1,2,· · · , Gθ1,M ,· · · , GθJ,1, GθJ,2,· · · , GθJ,M }
.
E-Generators is an individual subpopulation to solved
a subproblem (generation), which is cooperative with
E-Discriminators to solve the adversarial multi-objective
optimization problem of GANs. The optimization of
E-Generators is formulated as
min LGθ=LGθ1,LGθ2,· · · ,LGθJT,(3)
where LGθjis defined in Eq. 2, and θ={θ1, θ2,· · · , θJ}.
Based on the evolutionary algorithms, we take G-Variations
(mutations), G-Evaluation (fitness function) and G-Selection
(selection) for evolving E-Generators. The evolutionary process
of E-Generators is presented in Fig. 1and Algorithm 1.
1) G-Variation: CDE-GAN applies three complementary
mutations corresponding with three different minimization
objective w.r.t generator for the evolution of E-Generators,
i.e., G-Minimax mutation (
Mminimax
G
), G-Heuristic mutation
(
Mheuristic
G
), and G-Least-Square mutation (
Mls
G
). They are
corresponding to vanilla GAN (GAN) [1], non-saturated GAN
(NS-GAN) [1], and least square GAN (LSGAN) [43], [44],
respectively. In contrast to mutations of E-GAN involving gen-
erator(s) against a single specified discriminator, the mutations
of E-Generators train over multiple evolutionary discriminators.
The G-Minimax mutation corresponds to the minimization
objective of the generator in the vanilla GAN [1]. According
to Eq. 2, it is defined as
Mminimax
G=1
2
I
X
i=1
wiEz∼pz[log(1 −Dφi(Gθ(z)))].(4)
In fact, G-Minimax mutation is to minimize the Jensen-
Shannon Divergence (JSD) between the data and model
distributions. If discriminators distinguish generated samples
with high confidence, the gradient tend to vanish, G-Minimax
mutation fails to work; if discriminators cannot completely
distinguish real/fake sample, the G-Minimax mutation will
provide informative gradient for adversarial training. Thus, G-
Minimax mutation typically evolves the best offspring in the
latter training process for CDE-GAN.
Additionally, G-Heuristic mutation is non-saturating when
the discriminator convincingly rejects the generated samples,
and thus it avoids gradient vanish. According to Eq. 2, it is
formulated as follow:
Mheuristic
G=−1
2
I
X
i=1
wiEz∼pz[log(Dφi(Gθ(z)))].(5)
However, G-Heuristic mutation may direct to training instability
and generative quality fluctuations because it pushes the data
and model distributions away each other.
As for G-Least-Square mutation, it is inspired by LSGAN
[43], which applies this criterion to adapt both generator and
discriminator. According to Eq. 2, it can be written as
Mls
G=
I
X
i=1
wiEz∼pz[(Dφi(Gθ(z)−1)2].(6)
Similar to G-Heuristic, G-Least-Square mutation will effec-
tively avoid gradient vanish when the discriminator easily
recognizes the generated samples. Meanwhile, G-Least-Squares
mutation will partly avoid mode collapse, because it neither
assigns an extremely high cost to generate fake samples nor
assigns an extremely low cost to mode dropping. Thus, these
different mutations provide various training strategies for E-
Generators, which injects mutation diversity into training to
diversify the estimated density in capturing multi-modes and
constructs a complementary population of generators for steady
training. See [1], [24], [43], [45] for more corresponding
theoretical analysis of these initial objective functions.
2) G-Evaluation: After producing the offspring with differ-
ent mutations, we evaluate the individual’s quality for each
child using a fitness function
FG
that depends on the current
environment (i.e., E-Discriminators
Dφ
). Considering two
typical properties (quality
5
and diversity
6
) of generated sample,
FG
consists of two fitness scores, i.e., quality fitness scores
FGq
and diversity fitness score
FGd
, for evaluating the performance
of the offspring (generators) of E-Generators. On the one hand,
the generated samples produced by generator are feed into
discriminators and the sum value of the output are calculated,
which is termed as FGq:
FGq=
I
X
i=1
wiEz∼pz[Dφi(Gθ(z))].(7)
The higher quality score generators achieved, the more reality
generated samples gotten. Reflecting the quality performance
of generators at each evolutionary step, discriminators are
constantly upgraded to be optimal during the training process.
5
the generated samples are so realistic enough that it will fool the superior
discriminator. [46]
6
the model distribution is more possible to cover the real data distribution.
It could largely avoid mode collapse. [47]

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 6
Algorithm 1 The algotithm of CDE-GAN.
Require:
The generator
Gθ
; the discriminator
Dφ
; the number of iterations
T
; the discriminator’s updating steps per iteration
K
; the number of parents for E-Generator
J
; the number of parents for E-Discriminator
I
; the number of mutations for
generator M; the number of mutations for discriminator N; the hyper-parameter γof fitness function of E-Generators.*
1: Initialize generators’ parameter {θ1, θ2,· · · , θJ}, initialize discriminators’ parameter {φ1, φ2,· · · , φI}.
2: for t= 1,· · · , T do
3: for k= 1,· · · , K do
4: for i= 1,· · · , I do E-Discriminators Evolution
5: Sample a batch of xreal ∼pdata.
6: Sample a batch of z∼pz, and generate a batch of xfake with E-Generators.
7: Dφiproduces Noffspring Dφi,n via D-Variation, that is, updating Dφivia Eq. 13 and Eq. 14 respectively. 
D-Variation
8: Evaluate the Nevolved offspring of E-Discriminators via Eq. 15.D-Evaluation
9: end for
10:
Select the best-performing offspring
{Dφ1,1, Dφ2,1, . . . , DφI,1}
as next generation’s parents of E-Discriminators via
Eq. 16 and Eq. 17.D-Selection
11: end for
12: for j= 1,· · · , J do E-Generators Evolution
13: Sample a batch of z∼pz.
14: Gθj
produces
M
offspring
Gθj,m
via G-Variation, that is, updating
Gθj
via Eq. 4, Eq. 5and Eq. 6respectively.

G-Variation
15: Evaluate the Mevolved offspring of E-Generators via Eq. 9.G-Evaluation
16: end for
17:
Select the best-performing offspring
{Gθ1,1, Gθ2,1, . . . , GθJ,1}
as next generation’s parents of E-Generators via Eq. 10
and Eq. 11.G-Selection
18: end for
*Default values: B= 32,K= 3,M= 3 and N= 2.
On the other hand, we also focus on the diversity of generated
samples and attempt to gather a better group of generators to
circumvent the mode collapse issue in adversarial optimization
of GANs. According to [48], a gradient-based regularization
term can stabilize the GAN optimization and suppress mode
collapse. To this end, the minus log-gradient-norm of opti-
mizing
Dφ
is used to measure the diversity fitness score of
generated samples
FGd=−
I
X
i=1
wi(log ||∇Dφi−Ex∼pdata [log Dφi(x)]
−Ez∼pz[log(1 −Dφi(Gθ(z)))]||).
(8)
When an evolved generator obtains a relatively max value,
which corresponds to small gradients of
Dφ
, its generated
samples tend to spread out enough and to avoid the discrimi-
nators from having obvious countermeasures. Therefore,
FG
is formulated as
FG=FGq+γFGd,(9)
γ≥0
is used for balancing the quality and diversity of
generated samples. To this end, the performance
Fj,m
G
of each
evolved offspring
Gθj,m
is evaluated using
FG
. Generally, a
max fitness score
FG
directs to higher training efficiency and
better generative performance.
subsubsectionG-Selection
Finally, a simple yet useful survivor selection strategy
(µ, λ)
-selection [49] is employed to select the new parents
of next evolution according to the fitness score
FG
of existing
individuals. The selection function for the offspring of E-
Generators is defined as
nF1,1
G,F2,1
G,...,FJ,M
Go←sortmax nFj,m
Go.(10)
After sorting,
J
individuals
{Gθ1, Gθ2,· · · , GθJ}
possessing
the maximum fitness score can be survived for next evolution
during adversarial training. It is formulated as
θ1, θ2, . . . , θJ←θ1,1, θ2,1, . . . , θJ,1.(11)
D. E-Discriminators
Recently, some works attempt to revisit the multiple dis-
criminators setting by framing the simultaneous optimization
of different discriminator models as a multi-objective opti-
mization problem [9], [16]–[18]. Thus it would overcome
the problem of lacking informative gradient signal provided
by discriminators. To this end, we develop E-Discriminators,
which evolves a population of discriminators (parents)
{Dφ1, Dφ2,· · · , DφI}
to a set of new discriminators (offspring)
{Dφ1,1, Dφ1,2,· · · , Dφ1,N ,· · · , DφI,1, DφI,2,· · · , DφI ,N }
us-
ing its individual evolutionary algorithm. In fact, E-
Discriminators possesses two advantages to help CDE-GAN
steadily achieve promising generative performance: 1) the
evolutionary mechanism provides a dynamic strategy to
discriminators, thus the trade-off between generator(s) and
discriminators is well adjusted during training; 2) individ-
ual discriminators are unable to reject generated samples
perfectly and continue to provide informative gradients to

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 7
the generator throughout training. Each discriminator
Dφi
of E-Discriminators maximizes its own objective function.
Mathematically, we formulate E-Discriminators’ objective as
max LDφ=hMDφ1,MDφ2,· · · ,MDφIiT
,(12)
where
MDφi∈ {Mminimax
D,Mls
D}
is the variation (mutation)
of the evolutionary algorithm in E-Discriminators, and
φ=
{φ1, φ2,· · · , φI}.
Specifically, given the optimal
G∗
θ
, E-Discriminators evolves
with its own D-Variation,D-Evaluation and D-Selection during
training. The evolutionary process of E-Discriminators is
presented in Fig. 1and Algorithm 1.
1) D-Variation: Here, we take various objective functions
as the mutations for E-Discriminators. To keep the objective
functions of E-Discriminators are corresponding to the objective
functions of E-Generators, we take two different objective w.r.t
discriminators, including D-Minimax mutation (
Mminimax
D
),
and D-Least-Square mutation (
Mls
D
). According to [1], D-
Minimax mutation is a shared objective function for vanilla
GAN and NS-GAN, it can be formulated as
Mminimax
D=Ex∼pdata [log Dφi(x)]
+Ez∼pz[log(1 −Dφi(Gθ(z)))].(13)
Indeed, D-Minimax mutation adopts the sigmoid cross-entropy
loss for the discriminator, which will lead to the problem of
vanishing gradients when updating the generator using the fake
samples that are on the correct side of the decision boundary,
but are still far from the real data [44]. Interestingly, the
non-saturating loss
Mheuristic
G
will saturate when the input
is relatively large; the minimax objective
Mminimax
G
will
saturate when the input is relatively small. Thus, the generative
performance of E-GAN [24] will be limited when it bases on
single discriminator with minimax objective. In light of this
observation, we further adopt least squares objective function
for the second mutation of E-Discriminators:
MLS
D=1
2Ex∼pdata(x)(Dφi(x)−1)2
+1
2Ez∼pz(z)(Dφi(Gθ(z)))2
(14)
Benefiting the least square objective function penalizes samples
that lie in a long way on the correct side of the decision
boundary,
MLS
D
is capable of moving the fake samples toward
the decision boundary. To this end,
MLS
D
will help CDE-
GAN generate samples that are closer to the real data. Overall,
Mminimax
D
and
MLS
D
are complementary to each other and
will provide a promising optimization direction for the evolution
of E-Discriminators, which effectively adjusts the trade-off
between generator(s) and discriminators.
2) D-Evaluation: To evaluate the subpopulation evolved
by E-Discriminators, we take the minus log-gradient-norm
of optimizing each discriminator
Dφi
as fitness function
FD
, which is corresponding to the diversity fitness score of
generated samples in E-Generators:
FD=−log ||∇Dφi−Ex∼pdata [log Dφi(x)]
−Ez∼pz[log(1 −Dφi(Gθ(z)))]|| (15)
There are two reasons for this setting: 1) the gradient reveals
the train status of GANs. When the generator can generate
realistic samples, the discriminators will not reject the generated
sample confidently (i.e.,
Dφi
updated with small gradient);
when the generator collapses to a small region, the discriminator
will subsequently label collapsed points as fake with obvious
countermeasure (i.e.,
Dφi
updated with big gradient); 2) the
cooperative fitness function (Eq. 8and Eq. 15) between E-
Generators and E-Discriminators will keep the adversarial
consistency of them, which will improve the training stability
of CDE-GAN. Therefore,
FD
is effective for representing
whether the model falls in mode collapse, and thus it guides
E-Discriminators to evolve with the meaningful direction.
3) D-Selection: After evaluation, the new parents of next
evolution in E-Discriminators can be selected following the
principle of survival-of-the-fittest, which is similar to the
selection of E-Generators. We define selection function for
the offspring of E-Discriminators as
nF1,1
D,F2,1
D,...,FI,N
Do←sortmin nFi,n
Do.(16)
After sorting,
I
individuals
{Dφ1, Dφ2,· · · , DφI}
possessing
the minimum fitness score can be survived for next evolution
during adversarial training. It can be written as
φ1, φ2, . . . , φI←φ1,1, φ2,1, . . . , φI,1.(17)
E. Adversarial Multi-objective Optimization
CDE-GAN is optimized during the iterative evolution
(training) between E-Generators and E-Discrimiantors. Here,
we further demonstrate the total optimization of CDE-GAN.
In this work, E-Generators is trained with the multi-objective
optimization (Eq. 3) when
J > 1
; E-Discriminators is also
trained with the multi-objective optimization (Eq. 12) when
I > 1
[19], [50]. To this end, CDE-GAN terms the adver-
sarial training as a adversarial multi-objective optimization,
formulated as
min
Gθ
max
Dφ
=LGθ1,LGθ2,· · · ,LGθJT
+hMDφ1,MDφ2,· · · ,MDφIiT
.
(18)
Each
LGθj
and
MDφi
are optimized individually during evolu-
tion. Since the E-Generators and E-Discriminators are evolved
with their own evolutionary algorithms, the generator(s) and
discriminators are dynamically optimized in each evolutionary
step. Thus, CDE-GAN will well exploit the trade-off between
E-Generators and E-Discriminators and to conduct stable
and effective training. The multiple discriminators provide
informative feedback gradient to generator(s) for stabilizing
the training process.
IV. EXP ER IM EN TS A ND EVAL UATION
In subsequent sections, we introduce the evaluation metrics,
implementation details, and hyper-parameter analysis of experi-
ments. Furthermore, we qualitatively and quantitatively analyze
the generative performance of CDE-GAN to verify our claims.
Finally, we demonstrate the advantages of our method over 11
state-of-the-art methods, including modifying training objective

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 8
(a) IS for various balance factor (b) IS for various numbers of discriminators
Fig. 2: Experiments on the CIFAR-10 dataset for hyper-parameters analysis. (a) Inception score evaluation for different
CDE-GANs with various balance factor
γ={1,0.5,0.1,0.01}
. (b) Inception score evaluation for different CDE-GANs with
various numbers of discriminators I={1,2,4,8}.
TABLE I: ARCHITECTURES OF THE GENERATIVE AND DISCRIMINATIVE NETWO RK S USE D IN THIS WORK,I.E., DCGAN
MOD EL , MLP WITH 3 LAYE RS ,AN D MLP WITH 4 LAYE RS .
Generative network Discriminative network
DCGAN [14], [24], [42]
Input: Noise z∼pz, 100 Input: Image, (32 ×32 ×3)
[layer 1]Fully connected and Reshape to (4×4×512); ReLU; [layer 1]Convolution (4, 4, 128), stride=2; LeakyReLU;
[layer 2]Transposed Convolution (4, 4, 512), stride=2; ReLU; [layer 2]Convolution (4, 4, 256), stride=2; LeakyReLU;
[layer 3]Transposed Convolution (4, 4, 256), stride=2; ReLU; [layer 3]Convolution (4, 4, 512), stride=2; LeakyReLU;
[layer 4]Transposed Convolution (4, 4, 128), stride=2; ReLU; [layer 4]Fully connected (1); Sigmoid/Least squares;
[layer 5]Transposed Convolution (4, 4, 3), stride=2; Tanh; Output: Real or Fake (Probability)
Output: Generated Imgage, (32 ×32 ×3)
MLP with 3 Layers [43], [44]
Input: Noise z∼pz, 256 Input: Point
[layer 1]Fully connected (128); ReLU; [layer 1]Fully connected (128); LeakyReLU;
[layer 2]Fully connected (128); ReLU; [layer 2]Fully connected (128); LeakyReLU;
[layer 3]Fully connected (2); Linear; [layer 3]Fully connected (1); Sigmoid/Least squares;
Output: Generated Point Output: Real or Fake (Probability)
MLP with 4 Layers [14]
Input: Noise z∼pz, 256 Input: Point
[layer 1]Fully connected (128); ReLU; [layer 1]Fully connected (128); ReLU;
[layer 2]Fully connected (128); ReLU; [layer 2]Fully connected (128); ReLU;
[layer 3]Fully connected (128); ReLU; [layer 3]Fully connected (128); ReLU;
[layer 4]Fully connected (2); Linear; [layer 4]Fully connected (1); Sigmoid/Least squares;
Output: Generated Point Output: Real or Fake (Probability)
based GANs, multi-generator based GANs, multi-discriminator
based GANs, and evolutionary computation based GANs.
A. Evaluation Metric
We use the inception score (IS) [46] to quantitatively
evaluate the performance of the proposed method. It is a
common quantitative evaluation metric in the image generation
of GANs. IS uses the Inception model for every generated
image to get the conditional label distribution
p(y|x)
.
It is formulated as:
exp (ExKL(p(y|x)kp(y)))
. IS takes
simultaneously generative quality and diversity into account.
The higher IS is achieved, the better quality and diversity
of samples is generated. IS is calculated by Tensorflow code
version using randomly generated 50k samples in this paper.
Meanwhile, we also qualitatively evaluate our method with
human visual conception.
B. Implementation Details
In the following experiments, we use the default hyper-
parameter values listed in Algorithm 1. We conduct exten-
sive experiments on one synthetic dataset (i.e., a mixture
of 8 Gaussians arranged in a circle) and three real-word
benchmark datasets (i.e., CIFAR10 [51], LSUN-Bedrooms
[52], and CelebA [53]) to prove the effectiveness of CDE-
GAN. Furthermore, we adopt the same network architectures
(DCGAN) with existing in works [14], [24], [42] to conduct
real data experiments for facilitating direct comparison. For
the sake of fair comparison, we select MLP (with 3 layers [43],
[44] or 4 layers [14]) as the model architecture of CDE-GAN
to conduct toy experiments and generate 512 points to cover
modes. The model architectures are clearly displayed in Table I.
The noisy vector
z
is sampled from the uniform distribution
pz
with 100 dimensions and 256 dimensions for real-word datasets
and synthetic datasets, respectively. We employ Adam optimizer
with hyper-parameter (
α= 0.0002
,
β1= 0.5
,
β2= 0.99
) to
optimize our model. All experiments are performed on a single

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 9
NVIDIA 1080Ti graphic card with 11GB memory. We use
PyTorch for the implementation of all the experiments.
C. Experiment 1: Hyper-Parameters Analysis
In principle, there are two hyper-parameters closely corre-
sponding to the performance of CDE-GAN, i.e., the balance
factor
γ
(see Eq. 9) and the number of discriminators
I
. Since
the hyper-parameter
γ
is used for balancing the measurement
of sample quality and diversity, it directs generator selection
of E-Generators, and thus it will affect the effectiveness of
CDE-GAN. Meanwhile, we analyze how the number of dis-
criminators
I
affects the sample diversity of the corresponding
generator and select proper discriminators for balancing time
consuming and generative performance of CDE-GAN.
1) Balance Factor
γ
:In fact, the quality and diversity of
the synthesized objects are two key goals of the generative
task. Analogously, we also consider these two measurements
for CDE-GAN evaluation in generation task. Here, we embed
a balance factor
γ
into the generator’s fitness score to balance
the quality and diversity of generated samples during generator
updates. Similar to [24],
γ > 0
is considered. If
γ
is set as too
small, the diversity fitness score is almost not considered; while
γ
is set too large, the model is not stable since the gradient-
norm of discriminators
Dφ
could vary largely. To this end,
0< γ ≤1is considered for the setting of our experiments.
To select a proper
γ
for CDE-GAN, we run a grid search
to find its value on CIFAR-10. As shown in Fig. 2(a), we
take various balance factor
γ={1,0.5,0.1,0.01}
to conduct
experiments. Results show that CDE-GAN is out of work at
the beginning and achieves convergence with slow speed when
γ
is set as 1; while it gains promising generative performance
and comparable convergence speed, if
γ
is set as a relatively
small value, e.g., 0.01. Based on these observations, we take
γ= 0.1to conduct later experiments on real-world datasets.
2) Number of Discriminators
I
:Multi-discriminator based
GANs frame GAN’s training as a multi-objective optimiza-
tion problem, which will overcome the problem of lacking
informative gradient signal provided by the discriminator.
Here, we can also extend CDE-GAN as a multi-discriminator
based GAN. Specifically, we take E-Discriminators to evolve
multiple discriminators (
I≥1
) with various mutations.
According to [17], [19], [54], the number of discriminators is
closely corresponding to the diversity of the generated sample.
Therefore, we take experiments to analyze how the number of
discriminators affects the generative performance of CDE-GAN
and select a promising Ifor later experiments.
In Fig. 2(b), we report the box-plots of inception score
evaluation for different CDE-GANs with different numbers of
discriminators on CIFAR-10 across 3 independent runs. Results
clearly show that increasing the number of discriminators yields
better generative performance for CDE-GANs. Note that CDE-
GAN is more stable when more numbers of discriminator are
survived for evolution, because GP cannot largely improve
the performance of CDE-GAN when more discriminators
are set. The reason is that the generator will better fool the
discriminators in the E-Discriminators when more numbers of
discriminators are survived for evolution [9], [19]. To this end,
The E-Generators is more likely to meet positive feedback from
E-Discriminators, which is benefiting for the stable training
of CDE-GAN.. Moreover, we convince that CDE-GAN can
further improve generative performance if more discriminators
are survived during training. To take a trade-off between time
consuming and efficacy, we conduct latter experiments with
two discriminators for CDE-GAN, unless otherwise claimed.
D. Experiment 2: Generative Performance Evaluation
In this section, we qualitatively and quantitatively analyze
the generative performance of CDE-GAN to support our
claims. Here, we take vanilla GAN (GAN) [1], non-saturated
GAN (NS-GAN) [1], and least square GAN (LSGAN) [43],
[44], wasserstein GAN (WGAN) [11], and evolutionary GAN
(EGAN) [24] as baselines for comparison and discussion,
because these GAN models are closed to our method.
1) Qualitative Evaluation: Learning on a Gaussian mixture
distribution to evaluate the diversity of GANs is a popular
experiment setting, which intuitively reveals GANs whether to
suffer from mode collapse. When the model suffers from the
mode collapse problem, it will generate samples only around
a few modes. To validate the effectiveness of our proposed
method, analogously, we first qualitatively compare CDE-GAN
with different baselines on synthesis dataset of 2-D mixture
of 8 gaussian mixture distributions. For conducting a fair
comparison, we adopt the experimental design proposed in
[43], [44], which trains GANs with 3 layers of MLP network
architecture. Meanwhile, the survived parents number of E-
Discriminators (
I
) and E-Generators (
J
) of CDE-GAN are
set as 1, i.e., during each evolutionary step, only the best one
candidature in each evolution algorithm is kept. We train each
method over 400k generator iterations with the same network
architecture. As shown in Fig. 3(a), the dynamic results of data
distribution and the Kernel Density Estimation (KDE) plots
on different baselines are reported. We can see that all of the
baselines only generate samples around a few of valid modes of
the data distribution, i.e., 6 modes for vanilla GAN and LSGAN,
2 modes for NS-GAN, 8 modes for E-GAN (parts of modes
are weakly covered), which shows that they suffer from mode
collapse to a greater or lesser degree. Nevertheless, CDE-GAN
can successfully learn the Gaussian mixture distribution for
all modes. These experiments demonstrate that the cooperative
dual evolutionary strategy well circumvents the mode collapse.
Furthermore, we also try 4 layers of MLP network architec-
ture to conduct the same experiment to evaluate the stability of
CDE-GAN. In Fig. 3(b), The results show that all methods can
speed up their convergence speed compared to the results on 3
layers MLP architecture. Notably, all baselines tend to generate
a few modes, while CDE-GAN is less prone to this problem and
achieves better convergence speed. It reveals that CDE-GAN
possesses another advantage of architecture robustness.
For the sake of proving the potential power of generative
performance of CDE-GAN, we show several samples generated
by our proposed model trained on three datasets with
32 ×32
pixels in Fig. 4, i.e., CIFAR-10, LSUN-Bedrooms, and CelebA.
Note that the presented samples are fair random drew, not
cherry-picked. It can be seen, on CIFAR-10, that CDE-GAN

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 10
GAN
NS-GAN
LSGAN
E-GAN
CDE-GAN
Iteration 10k Iteration 100k Iteration 200k Iteration 300k Iteration 400k
(a) Gaussian kernel estimation with MLP of 3 layers
(b) Gaussian kernel estimation with MLP of 4 layers
Fig. 3: Dynamic results of Gaussian kernel estimation over generator iteration for different GANs. For each pair of images, the
left one is data distribution (real data is represented in blue, generated data is represented in red), and the right one is KDE
plots of the generated data corresponded to its left generated data. From top to bottom, the rows are the results of vanilla GAN,
NS-GAN, LSGAN, E-GAN, and CDE-GAN (Ours).

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 11
TABLE II: COMPARISON WITH E-GAN ON CIFAR-10 WITH
OR WITHOUT GP. †[24]
Methods Inception Score
E-GAN (µ= 1, without GP) 6.88 ±0.10
E-GAN (µ= 2, without GP) 6.71 ±0.06
E-GAN (µ= 4, without GP) 6.96 ±0.09
E-GAN (µ= 8, without GP) 6.72 ±0.09
E-GAN (µ= 1, with GP)†7.13 ±0.07
E-GAN (µ= 2, with GP)†7.23 ±0.08
E-GAN (µ= 4, with GP)†7.32 ±0.09
E-GAN (µ= 8, with GP)†7.34 ±0.07
(Ours) CDE-GAN (I= 1, without GP) 6.85 ±0.05
(Ours) CDE-GAN (I= 2, without GP) 6.93 ±0.09
(Ours) CDE-GAN (I= 4, without GP) 7.06 ±0.09
(Ours) CDE-GAN (I= 8, without GP) 7.35 ±0.06
(Ours) CDE-GAN (I= 1, with GP) 7.05 ±0.05
(Ours) CDE-GAN (I= 2, with GP) 7.18 ±0.05
(Ours) CDE-GAN (I= 4, with GP) 7.48 ±0.10
(Ours) CDE-GAN (I= 8, with GP) 7.51 ±0.05
is capable of generating visually recognizable images of
frogs, airplanes, horses, etc. CDE-GAN can produce bedrooms
with various styles and views, and beds and windows in
the rooms are clearly displayed on LSUN-Bedrooms. It can
also synthesize face images possessing various attributes (e.g.,
gender, age, expression, and hairstyle). These results confirm
the quality and diversity of samples generated by our method.
2) Quantitative Evaluation: Our qualitative observations
above are confirmed by the quantitative evaluations. For the
sake of demonstrating the merits of the proposed CDE-GAN
over the baselines, we train these methods on CIFAR-10 and
plot inception scores over the training process with the same
network architecture. As shown in Fig. 5, CDE-GAN can get a
higher inception score within 100k generator iterations. After
40k iterations, specifically, CDE-GANs with different settings
consistently perform better over all the baselines. Meanwhile,
the baselines fall in different training problems, e.g., instability
at convergence (wgan and lsgan), invalid (GAN). This shows
that the cooperative dual evolution method is benefiting for
adversarial training of GANs.
Since EGAN [24] is most similar to our method, we further
take E-GAN as a baseline to compare with CDE-GAN for
stability analysis. In Table II, we take various number of
generators
µ={1,2,4,8}
for E-GAN and various number of
discriminators
I={1,2,4,8}
for CDE-GAN. We train each
model in 150k generator iterations to conduct experiments. Note
that we implement results of E-GAN using the experimental
setting of literature [24] and codes provided by authors
7
.
Results show that CDE-GAN achieves better performance on
CIFAR-10 than E-GAN. Furthermore, if E-GAN uses gradient
penalty (GP) term during training, its generative performance
is greatly improved compared to the results of its original
version (e.g., 0.62 improvement for IS when
µ
is set as
8); while our CDE-GAN achieves a little improvement if
GP is used (e.g., 0.16 improvement for IS when
I
is set
as 8). This reveals that E-GAN is unstable during training
due to its single evolution, and thus GP term is effective for
regularizing the discriminator to provide informative gradients
for updating the generators. Benefiting from cooperative dual
7https://github.com/WANG-Chaoyue/EvolutionaryGAN-pytorch
evolution, CDE-GAN injects dual diversity into training, and
thus it can cover different data modes. Furthermore, with the
number of discriminator increasing, the balance of generator
and discriminators of CDE-GAN is well adjusted and thus
it continuously achieves obvious improvement. This shows
that adversarial multi-objective optimization is effective for
stabilizing the training process of CDE-GAN. To this end, the
discriminators can continuously provide informative gradient
for generator’s updation, which performs the function of GP.
TABLE III: COMPARISON WITH STATE-OF -TH E- ART
MET HO DS O N CIFAR-10. THE N,K,T,µ,AND I
REP RE SE NT T HE NUMBERS OF DISCRIMINATORS OR
GEN ER ATOR S FO R DIFFERENT METHODS,Addit. Superv.
Info. DEN OTE S THAT ADDITIONAL SUPERVISED
INFORMATION IS USED BY MEHTO D. T HE BE ST TW O
RES ULTS A RE MARKED WITH BOLD,AND UNDERLINE.
Methods Addit. Superv. Info. Inception Score
Real data 11.24±0.12
GAN-GP [1]%6.93 ±0.08
DCGAN [42]%6.64 ±0.14
WGAN-GP [14]%6.68 ±0.06
SN-GAN [15]%7.42 ±0.08
GMAN (N= 5) [17]%6.40 ±0.19
D2GAN [16]%7.15 ±0.07
HV(K= 24) [19]%7.32 ±0.26
MicroGAN (K= 2) [54]%6.77 ±0.00
MIX+WGAN (T= 5) [8]%4.36 ±0.04
MGAN (K= 10) [21]!8.33 ±0.10
E-GAN (µ= 8) [24]%7.34 ±0.07
(Ours) CDE-GAN (I= 8)%7.51 ±0.05
E. Experiment 3: Comparisons With State-of-the-Art Methods
In this section, we compare CDE-GAN with 11 state-of-
the-art GANs to show its advantages. We first report the IS
obtained by our CDE-GAN and baselines on CIFAR-10 in
Table III. Overall, experimental results show that CDE-GAN
outperforms almost all baselines on the optimal settings (except
MGAN). The reason is that MGAN takes additional supervised
information (generators’ labels) for supporting semi-supervised
learning. Nevertherless, CDE-GAN is an entirely unsupervised
manner, this is the concentration of this paper. Thus, integrating
it into evolutionary GANs to conduct semi-supervised learning
could be a promising avenue for our future work.
It is worthy to note that CDE-GAN terms adversarial training
as a multi-objective optimization problem when
I≥2
. Thus
we further discuss the advantage of CDE-GAN compared
with multi-discriminator based GANs. In the second group
of Table III, all multi-discriminator based GANs using various
numbers of discriminators are inferior to our CDE-GAN, i.e.,
GMAN (
K= 5
), D2GAN (2 discriminators), HV(
K= 24
),
MicroGAN (
K= 2
). This further demonstrates that cooperative
dual evolution is effective for multi-objective optimization and
adversarial training of GANs.
Finally, we display visual quality comparisons between sev-
eral state-of-the-art methods (i.e., Stabilizing-GAN [9], acGAN
[18], MAD-GAN [22], Lipizzaner [23], and Mustangs [26]) and
our method on CelebA. See from Fig. 6, the most of the face

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 12
(a) CIFAR-10 (b) LSUN-Bedrooms (c) CelebA
Fig. 4: Samples generated by our proposed CDE-GAN on various natural image datasets, i.e., CIFAR-10, LSUN-Bedrooms and
CelebA. Please see many more results in the supplementary file.
Fig. 5: Inception score of different GANs on CIFAR-10.
images generated by Lipizzaner and Mustangs are incomplete
or blurred, while our CDE-GAN generates high-fidelity face
images. This demonstrates that the proposed cooperative dual
evolutionary strategy performs better on adversarial training
compared to other evolutionary computation based GANs.
Furthermore, CDE-GAN also performs advantages over other
GAN methods, i.e., MAD-GAN with 3 generators, Stabilizing-
GAN with 24 discriminators, acGAN with 5 discriminators.
Note that MAD-GAN trained its model using additional
supervised information (generators’ labels) and Stabilizing-
GAN used the cropped version of the images of CelebA. More
visual comparisons on CIFAR-10 and LSUN-Bedrooms are
provided in the supplementary file.
V. DISCUSSION
We would like to have more discussion here about the
advantages and limitations of the proposed CDE-GAN method.
First, we analyze why and how our method can circumvent
the mode collapse and instability problem of GANs.
•
Compared to single evolution strategy (i.e., E-GAN [24]),
CDE-GAN injects dual diversity into training benefiting
from the cooperative dual evolution. It decomposes
the complex adversarial optimization problem into two
subproblems (i.e., generation and discrimination), and each
subproblem is solved by a separated subpopulation (i.e.,
E-Generators and E-Discriminators), evolved by its own
evolutionary algorithm (including individual variations,
evaluation, and selection). Furthermore, The complemen-
tary mutations in E-Generators and E-Discriminators are
helpful for CDE-GAN to evolve in different possible
directions during various training stages. In this way, the
dual evolutionary population injects dual diversity into the
training, and thus it can effectively cover different data
modes. This significantly mitigates training pathologies of
mode collapse for GANs. Experiments in Section
IV-D
and IV-E intuitively verify this claims.
•
CDE-GAN terms adversarial training as an adversarial
multi-objective optimization problem, and thus the multi-
ple discriminators provide informative feedback gradient
to the generator for stabilizing training process. In ideal,
we prefer that generator always has strong gradients from
the discriminator during training. Since the discriminator
quickly learns to distinguish real and fake samples,
the single-objective optimization-based GANs make this
difficult to ensure. To this end, they cannot provide mean-
ingful error signals to improve the generator thereafter.
In contrast, multi-objective optimization simultaneously
optimizes the losses provided by different models to favor
worse discriminators. Thus discriminators provide more
informative gradients to the generator. The experiments
in Section IV-C2 and IV-D2 support this conclusion.
•
Soft Mechanism well balances the trade-off between E-
Generators and E-Discriminators to help CDE-GAN con-
duct effective adversarial training. In fact, the degenerate
results of GANs can be avoided by employing learner
(discriminator) with limited capacity and corrupting data
samples with noise [9], [17], [18]. To this end, Soft
Mechanism softens the maximization of discriminators
to weaken the discriminators, which avoids the problem
of whack-a-mole dilemma during training and enables
stable training of CDE-GAN. Section
III-B
provides more
theoretical analysis.

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 13
(a) MAD-GAN(k= 3) (b) Lipizzaner (c) Mustangs
(d) Stabilizing-GAN(K= 24) (e) acGAN(N= 5) (f) CDE-GAN(I= 2)
Fig. 6: Visual comparisons of different methods on CelebA. The samples generated by different methods are provided by the
original literatures, i.e., MAD-GAN [22], Lipizzaner [23], Mustangs [26], Stabilizing-GAN [9], and acGAN [18].
Indeed, CDE-GAN limits in costing more time in each
iteration. Theoretically, let’s take updating a generator iteration
as a training step. CDE-GAN will cost
O(K·I·N)+O(J·M)
operations for one step (all notations are defined in Algorithm
1, and
J
is set as 1 in this paper). In practice, the time-
consuming of CDE-GANs at each iteration is reported in
Table IV. Specifically, to train a CDE-GAN model for
32 ×32
images using DCGAN architecture with different numbers
of discriminators (
I={1,2,4,8}
), it will cost around
0.079±0.003
,
0.079±0.003
,
0.158±0.021
, and
0.303±0.002
seconds respectively for one generator iteration (excluding
generating images for score test) on a single GPU. Since
the cooperative dual evolutionary strategy is effective for
adversarial training, CDE-GAN performs significantly fewer
training steps to achieve the same generative performance than
other baselines (see Fig. 5).
TABLE IV: TIME CONSUMING OF CDE-GANS AT EACH
GEN ER ATOR IT ER ATIO N.
Methods Time/Iteration (Seconds)
CDE-GAN(I= 1) 0.039±0.000
CDE-GAN(I= 2) 0.079±0.003
CDE-GAN(I= 4) 0.158±0.021
CDE-GAN(I= 8) 0.303±0.002
VI. CONCLUSION
In this paper, we proposed a novel GAN (CDE-GAN),
incorporating cooperative dual evolution with respect to E-
Generators and E-Discriminators into a unified evolutionary
adversarial framework, to circumvent adversarial optimization
difficulties of GANs, i.e., mode collapse and instability. Notably,
the dual evolution provides a dynamic strategy to the genera-
tor(s) and discriminators, exploits the complementary properties,
and injects dual mutation diversity into learning. This diversifies
the estimated density in capturing multi-modes and improves
the generative performance of CDE-GAN. Additionally, we
introduced a Soft Mechanism to balance E-Generators and
E-Discriminators for conducting effective and stable training.
The competitive results on one synthetic dataset (i.e., the 2D
mixture of 8 Gaussian) and three real-world image datasets (i.e.,
CIFAR-10, LSUN-Bedrooms, and CelebA), demonstrate the
superiority and great potentials of cooperative dual evolution
for GANs. Extensive experiments also show that CDE-GAN
performs obvious advantages over all the compared state-of-
the-art methods.
In the future, we will further improve CDE-GAN for
incorporating additional supervised information into learning
and speeding up learning. Meanwhile, we will also apply

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 14
evolutionary computation based GANs to other generative
tasks, e.g., text synthesis, video prediction.
VII. ACKNOWLEDGMENT
The authors would like to thank Dr. Chaoyue Wang at the
School of Computer Science, University of Sydney, for his
assistance with coding and theoretical suggestions.
REFERENCES
[1]
I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley,
S. Ozair, A. C. Courville, and Y. Bengio, “Generative adversarial nets,”
in Proceedings Advances in Neural Information Processing Systems
(NeurIPS), 2014.
[2]
J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image
translation using cycle-consistent adversarial networks,” in Proceedings
IEEE International Conference on Computer Vision (ICCV), 2017, pp.
2242–2251.
[3]
S. Tulyakov, M.-Y. Liu, X. Yang, and J. Kautz, “MoCoGAN: Decom-
posing Motion and Content for Video Generation,” in Proceedings IEEE
Conference on Computer Vision and Pattern Recognition (CVPR), 2018,
pp. 1526–1535.
[4]
J. Xie, Y. Lu, R. Gao, S. Zhu, and Y. Wu, “Cooperative training
of descriptor and generator networks,” IEEE Transactions on Pattern
Analysis and Machine Intelligence, vol. 42, pp. 27–45, 2020.
[5]
L. Yu, W. Zhang, J. Wang, and Y. Yu, “Seqgan: Sequence generative
adversarial nets with policy gradient,” in Proceedings AAAI Conference
on Artificial Intelligence, 2017.
[6]
Z. Liu, J. Wang, and Z. Liang, “Catgan: Category-aware generative adver-
sarial networks with hierarchical evolutionary learning for category text
generation,” in Proceedings AAAI Conference on Artificial Intelligence,
2020.
[7]
I. J. Goodfellow, “Nips 2016 tutorial: Generative adversarial networks,”
arXiv preprint arXiv:1701.00160, 2016.
[8]
S. Arora, R. Ge, Y. Liang, T. Ma, and Y. Zhang, “Generalization
and equilibrium in generative adversarial nets (gans),” in Proceedings
International Conference on Machine Learning (ICML), 2017, pp. 224–
232.
[9]
B. Neyshabur, S. Bhojanapalli, and A. Chakrabarti, “Stabilizing gan train-
ing with multiple random projections,” arXiv preprint arXiv:1705.07831,
2017.
[10]
D. Berthelot, T. Schumm, and L. Metz, “Began: Boundary equilibrium
generative adversarial networks,” arXiv preprint arXiv:1703.10717, 2017.
[11]
M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein generative
adversarial networks,” in Proceedings International Conference on
Machine Learning (ICML), 2017, pp. 214–223.
[12]
L. Metz, B. Poole, D. Pfau, and J. Sohl-Dickstein, “Unrolled generative
adversarial networks,” in Proceedings International Conference on
Learning Representations (ICLR), 2017, pp. 1–25.
[13]
D. Warde-Farley and Y. Bengio, “Improving generative adversarial
networks with denoising feature matching,” in Proceedings International
Conference on Learning Representations (ICLR), 2017, pp. 1–11.
[14]
I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. C. Courville,
“Improved training of wasserstein gans,” in Proceedings Advances in
Neural Information Processing Systems (NeurIPS), 2017, pp. 5767–5777.
[15]
T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida, “Spectral normaliza-
tion for generative adversarial networks,” in Proceedings International
Conference on Learning Representations (ICLR), 2018, pp. 1–26.
[16]
T. D. Nguyen, T. Le, H. Vu, and D. Q. Phung, “Dual discriminator gen-
erative adversarial nets,” in Proceedings Advances in Neural Information
Processing Systems (NeurIPS), 2017, pp. 2670–2680.
[17]
I. Durugkar, I. Gemp, and S. Mahadevan, “Generative multi-adversarial
networks,” in Proceedings International Conference on Learning Repre-
sentations (ICLR), 2017, pp. 1–14.
[18]
T. Doan, J. Monteiro, I. Albuquerque, B. Mazoure, A. Durand, J. Pineau,
and R. D. Hjelm, “Online adaptative curriculum learning for gans,” in
Proceedings AAAI Conference on Artificial Intelligence, 2019.
[19]
I. Albuquerque, J. Monteiro, T. Doan, B. Considine, T. H. Falk, and
I. Mitliagkas, “Multi-objective training of generative adversarial networks
with multiple discriminators,” in Proceedings International Conference
on Machine Learning (ICML), 2019, pp. 202–211.
[20]
I. O. Tolstikhin, S. Gelly, O. Bousquet, C.-J. Simon-Gabriel, and
B. Schölkopf, “Adagan: Boosting generative models,” in Proceedings
Advances in Neural Information Processing Systems (NeurIPS), 2017,
pp. 5424–5433.
[21]
Q. Hoang, T. D. Nguyen, T. Le, and D. Q. Phung, “Mgan: Training
generative adversarial nets with multiple generators,” in Proceedings
International Conference on Learning Representations (ICLR), 2018, pp.
1–24.
[22]
A. Ghosh, V. Kulharia, V. P. Namboodiri, P. H. S. Torr, and P. K. Dokania,
“Multi-agent diverse generative adversarial networks,” in Proceedings
IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
2018, pp. 8513–8521.
[23]
T. Schmiedlechner, A. Al-Dujaili, E. Hemberg, and U.-M. O’Reilly,
“Towards distributed coevolutionary gans,” in Proceedings AAAI 2018
Fall Symposium, 2018, pp. 1–6.
[24]
C. Wang, C. Xu, X. Yao, and D. Tao, “Evolutionary generative adversarial
networks,” IEEE Transactions on Evolutionary Computation, vol. 23, pp.
921–934, 2019.
[25]
V. Costa, N. Lourenço, J. Correia, and P. Machado, “Coegan: Evaluating
the coevolution effect in generative adversarial networks,” in Proceedings
The Genetic and Evolutionary Computation Conference (GECCO), 2019,
pp. 374–382.
[26]
J. Toutouh, E. Hemberg, and U.-M. O’Reilly, “Spatial evolutionary
generative adversarial networks,” in Proceedings The Genetic and
Evolutionary Computation Conference (GECCO), 2019, pp. 472–480.
[27]
Y. Sun, G. G. Yen, and Z. Yi, “Evolving unsupervised deep neural
networks for learning meaningful representations,” IEEE Transactions
on Evolutionary Computation, vol. 23, pp. 89–103, 2019.
[28]
Y. Sun, B. Xue, M. Zhang, and G. G. Yen, “A particle swarm optimization-
based flexible convolutional autoencoder for image classification,” IEEE
Transactions on Neural Networks and Learning Systems, vol. 30, pp.
2295–2309, 2019.
[29]
Y. Sun, B. Xue, G. G. Yen, and M. Zhang, “Evolving deep convolu-
tional neural networks for image classification,” IEEE Transactions on
Evolutionary Computation, vol. 24, pp. 394–407, 2020.
[30]
Y. Sun, H. Wang, B. Xue, Y. Jin, G. G. Yen, and M. Zhang, “Surrogate-
assisted evolutionary deep learning using an end-to-end random forest-
based performance predictor,” IEEE Transactions on Evolutionary
Computation, vol. 24, pp. 350–364, 2020.
[31]
J. Toutouh, E. Hemberg, and U.-M. O’Reilly, “Re-purposing het-
erogeneous generative ensembles with evolutionary computation,” in
Proceedings The Genetic and Evolutionary Computation Conference
(GECCO), 2020, pp. 425–434.
[32]
Y. Sun, B. Xue, M. Zhang, and G. G. Yen, “Completely automated
cnn architecture design based on blocks,” IEEE Transactions on Neural
Networks and Learning Systems, vol. 31, pp. 1242–1254, 2020.
[33]
Y. Sun, B. Xue, G. G. Yen, and M. Zhang, “Automatically designing
cnn architectures using genetic algorithm for image classification,” IEEE
Transactions on cybernetics, 2020.
[34]
C. K. Goh and K. C. Tan, “A competitive-cooperative coevolutionary
paradigm for dynamic multiobjective optimization,” IEEE Transactions
on Evolutionary Computation, vol. 13, pp. 103–127, 2009.
[35]
M. N. Omidvar, X. Li, Y. Mei, and X. Yao, “Cooperative co-evolution
with differential grouping for large scale optimization,” IEEE Transactions
on Evolutionary Computation, vol. 18, pp. 378–393, 2014.
[36]
S. He, G. Jia, Z. Zhu, D. A. Tennant, Q. Huang, K. Tang, J. Liu,
M. Musolesi, J. K. Heath, and X. Yao, “Cooperative co-evolutionary
module identification with application to cancer disease module discovery,”
IEEE Transactions on Evolutionary Computation, vol. 20, pp. 874–891,
2016.
[37]
X. Lu, S. Menzel, K. Tang, and X. Yao, “Cooperative co-evolution-
based design optimization: A concurrent engineering perspective,” IEEE
Transactions on Evolutionary Computation, vol. 22, pp. 173–188, 2018.
[38]
X. Y. Zhang, Y. jiao Gong, Y. Lin, J. Zhang, S. Kwong, and J. Zhang,
“Dynamic cooperative coevolution for large scale optimization,” IEEE
Transactions on Evolutionary Computation, vol. 23, pp. 935–948, 2019.
[39]
J. Zou, Q. Li, S. Yang, J. Zheng, Z. Peng, and T. Pei, “A dynamic
multiobjective evolutionary algorithm based on a dynamic evolutionary
environment model,” Swarm Evolution Computation, vol. 44, pp. 247–
259, 2019.
[40]
W. Ding, C.-T. Lin, and Z. Cao, “Deep neuro-cognitive co-evolution for
fuzzy attribute reduction by quantum leaping pso with nearest-neighbor
memeplexes,” IEEE Transactions on Cybernetics, vol. 49, pp. 2744–2757,
2019.
[41]
Q. Zhang, S. Yang, S. Jiang, R. Wang, and X. Li, “Novel prediction
strategies for dynamic multi-objective optimization,” IEEE Transactions
on Evolutionary Computation, vol. 24, pp. 260–274, 2020.
[42]
A. Radford, L. Metz, and S. Chintala, “Unsupervised representation
learning with deep convolutional generative adversarial networks,” in
Proceedings International Conference on Learning Representations
(ICLR), 2016, pp. 1–15.

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 15
[43]
X. Mao, Q. Li, H. Xie, R. Y. K. Lau, Z. Wang, and S. P. Smolley,
“Least squares generative adversarial networks,” in Proceedings IEEE
International Conference on Computer Vision (ICCV), 2017, pp. 2813–
2821.
[44]
X. Mao, Q. Li, H. Xie, Z. Wang, R. Y. K. Lau, and S. P. Smolley, “On
the effectiveness of least squares generative adversarial networks,” IEEE
Transactions on Pattern Analysis and Machine Intelligence, vol. 41, pp.
2947–2960, 2019.
[45]
M. Arjovsky and L. Bottou, “Towards principled methods for training
generative adversarial networks,” in Proceedings International Conference
on Learning Representations (ICLR), 2017, pp. 1–17.
[46]
T. Salimans, I. J. Goodfellow, W. Zaremba, V. Cheung, A. Radford,
and X. Chen, “Improved techniques for training gans,” in Proceedings
Advances in Neural Information Processing Systems (NeurIPS), 2016,
pp. 2234–2242.
[47]
M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter,
“Gans trained by a two time-scale update rule converge to a local nash
equilibrium,” in Proceedings Advances in Neural Information Processing
Systems (NeurIPS), 2017, pp. 6629–6640.
[48]
V. Nagarajan and J. Z. Kolter, “Gradient descent gan optimization
is locally stable,” in Proceedings Advances in Neural Information
Processing Systems (NeurIPS), 2017, pp. 5591–5600.
[49]
O. Kramer, Machine Learning for Evolution Strategies. Cham,
Switzerland: Springer, 2016, vol. 20.
[50]
K. Deb, “Multi-objective optimization using evolutionary algorithms,”
vol. 16. Chichester, U.K.: Wiley, 2001.
[51]
A. Krizhevsky, “Learning multiple layers of features from tiny images.”
Department Computer Science, University Toronto, Toronto, ON, Canada,
Report TR-2009, 2009.
[52]
F. Yu, Y. Zhang, S. Song, A. Seff, and J. Xiao, “Lsun: Construction of a
large-scale image dataset using deep learning with humans in the loop,”
arXiv preprint arXiv:1506.03365, 2015.
[53]
Z. Liu, P. Luo, X. Wang, and X. Tang, “Deep learning face attributes
in the wild,” Proceedings IEEE International Conference on Computer
Vision (ICCV), pp. 3730–3738, 2015.
[54]
G. Mordido, H. Yang, and C. Meinel, “microbatchgan: Stimulating
diversity with multi-adversarial discrimination,” in Proceedings Winter
Conference on Applications of Computer Vision (WACV), 2020, pp. 3050–
3059.
Shiming Chen
is currently a full-time Ph.D student
in the School of Electronic Information and Com-
munications, Huazhong University of Sciences and
Technology (HUST), China. He serves as the reviewer
for prestigious journals such as IEEE Transactions
on Image Processing, IEEE Transactions on Systems,
Man, and Cybernetics: Systems, IEEE Transactions
on Industrial Informatics, Information Fusion, Infor-
mation Sciences, and Applied Soft Computing. His
current research interests span computer vision and
machine learning with a series of topics, such as
generative modeling and learning, zero-shot learning, and image retrieval.
Wenjie Wang
is currently pursuing the M.Sc. degree
in the School of Electronic Information and Com-
munications(EIC), Huazhong University of Sciences
and Technology(HUST), China, in 2019. He received
the B.E. degree in the School of EIC, HUST. His
current research interests include multimodal learning,
computer vision, and machine learning.
Beihao Xia
is currently pursuing the Ph.D. degree
in the School of Electronic Information and Com-
munications (EIC), Huazhong University of Sciences
and Technology(HUST), China. He received the
B.E. degree in College of Computer Science and
Electronic Engineering, Hunan University(HNU),
China, in 2015, and the M.Sc. degree in the School of
EIC, HUST, China, in 2018, respectively. His current
research interests include image/video processing,
computer vision, and machine learning.
Xinge You
(Senior Member, IEEE) is currently a Pro-
fessor with the School of Electronic Information and
Communications, Huazhong University of Science
and Technology, Wuhan. He received the B.S. and
M.S. degrees in mathematics from Hubei University,
Wuhan, China, in 1990 and 2000, respectively, and
the Ph.D. degree from the Department of Computer
Science, Hong Kong Baptist University, Hong Kong,
in 2004. His research results have expounded in 20+
publications at prestigious journals and prominent
conferences, such as IEEE T-PAMI, T-IP, T-NNLS, T-
CYB, T-CSVT, CVPR, ECCV, IJCAI. He served/serves as an Associate Editor
of the IEEE Transactions on Cybernetics,IEEE Transactions on Systems, Man,
Cybernetics:Systems. His current research interests include image processing,
wavelet analysis and its applications, pattern recognition, machine earning,
and computer vision.
Qinmu Peng
is currently an Assistant Professor
in the School of Electronics Information and Com-
munications, Huazhong University of Science and
Technology (HUST), China. He received the Ph.D.
degree in computer science from Hong Kong Baptist
University, Hong Kong, in 2015. His research results
have expounded in 20+ publications at prestigious
journals and prominent conferences, such as PNAS,
IEEE T-NNLS, T-SMCA, T-HMS, T-MM, IJCAI. His
current research interests include multimedia analysis,
computer vision, and medical image analysis.
Zehong Cao
is a Lecturer with the Discipline of
Information and Communication Technology, School
of Technology, Environments and Design, University
of Tasmania, Australia, and an Adjunct Fellow with
the School of Computer Science, UTS. He received
the Ph.D. degree in information technology from
the University of Technology Sydney, Australia, in
2017.He has authored more than 50 papers published
in top-tier International Conferences such as AAMAS
and AAAI, and IEEE and ACM Transactions Series,
with 5 ESI highly cited papers. Dr. Cao is the Leading
Guest Editor of IEEE Transactions on Fuzzy Systems, and IEEE Transactions
on Industrial Informatics (2020), and the Associate Editor for Neurocomputing
(2019-) and Scientific Data (2019-). His current research interests include
computer vision, machine learning, computational intelligence, and bio-signal
processing.
Weiping Ding
(M’16-SM’19) is currently a Full
Professor with the School of Information Science and
Technology, Nantong University, Nantong, China. He
received the Ph.D. degree in Computation Applica-
tion, Nanjing University of Aeronautics and Astro-
nautics (NUAA), Nanjing, China, in 2013. He was a
Visiting Scholar at the University of Lethbridge(UL),
Alberta, Canada, in 2011. From 2014 to 2015, He
is a Postdoctoral Researcher at the Brain Research
Center, National Chiao Tung University (NCTU),
Hsinchu, Taiwan. In 2016, He was a Visiting Scholar
at the National University of Singapore (NUS), Singapore. From 2017 to
2018, he was a Visiting Professor at the University of Technology Sydney
(UTS), Ultimo, NSW, Australia. He has published more than 80 research
peer-reviewed journal and conference papers, including IEEE T-FS, T-NNLS,
T-CYB, T-BME, T-II, T-ETCI, T-ITS and CIKM, etc. He has three ESI highly
cited papers. Dr. Ding currently serves on the Editorial Advisory Board of
Knowledge-Based Systems and Editorial Board of Information Fusion,Applied
Soft Computing. He serves/served as an Associate Editor of IEEE Transactions
on Fuzzy Systems,Information Sciences,Swarm and Evolutionary Computation,
and Journal of Intelligent & Fuzzy Systems, and Co-Editor-in-Chief of Journal
of Artificial Intelligence and System. He is also the Leading Guest Editor
of IEEE Transactions on Evolutionary Computation and Information Fusion
(2020). His main research directions involve data mining, granular computing,
evolutionary computing, machine learning, and big data analytics.

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 1
Supplementary materials to “CDE-GAN:
Cooperative Dual Evolution Based Generative
Adversarial Network”
Shiming Chen1, Wenjie Wang1, Beihao Xia1, Xinge You1†,Senior Member, IEEE, Qinmu Peng1, Zehong Cao2,
and Weiping Ding3,Senior Member, IEEE
1Huazhong University of Science and Technology, 2University of Tasmania, 3Nantong University
{shimingchen, wangwj54, xbh_hust, youxg, pengqinmu}@hust.edu.cn,
zehong.cao@utas.edu.au,ding.wp@ntu.edu.cn
In the main paper, we have displayed some samples generated by our proposed CDE-GAN on various natural image datasets,
which have been given to show the effectiveness of the proposed CDE-GAN model. In this supplementary, we present more
experimental results:
•Visual comparisons of different methods on CIFAR-10 and LSUN-Bedrooms.
•More generated results on three image benchmark datasets, i.e., CelebA, CIFAR-10, and LSUN-Bedrooms.
A. Visual Comparison for Different Methods
(a) D2GAN [17] (b) acGAN [19] (c) CDE-GAN
Fig. 1: Visual comparisons of different methods on CIFAR-10. The samples generated by different methods are provided by the
original literatures.
(a) LSGAN‡[45] (b) WGAN-GP‡[15] (c) CDE-GAN
Fig. 2: Visual comparisons of different methods on LSUN-Bedrooms. ‡Results provided by [15].
†Corresponding author

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 2
B. Results Generated by CDE-GAN
Fig. 3: Samples generated by CDE-GAN on CelebA.

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 3
Fig. 4: Samples generated by CDE-GAN on CIFAR-10.

IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 4
Fig. 5: Samples generated by CDE-GAN on LSUN-Bedrooms.