Content uploaded by Sebastian Lang
Author content
All content in this area was uploaded by Sebastian Lang on Feb 14, 2021
Content may be subject to copyright.
Expert Systems With Applications 172 (2021) 114666
Available online 4 February 2021
0957-4174/© 2021 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/4.0/).
NeuroEvolution of augmenting topologies for solving a two-stage hybrid
ow shop scheduling problem: A comparison of different solution strategies
Sebastian Lang
a
,
*
, Tobias Reggelin
b
, Johann Schmidt
c
, Marcel Müller
d
, Abdulrahman Nahhas
e
a
Fraunhofer Institute for Factory Operation and Automation IFF, Sandtorstraße, 22, 39106 Magdeburg, Germany
b
Department of Logistics Systems, Institute of Logistics and Material Handling Systems, Otto von Guericke University Magdeburg, Universit¨
atsplatz, 2, 39106 Magdeburg,
Germany
c
Articial Intelligence Lab, Institute for Intelligent Cooperating Systems, Otto von Guericke University Magdeburg, Universit¨
atsplatz 2, 39106 Magdeburg, Germany
d
Department of Logistics Systems, Institute of Logistics and Material Handling Systems, Otto von Guericke University Magdeburg, Universit¨
atsplatz 2, 39106 Magdeburg,
Germany
e
Very Large Business Application Lab, Department of Technical and Business Information Systems, Otto von Guericke University Magdeburg, Universit¨
atsplatz 2, 39106
Magdeburg, Germany
ARTICLE INFO
Keywords:
Articial intelligence
Machine learning
Neural network
Neuro evolution
Scheduling
Flow shop
ABSTRACT
The article investigates the application of NeuroEvolution of Augmenting Topologies (NEAT) to generate and
parameterize articial neural networks (ANN) on determining allocation and sequencing decisions in a two-stage
hybrid ow shop scheduling environment with family setup times. NEAT is a machine-learning and neural ar-
chitecture search algorithm, which generates both, the structure and the hyper-parameters of an ANN. Our
experiments show that NEAT can compete with state-of-the-art approaches in terms of solution quality and
outperforms them regarding computational efciency. The main contributions of this article are: (i) A com-
parison of ve different strategies, evaluated with 14 different experiments, on how ANNs can be applied for
solving allocation and sequencing problems in a hybrid ow shop environment, (ii) a comparison of the best
identied NEAT strategy with traditional heuristic and metaheuristic approaches concerning solution quality and
computational efciency.
1. Introduction
The two-stage hybrid ow shop scheduling (THFS) problem is a well-
known combinatorial optimization problem in the eld of production
and logistics. In this problem, a set of jobs must complete two processing
stages. Each processing stage consists of at least one machine, and
minimum one stage consists of two or more parallel machines. All jobs
must visit both processing stages in the same technological order. Every
machine of each stage can process every job. However, each job is dis-
patched to exactly one machine in each stage (Emmons & Vairaktarakis,
2013; Pinedo, 2016). In the past, researchers proposed several methods
to solve the generalized form of the THFS problem, the hybrid ow shop
(HFS) problem with n stages. Ruiz and V´
azquez-Rodríguez (2010) pre-
sent a comprehensive literature review of various HFS congurations
and solution techniques investigated between 1970 and 2009.
Many articles propose mixed-integer linear programs (MILP) for
modelling THFS problems. For instance, Guinet et al. (1996) and Guinet
and Solomon (1996) present general MILP formulations of the HFS
problem that minimizes the makespan and the total tardiness, respec-
tively. Common solution techniques are classied as exact solution
techniques, heuristics or metaheuristics. Exact solution techniques are
mostly based on the branch and bound algorithm. They guarantee to
nd the optimal solution, if executed long enough. However, exact so-
lution techniques can only solve small problem instances in reasonable
time, since the THFS problem is NP-complete in the strong sense
(Hoogeveen et al., 1992; van Wassenhove et al., 1992).
Heuristics are constructive solution approaches that are able to
generate a schedule in very short time. The range of heuristics extends
from simple dispatching rules like EDD (earliest due date) or SPT
(shortest processing time), to more complex and sophisticated algo-
rithms, such as divide-and-conquer strategies (Fernandez-Viagas et al.,
2020), to heuristics that are created for a specic problem, for instance
the distributed permutation ow shop problem (Pan et al., 2019). The
variety of heuristics differs considerably in terms of solution quality and
* Corresponding author.
E-mail addresses: sebastian.lang@iff.fraunhofer.de (S. Lang), tobias.reggelin@ovgu.de (T. Reggelin), johann.schmidt@ovgu.de (J. Schmidt), marcel1.mueller@
ovgu.de (M. Müller), abdulrahman.nahhas@ovgu.de (A. Nahhas).
Contents lists available at ScienceDirect
Expert Systems With Applications
journal homepage: www.elsevier.com/locate/eswa
https://doi.org/10.1016/j.eswa.2021.114666
Received 3 November 2020; Received in revised form 25 January 2021; Accepted 25 January 2021
Expert Systems With Applications 172 (2021) 114666
2
implementation effort. For instance, simple dispatching rules are
straightforward to implement and capable for real-time decision-mak-
ing. Especially for large and constraint problems, however, the resulting
production schedules are often far away from the optimal solution
(Chandra & Talavage, 1991). Problem tailored heuristics, on the other
hand, generate solutions that might be non-optimal, but that are usually
of sufcient quality. However, the fact that these heuristics are tailored
to specic problems could be a barrier for industrial applications, in
particular for small and medium-sized companies. Considering the di-
versity of industrial production systems, there is a large number of
heuristics for a wide range of production scheduling problems. For
instance, T’kindt and Billaut (2006) present more than 60 different
heuristics for different shop oor congurations. Thus, a company re-
quires extensive domain knowledge in operations research as well as in
developing and implementing algorithms to deploy a problem tailored
heuristic for its production planning and control.
Against this background, metaheuristics are increasingly used for
industrial production scheduling, for instance evolutionary algorithms
in German automotive companies (Klug, 2017). The generic, random
based search strategies of metaheuristics (Blum & Roli, 2003) are often
inspired by physical processes, e.g. water wave optimization (Zhao
et al., 2020), biological phenomena, e.g. genetic algorithms (Chiou
et al., 2012), swarm behaviour, e.g. particle swarm optimization (Zhao
et al., 2019) or human behaviour, e.g. harmony search methods (Zhao
et al., 2017). Metaheuristics are therefore easy to apply to any kind of
optimization problem. However, their random search for solutions
might be also a drawback. On the one hand, it can take a long time for
metaheuristics to converge to a sufcient solution. On the other hand, it
is not possible to predict when a metaheuristic will nd a solution of
sufcient quality. For this reason, they are only limited suitable to
support production planning and control in highly dynamic and sto-
chastic HFS environments, in which the input data of the optimization is
subject to short-term random and hardly predictable uctuations.
In this article, we analyse a THFS problem with family setup times.
Thus, a machine requires additional time for preparation, if two suc-
ceeding jobs belong to different product families. The problem relies on
an industrial case study of a printed circuit board (PCB) assembly.
Aurich et al. (2016) and Nahhas et al. (2016) already investigated the
metaheuristics tabu search, simulated annealing and genetic algorithm
as well as a self-developed integrated simulation-based optimization
(ISBO) heuristic to solve the problem. The authors compare the results
with a setup minimizing family production strategy, which the company
currently applies. As expected, the metaheuristics and the ISBO
outperform the company’s strategy in terms of makespan and total
tardiness minimization. The authors report for each metaheuristic a
computational time of several hours to nd sufcient production
schedules for problem instances with 143 to 175 jobs. However, the PCB
manufacturer sometimes requires feasible scheduling decisions in much
shorter time, as the company acts within a highly dynamic production
environment. In particular, new jobs arrive randomly and sometimes
need to be scheduled at very short notice. The reason for this is that
customers are able to place new orders directly in the PCB manufac-
turer’s ERP system. Furthermore, some of the key customers demand
very short lead times between the placement and the delivery of orders.
The ISBO meets the requirements of the company, since the heuristic is
able to construct a solution of approximately the same quality as the
metaheuristics in 8 to 18 s. However, adapting the ISBO to other prob-
lems requires certain knowledge in the development of optimization
heuristics, as the authors tailored the ISBO to the specications of the
company’s production environment.
A promising alternative to traditional solution techniques are
methods (e.g. reinforcement learning (RL)) and models (e.g. articial
neural networks (ANN)) of machine learning (ML), because they enable
to compute decisions in real-time by incorporating multiple information
of a given environment, for instance a production system. Thus, research
activities have recently returned to the investigation of ML methods for
solving production scheduling problems.
This research trend is not fundamentally novel. Many articles pub-
lished between 1985 and 2000 describe supervised learning strategies
with fully connected Hopeld networks or unsupervised learning stra-
tegies with Kohonen maps and elastic networks to compute solutions for
combinatorial optimization problems. These ML-based solution tech-
niques were already reviewed by several authors (Burke & Ignizio, 1992;
Çalis¸ & Bulkan, 2015; Looi, 1992; Potvin & Smith, 2003; Smith, 1999;
Syed & Pardalos, 2013). However, Hopeld networks turned out to be
prone to local minima and difcult to parameterize (Wilson & Pawley,
1988), while Kohonen maps and elastic networks are considered to be
only suitable for low-dimensional traveling salesman and vehicle rout-
ing problems, as both methods aim to explore the geometry of a problem
(Potvin & Smith, 2003).
The current research is driven by recent methods for learning
multilayered ANN, which are summarized under and term ‘Deep
Learning’ (LeCun et al., 2015). There are only few articles describing the
application of deep supervised learning for production scheduling
problems. The reason might be that producing optimal training labels is
time consuming, as it requires to nd the optimal solution for multiple
problem instances. For instance, Zang et al. (2019) present a deep ANN
based scheduler for a job shop problem. The authors use branch and
bound and a genetic algorithm to generate training labels. The majority
of ML related scheduling research is currently focussing on deep rein-
forcement learning (DRL) methods. In contrast to supervised learning,
the benet of DRL is that the training of a model does not require labeled
data. Instead, a reward function generates training signals by evaluating
scheduling decisions with a positive or negative value. Most of the
related articles describe the application of DRL for selecting dispatching
rules, such as EDD and SPT, depending on the state of the production
system (Lin et al., 2019; Luo, 2020; Shiue et al., 2018; Wang et al.,
2020). In this case, the agent’s decisions only have an indirect effect on
the scheduling of jobs, as the selected dispatching rule is responsible for
job scheduling. Another way of using DRL for production scheduling is
to let the agent decide directly about the scheduling of jobs. This
approach is reported, for instance, by Stricker et al. (2018) and
Waschneck et al. (2018) for the semiconductor industry, where the
agents are responsible for both, the allocation and sequencing of jobs.
However, although DRL methods are less costly in terms of generating
training data, the design of a suitable reward function and ANN topology
still requires a lot of experimental effort (Irpan, 2018).
Against this background, we investigate the ML algorithm Neuro-
Evolution of Augmenting Topologies (NEAT) for generating and
parameterizing ANNs that compute scheduling decisions in real-time.
NEAT is a neural architecture search algorithm, based on genetic algo-
rithm, that identies both, the topology and the hyper-parameters (e.g.
weights, biases) of an ANN. Our motivation to investigate NEAT for
production scheduling is based on the following hypotheses: (i) During
inference, the ANNs generated by NEAT determine scheduling decisions
in real-time. Thus, parameterized ANNs compute solutions for sched-
uling problems in much shorter time as exact solution techniques and
metaheuristics. (ii) Even the runtime that NEAT requires to converge to
a suitable ANN may be less than the runtime of a metaheuristic to
converge to a solution for a given problem instance. This assumption is
based on the observation that NEAT’s initial population only considers
ANNs with the most simple topology (i.e. an input and an output layer
without any hidden neurons in between), which NEAT complexies only
gradually from generation to generation. (iii) In contrast to dispatching
rules, ANNs are able to incorporate information from their environment
for decision-making. Thus, we expect that ANNs, which are parame-
terized by NEAT, generate better solutions as conventional dispatching
rules (e.g. EDD, SPT), which do not consider any information from the
surrounding production system. (iv) NEAT is a metaheuristic and thus
easy to deploy for any kind of optimization problem. NEAT and the
scheduling model are implemented independently and are only con-
nected via an input/output interface. Therefore, we estimate a lower
S. Lang et al.
Expert Systems With Applications 172 (2021) 114666
3
effort for implementing NEAT compared to many problem tailored
heuristics and DRL methods. (v) In contrast to DRL methods, NEAT is not
only suitable for optimizing the parameters of an ANN, but also for
searching an optimal ANN topology. By this means, NEAT reduces the
human effort for engineering, as NEAT automates the design process for
ANNs.
NEAT for production scheduling problems has not yet been
researched. In fact, we could only nd three publications that describe
the application of neuro-evolution strategies (including NEAT) for
scheduling problems. Only one publication investigate neuro-evolution
for a scheduling problem related to production and logistics (Mao et al.,
2007), while the other two address resource allocation on chip-
multiprocessors (Gomez et al., 2001) and job schedling on servers
(Whiteson, 2005). However, each problem can be reduced to a sched-
uling problem with identical parallel machines, which is fairly less
complex than a THFS with family setup times. For the present problem,
we already showed in a previous paper that NEAT can provide sufcient
solutions when used to parameterize an ANN for job sequencing (Lang
et al., 2020). The contribution of this article is an extensive analysis of
the NEAT algorithm for ow-shop scheduling. In particular, we analyse
ve different strategies for encoding scheduling problems as ML tasks
and conduct several computational experiments to evaluate the perfor-
mance and efciency of each strategy. In this context, we also present an
improved version of our job sequencing strategy of our previous paper
(Lang et al., 2020). Finally, we compare the best identied NEAT stra-
tegies with traditional heuristic and metaheuristic approaches in terms
of solution quality and computational efciency.
The remaining article comprises six sections. The next section sum-
marizes how the NEAT algorithm operates. In the third section, we
provide a description of the THFS problem on which we evaluate our
solution strategies. In section four, we present our investigated strategies
for adapting NEAT to scheduling problems. The fth section summarizes
our design of experiments. Section six contains the computational re-
sults. The last section is dedicated to the conclusion and outlook.
2. Neuroevolution of augmenting topologies
NEAT is a genetic algorithm that searches for suitable ANN topol-
ogies and appropriate parameters for a given ML-task.
A genetic algorithm is a metaheuristic whose search strategy is
inspired by the process of natural evolution. Any genetic algorithm starts
with randomly generating an initial set (population) of solution candi-
dates (genomes). Afterwards, the algorithm randomly modies (muta-
tion) single solutions and/or recombines (crossover) arbitrary pairs of
solutions to new ones over several iterations (generations), until a
condition of termination is met, e.g. a maximum number of generations,
a threshold of the objective function, etc. (Reeves, 2010).
According to Haykin (1999), an ANN is a “[…] massively parallel
distributed processor consisting of simple processing units, which has a
natural propensity for storing experiential knowledge and making it
available for use.” This denition may appear vague, since Haykin
considers an ANN to be an adaptive machine that can be either physical
or digital. In our work, however, the term ANN refers to an algorithmic
model represented by a computational graph, which consists of a set of
nodes and edges. A node is called ‘Neuron’. It merges one or several
input streams with a net input function and transforms them into a single
output value by computing an activation function. Neurons are orga-
nized in layers. An ANN consists of at least one layer. Most ANN ar-
chitectures involve three layers or more, where a single input layer
receives the input data, a single output layer computes the results, and
the layers in between are responsible for the detection of dependencies
in the input data. A neuron is connected over edges to other neurons,
which usually belong to adjacent layers. These connections are associ-
ated with weights and called ‘Synapses’. They determine the sensitivity
and the responsiveness of a neuron to a value modication within an
upstream neuron.
NEAT was introduced by Stanley and Miikkulainen (2002a), Stanley
and Miikkulainen (2002b). Due to its typical characteristics as a genetic
algorithm, NEAT shares similarities with reinforcement learning
methods. For instance, both methods do not require any labeled training
data, because learning signals are generated by a tness function (ge-
netic algorithm) / a reward function (reinforcement learning), which
evaluates the solution quality of any ANN output. NEAT is considered as
the rst neuro-evolution strategy that can efciently adapt the structure
of an ANN, mainly because of four features (Stanley, 2004; Stanley and
Miikkulainen, 2002b):
(i) In contrast to other neuro-evolution approaches, the genetic
encoding of NEAT genomes is dynamic and expandable, i.e. not
restricted by a xed dimension. Each genome consists of a set of
neuron genes and synapse genes. A neuron gene comprises all
information of a neuron, i.e. its bias, response and activation
function. A synapse gene consists of an id, which indicates the
interconnected neurons, the current weight of the interconnec-
tion, the status of the interconnection (enabled/disabled), and its
innovation number, which is necessary to determine whether two
genomes can perform a crossover. NEAT provides two mutation
features to extend genomes and thus to adapt the structure of
ANNs. A synapse mutation generates a new connection between
two unconnected neurons, while a neuron mutation removes an
existing connection and adds a new neuron instead, which
maintains a connection to the previously connected neurons.
Fig. 1 illustrates the encoding of genomes in NEAT as well as an
example for both, a neuron and synapse mutation. The exible
and expandable encoding is highly efcient, because the size of
genomes corresponds to the amount of encoded information. The
encoding also provides high scalability, as NEAT can construct
ANNs of any complexity by simply adding additional information
to existing genomes.
(ii) A meaningful crossover of information between genomes that
represent different topologies is challenging, as network topol-
ogies of different sizes also require different genome dimensions
to encode all information. To deal with this problem, the algo-
rithm keeps track of the origin of each synapse gene by assigning
to each gene a unique innovation number. It is not necessary to
track also the origin of neuron genes, as the IDs of the synapse
genes already comprise all necessary information about the
structure of an ANN, i.e. which neurons maintain a connection to
each other. Fig. 2 gives an example of a crossover between two
genomes that are partly different in terms of their synapse genes.
When pairing two genomes, NEAT randomly selects matching
synapse genes, i.e. genes that are present in both parent genomes.
For the remaining genes that are different, NEAT only considers
the genes of the parent genome with better tness. In case that
both parents have the same tness, NEAT adds the unmatching
synapse genes of both parents to the offspring genome.
(iii) The algorithm divides the total population in different species
according to topological similarities, so that genomes compete
only within their species. By this means, the authors address the
problem of directly rejecting genomes with structural innovations
from the population, because they usually cannot immediately
compete with longer established genomes having been able to
optimize the synapse weights of their current network topology
over several generations. NEAT divides the population into spe-
cies depending on the genetic distance between the genomes. In
our case, NEAT calculates the genetic distance according to the
equations (1) to (3).
D=dn+ds(1)
S. Lang et al.
Expert Systems With Applications 172 (2021) 114666
4
dn=cn*Δn
max(N(g1),N(g2)) (2)
ds=cs*Δs
max(S(g1),S(g2)) (3)
D, dn and ds denote the overall, neuronal and synaptic genetic dis-
tance. The neuronal and synaptic genetic distance are calculated
similarly, by dividing the amount of different neurons Δn/synapses
Δs by the maximum number of neurons/synapses of the genomes to
be compared. The factors cn and cs are specied by the user and
denote the compatibility disjoint coefcients for ne-tuning the
neuron and synaptic genetic distances. In each generation, NEAT
recalculates for each genome the afliation to a species. For each
species, only the genome with the highest tness remains as repre-
sentative of the species. For all other genomes, NEAT iterates over
the existing species and computes the genetic distance between the
genome and the species’ representative. The genome will be assigned
to the species, if the genetic distance exceeds a user-dened
compatibility threshold. If a genome cannot be assigned to an
existing species, NEAT creates a new species with the genome as
representative.
(iv) Other neuro-evolution implementations suffer from low compu-
tational efciency, because they already need to maintain
different topologies in the initial population. That leads to high-
dimensional genomes even before the rst iteration starts. As
discussed above, for many neuro-evolution approaches it is
necessary to consider different topologies in the initial popula-
tion, as a signicant change in the topology will likely lead to an
initial tness loss. Due to the concept of specication, NEAT does
not require to consider different network structures while
initializing a population. Thus, initial genomes do not consider
hidden neurons and differ only in terms of the encoded parame-
ters like weights and biases. NEAT extends the topology of ANNs
incrementally, and only those topologies survive that can
compete with other genomes of the same species. NEAT does not
restrict the number of hidden neurons. Thus, the algorithm is
theoretically applicable for problems of any complexity.
We use the open-source library neat-python (McIntyre et al., 2019)
to implement the different solution strategies. Fig. 3 shows NEAT as ow
chart. In the following, we briey explain the illustrated steps of the
algorithm.
NEAT requires a set of user-dened hyper-parameters and a tness
function for initialization. The hyper-parameters (NEAT conguration)
Fig. 1. Genetic encoding and mutation features in NEAT: The genome on the left-hand side undergoes rst a neuron mutation and thereafter a synapse mutation. In
the given example, NEAT randomly removes a synapse ((I2, O1)), creates a new neuron with random attributes (H1), and initializes randomly weighted synapses ((I2,
H1); (H1, O1)) to the previously connected neurons. Thereafter, NEAT creates a new synapse ((I1, H1)) with random weight between two randomly selected un-
connected neurons (I1; H1).
S. Lang et al.
Expert Systems With Applications 172 (2021) 114666
5
are important for controlling the crossover and mutation of genomes.
The tness function, on the other hand, evaluates the solution quality of
the genomes depending on the scheduling decisions made. We will
discuss both, the hyper-parameters and the tness function in detail,
when describing the design of experiments and the different NEAT so-
lution strategies. When initializing the population, NEAT only creates
genomes with an input and output layer. Thus, the genomes of the rst
generation only differ in terms of their parameters, such as synapse
weights and biases, but not in terms of their network topologies. After
evaluating the tness of each genome, the algorithm aborts if a user-
dened termination criterion is met, e.g. exceeding a certain number
of generations or a threshold of the tness criterion. Otherwise, the al-
gorithm evolves the population by rst performing several random
crossovers between genomes, followed by several random mutations in
Fig. 2. Exemplary crossover of two genomes. Both parent genomes are matching in terms of the synapse genes with innovation number 1 and 2. In the given
example, NEAT pick both matching genomes to construct the offspring genome. Furthermore, NEAT considers all unmatching genes in the offspring genome, which
indicates that the parent genomes have the same tness.
S. Lang et al.
Expert Systems With Applications 172 (2021) 114666
6
the resulting offspring. Besides structural mutations, which we have
explained in Fig. 1, NEAT might also perform non-structural mutations,
for instance concerning the synapse weights, neuron biases and re-
sponses. Non-structural mutations are subject to a user-dened proba-
bility and a zero-centred normal distribution with a user-dened
standard deviation. In case of a non-structural mutation, NEAT samples
a number from the normal distribution and adds the number to the
current value of the parameter. Before recalculating the tness of each
genome, NEAT divides the population into species according to the ge-
netic distance of the genomes to the representatives of every species.
3. Two-stage hybrid ow shop scheduling problem with family
setup times
The following section provides a summary of the THFS problem with
family setup times, which we use to benchmark our NEAT solution
strategies. We implement the problem as discrete-event simulation
(DES) model, in order to evaluate the tness of NEAT genomes. The
second part of this section describes the implementation of the DES
model in detail.
3.1. Problem statement
The considered problem originates from a real use case of a PCB
manufacturer. The investigated ow shop has two production stages and
thus represents a THFS. The rst stage consists of four identical parallel
machines, i.e. Surface Mounting Device (SMD) placement machines. The
SMD machines of the rst stage are responsible for mounting product
family specic components on the PCBs. The second production stage
comprises ve identical parallel machines for Automated Optical In-
spection (AOI).
The rst production stage underlies major and minor family-
sequence-dependent setup times. This implies that the sequence of
jobs on any SMD machine is subject to both major and minor setup times
for preparing the machine. An SMD requires a major setup of 65 min if
the job to be processed belongs to a different family compared to the
family of the preceding job. Otherwise, a minor setup of 20 min prepares
the SMD to process the next job of the same family. The second pro-
duction stage underlies sequence-independent setups. The setup time
between two jobs is always 25 min. Fig. 4 illustrates the structure of the
underlying production system.
Fig. 3. Flow chart of the NEAT algorithm.
S. Lang et al.
Expert Systems With Applications 172 (2021) 114666
7
The considered THFS is a deterministic scheduling problem, in which
stochastic inuences, such as machine breakdowns, are not taken into
account. Based on the experimental design of past publications, with
which we will compare the results of our approach, we assume that all
information of the jobs to be released are available before the start of the
optimization. The capacity of each queue in front of a machine is
innite. Further problem characteristics are:
•Each job belongs to a specic job family
•Each job has a specic processing time for the rst and second pro-
duction stage that is not related to the job’s family
•Each job has a specic due date, which is not related to the job’s
family. It is preferable but not necessary to complete a job before its
due date.
•At any given time, only a single machine in the rst stage can process
a specic product family, i.e. it is not allowed that more than a single
machine produces the same product family at any point in time. This
restriction is necessary, since the company has only one setup kit for
each product family.
•The problem is a permutation ow shop. For HFS problems, this
constraint implies that jobs passing the rst stage are scheduled ac-
cording to an initial job sequence. For the second stage, however,
each job is allocated to the machine that will become available at the
earliest possible time (Emmons & Vairaktarakis, 2013). This
constraint corresponds to the production process of the company and
is justied by the fact that scheduling decisions for SMD machines
possess the greatest inuence on the performance of such production
systems (Csaszar et al., 2000).
3.2. Problem instances
We evaluate the different solution strategies of NEAT based on four
problem instances. The problem instances originate from the ERP system
of the considered PCB manufacturer. The problem instances represent
historical production data, each covering a planning period of approx-
imately 3 weeks. The entire datasets are available online.
1
For all ex-
periments, which we present in Section 5, we use dataset 2 and dataset 3
for training and dataset 1 and dataset 4 for testing the solution quality of
the best identied genome. In the following, we want to explain our
train-test split based on a brief statistical analysis of the datasets.
First, we quantify the amount of outliers, as a prevailing amount of
outliers can have a negative impact on the ANNs performance. We
dene outliers equivalent to extreme values. The tailness and hence the
amount of extreme values in a data distribution can be quantied by the
kurtosis. We rst calculate the kurtosis for each dataset and each feature.
Then, we calculate for each dataset the average normalized kurtosis over
all features. The results are collected in Table 1. As we can see, dataset 1
Fig. 4. THFS with family setup times: System structure and processes.
Table 1
Statistical analysis of the considered problem instances (dj:due date, tSMD :
processing time on SMDs, tAOI :processing time on AOIs,
μ
:mean,
σ
:standard
deviation, M:median, M:absolute deviation of median, S:skewness, K:
kurtosis).
Dataset 1
(Testing)
Dataset 2
(Training)
Dataset 3
(Training)
Dataset 4
(Testing)
Number of
jobs
164 170 175 143
Number of
job families
41 37 36 35
dj min 1305 1305 1305 1305
max 27,405 27,405 27,405 27,405
μ
13,877.56 13,779.26 14,034.34 14,072.10
σ
8051.04 7852.94 7835.24 7690.56
M 14,355.00 13,050.00 13,050.00 14,355.00
M 6525.00 6525.00 6525.00 6525.00
S 0.05 0.12 0.08 0.08
K 0.20 0.19 0.18 0.16
tSMD min 4 2 4 4
max 3142 3736 3293 3209
μ
333.45 366.74 350.14 393.36
σ
486.02 506.26 517.21 521.68
M 153.50 168.00 162.00 192.00
M 116.00 138.00 129.00 160.00
S 3.21 2.96 2.98 2.59
K 69.17 56.08 62.68 34.97
tAOI min 4 3 5 3
max 4351 5590 3528 4300
μ
442.24 521.79 427.07 554.50
σ
648.89 800.88 584.09 736.85
M 219.00 251.50 177.00 277.00
M 152.00 193.00 145.00 229.00
S 3.49 3.43 2.52 2.59
K 82.85 98.04 35.49 34.23
avg.K 0.4588 0.3928 0.3913 0.4950
1
https://www.ilm.ovgu.de/hicss_problem_instances.html.
S. Lang et al.
Expert Systems With Applications 172 (2021) 114666
8
and 4 have the highest average normalized kurtosis, and hence the
largest number of outliers. We conclude that when we use dataset 2 and
dataset 3 for training, the impact of outliers is less signicant.
Second, we measure the correlation between the features, as corre-
lating features introduce a potentially harmful bias. We can quantify the
correlation by computing the covariance matrix. Fortunately, we found
that no signicant correlation exists between features.
Fig. 5 shows the distribution of the individual features for all data-
sets. As stated by Gong et al. (2019), a high data diversication improve
the efciency of obtaining discriminative features from samples. Look-
ing on the shapes of the distributions in Fig. 5, all four datasets seem to
be highly similar. However, in order to prove that the datasets do not
share any data points with each other, we can conduct a brute force
search for shared data points, which is feasible in tractable time due to
the low cardinality of the sets. We loop over all samples in all datasets
and check for intersections. We found that no data points are shared
among datasets. This intersection test is, of course, not able to accurately
measure statistical similarities between the datasets. For this reason, we
are interested in the concept drift in the datasets. Because we are using
some data for training the model and some for testing the performance
on unseen data, the similarity between the datasets is important.
If we train on conceptually different data compared to the test data, a
good performance on the test data would indicate a generalisation of the
training data. A commonly applied metric to measure non-similarity
between data is the Wasserstein metric. This measurement, however,
is solely applicable to univariate distributions. A straightforward and
computationally efcient solution for multivariate distributions is the
aggregated Wasserstein metric wp, which is mathematically dened as:
wpDi,Dj=1
|F|
F,F′∈F
inf
γ∈Γ(D(F)
i,D(F′)
j)RxR
||x−y| |pdγ(x,y)
1/p
(4)
where F is the set of features (due date, family, SMD processing time,
AOI processing time) and Γ(∙)denotes the collection of probability
measures. We employ the rst order Wasserstein metric, i.e. p=1 and
the l1 norm. We compute the average Wasserstein metric for all dataset
tuples, i.e. i,j∈ [1,⋯,4]. Since the Wasserstein metric has no upper
bound, i.e. W∈ [0,∞),we provide the following comparative value: The
Wasserstein distance between 1000 random samples of two standard
normal distributions is 0.0297 ±1.04 ×10−17 (mean ±standard devi-
ation). If we increase the variance of the second bell curve, i.e.
N1(0.0,1.0)and N2(0.0,10.0), the Wasserstein distance increase to
7.1963 ±1.78 ×10−15. The results are summarized in Fig. 6. Dataset 1
and 4 show the highest average Wasserstein distance. Due to the small
number of outliers, we use dataset 2 and 3 for training. We use dataset 1
and 4 for testing, as they show the largest distance to the other datasets
and thus differ the most from the training data.
3.3. Objective function
A production schedule results from a set of allocation and sequencing
Fig. 5. Data distribution (histograms) of (a) SMD processing times, (b) AOI processing times, (c) due dates, and (d) families for dataset 1 (blue), dataset 2 (red),
dataset 3 (green), and dataset 4 (violet). (For interpretation of the references to colour in this gure legend, the reader is referred to the web version of this article.)
Fig. 6. Average aggregated Wasserstein metric to measure the similarity be-
tween the datasets. Bright colours (low values) indicate similarity, while dark
colors (high values) indicate non-similarity. (For interpretation of the refer-
ences to colour in this gure legend, the reader is referred to the web version of
this article.)
S. Lang et al.
Expert Systems With Applications 172 (2021) 114666
9
decisions. The number of allocation decisions and the number of
sequencing decisions correspond to the number of jobs to be scheduled.
An allocation decision implies the assignment of a job to a specic SMD
machine. A sequencing decision determines at which position a job is
considered within a production sequence.
We assess the quality of a production schedule based on the resulting
total tardiness T and makespan CMax. The total tardiness is the sum of
tardiness over all jobs n, as demonstrated by Eq. (4). According to Eq.
(5), the tardiness of a single job Tj is the difference between its
completion time Cj and its due date dj, if the job is completed after its
due date. The makespan CMax is the maximum completion time over all
jobs, as described by equation (6)
T=
n
j=1
Tj∀j∈ (1,⋯,n)(5)
Tj=max(0,Cj−dj)(6)
CMax =maxCj∀j∈ (1,⋯,n)(7)
According to the preferences of the company, the primary objective
is to reduce the total tardiness. In case that there are several solutions
providing a total tardiness of zero, the company prefers the solution with
the lowest makespan. Against this background, we select the minimi-
zation of total tardiness as objective function.
3.4. Modeling the problem as discrete-event simulation model
Based on the problem statement, we develop a DES model that
considers all processes and constraints of the THFS problem. The pur-
pose of the DES model is to provide training signals for NEAT by
computing the tness, i.e. the objective function, for each genome.
Although there are many off-the shelf tools for DES modeling, we
decided for the open-source Python library salabim to develope the
model (van der Ham, 2018), in order to reduce the effort for integrating
NEAT with the DES model. Salabim has many similarities with the long-
established SIMULA language, because it resembles modeling concepts
such as activating, passivating and holding objects (Dahl & Nygaard,
1966). Fig. 7 shows the processes of the DES model as a owchart.
The progress of the simulation depends on several interrelated
methods, which are associated with ow objects (jobs) and stationary
resources (source, SMDs and AOIs). At the beginning of the simulation,
the source creates all jobs of the dataset. As soon as a new job is
initialized, it enters a job buffer, where it is stored until the remaining
jobs are created by the source. In particular, the job changes to passive
Fig. 7. Process logic of the DES model as owchart.
S. Lang et al.
Expert Systems With Applications 172 (2021) 114666
10
state by which means the execution of the job’s process logic stops. As
soon as all jobs are created, the source object sends messages to all jobs
in the job buffer to proceed the execution of the process logics. After-
wards, each job requests the allocation to an SMD line. The allocation of
jobs is either subject to the decisions of an ANN that is parsed from a
NEAT genome or managed by a simple allocation rule. Both, the allo-
cation and the sequencing of jobs, are explained in detail in the next
section. As soon as a job enters the queue of the assigned SMD, it rst
checks whether the SMD is currently in passive state. If this is the case,
the job sends an activation message to the corresponding SMD and
switches itself to passive state. The process method of each SMD is a
continuously executed loop, which interrupts every time the preceding
SMD queue is empty. If the SMD is idle, the SMD requests the next job to
be processed. Similar to the allocation of jobs, sequencing decisions are
either subject to the decisions of an ANN or conducted according to a
dispatching rule Thereafter, the process method of the SMD checks
whether the selected job belongs to the same product family as the SMDs
current setup. If so, the SMD waits 20 min to simulate a minor setup.
Otherwise, the SMD requires a major setup of 65 min. Before considering
the major setup time, the process method checks whether another ma-
chine is currently holding the desired setup type. If this is the case, the
SMD must wait for the release of the demanded setup type. It can happen
that several SMDs wait for the release of the same setup type. We solve
this conict with a queueing system. Each product family is associated
with a queue, which an SMD joins when waiting for the corresponding
setup type. If a machine changes the setup, the preceding setup type
becomes available for the rst job in the corresponding queue. After
completing the minor or major setup, the SMD object waits the job-
specic processing time tSMD before releasing the job.
The process logic of AOIs is simpler compared to those of SMDs, since
the setup of machines underlies no constraints. On the second stage, jobs
are always allocated to the machine that currently has the lowest
workload. Each AOI process the jobs in the preceeding buffer according
to the FIFO (rst in rst out) dispatching rule. When receiving the next
job, the AOI waits for 25 min in addition to the job-specic processing
time tAOI before releasing the job. The process logic of AOIs also runs in a
loop. However, the logic pauses when the preceding AOI queue does not
contain any jobs. In that case, the AOI passivates itself. The process logic
will continue from where it stopped, when the AOI returns to active
state. The reactivation of AOIs depends on the arrival of jobs in the
second stage. Every time a job enters the queue of an AOI, it reactivates
the AOI in case the machine is currently passive.
Before a job leaves the system, the DES model computes the tardiness
of the job and updates the total tardiness. If the leaving job is the last job
in the system, the model also computes the makespan before terminating
the simulation.
In order to validate our model, we use a simple animation that vi-
sualizes the states of jobs and machines while executing the simulation.
Fig. 8 shows a snapshot of the animated DES model.
4. NEAT solution strategies
We present ve different strategies to train and deploy ANNs for
computing scheduling decisions in our THFS model. The strategies differ
in terms of how the ANN approaches the problem:
•As an allocation problem or
•As a sequencing problem.
This distinction is based on the fact that any HFS problem can be
represented as a nite number of allocation problems (the assignment of
jobs to machines) and sequencing problems (the order in which jobs are
processed on a machine). Furthermore, we distinguish strategies in
terms of how the generated ANN interacts with the DES model:
•The ANN is unconnected to the DES model and passes solutions as
input parameters to the model or
•The ANN is passed to the DES model and will be integrated as call-
able allocation or sequencing method.
Fig. 9 shows four of the ve implemented strategies. In the following
subsections, we describe the two sequencing-based strategies and the
two allocation-based solution strategies. Furthermore, we introduce a
fth solution strategy that solves the allocation and sequencing problem
with two ANNs at the same time. While the rst four strategies mainly
differ in terms of the input and output layer of the generated ANNs, the
main difference of the fth strategy is the general implementation of the
NEAT algorithm.
4.1. Sequencing based solution strategies: Pre- and post-sequencing
In the sequencing based solution strategies, NEAT searches for an
Fig. 8. Animated DES model.
S. Lang et al.
Expert Systems With Applications 172 (2021) 114666
11
ANN that determines the processing order of a set of jobs. Sequencing
based strategies do not utilize ANNs for allocating jobs to machines.
Instead, a simple dispatching rule allocates a job to the machine with the
lowest workload. There are two options for sequencing:
•The generation of an initial job sequence before allocating any job to
a machine (pre-sequencing).
•The allocation of jobs to machines before assigning the job to a po-
sition within the production sequence of the allocated machine (post-
sequencing).
Fig. 10 shows the encoded ANN input and output for both, the pre-
and post-sequencing approach. The main idea of both strategies is that
the ANN analyses job attributes and may use current states of the
production system to compute a priority for each job. In both strategies,
the computed sequence priority is a value between 0 and 1. The model
sorts jobs by descending sequence priority. However, the pre-sequencing
and post-sequencing strategy differ in terms of the ANN’s input pa-
rameters and the point of time when a sequence is generated.
The pre-sequencing strategy arranges jobs to a sequence before the
simulation starts. The resulting sequence will be passed as input to the
DES model. No system states or other dynamic variables are known in
advance, as the entire sequencing process nishes before the simulation
run starts. Thus, the ANN only uses job attributes, i.e. the due date, the
job family, the processing time on the rst stage (tSMD)and the pro-
cessing time on the second stage (tAOI ), to compute the sequence prior-
ity. The SMD machines process incoming jobs according to the FIFO
strategy. We already described the pre-sequencing strategy in (Lang
Fig. 9. An overview of the implemented solution strategies. (PRE-SEQ: pre-sequencing, POST-SEQ: post-sequencing; F-ALLOC: family allocation, J-ALLOC:
job allocation).
Fig. 10. Solving the (a) pre-sequencing problem and (b) post-sequencing problem with an ANN regression model.
S. Lang et al.
Expert Systems With Applications 172 (2021) 114666
12
et al., 2020) and compared its solution quality and computational ef-
ciency with other scheduling approaches.
In the post-sequencing strategy, NEAT passes a generated ANN
directly to the DES model, which embeds the network as sequencing
method. Every time an SMD receives a new job, the ANN rst computes
the priority index of the job before deciding where to insert the job in a
sequence. Since the post-sequencing happens while the simulation is
running, the ANN can analyse dynamic system states of the THFS. We
additionally use the current setup type of the assigned SMD as a system
state to assess the sequence priority of a job. The intention is that it
might be benecial to consider jobs at the very beginning of a sequence,
if the current machine setup corresponds to the job’s family.
4.2. Allocation based solution strategies: Family and job allocation
The allocation based strategies aim to nd an ANN that assigns jobs
to SMD machines. For both allocation based strategies, we use a dis-
patching rule (e.g. FIFO) for the sequencing of jobs. Fig. 11 shows the
encoding of the ANN input and output for both allocation strategies.
The job allocation strategy integrates the ANN into the DES model,
which implies that allocation decisions are made while the simulation is
executed. In addition to the attributes of a job, the ANN receives the
current workload of all SMDs and the unseized workload at the time of
the allocation decision.
As the name indicates, the family allocation strategy does not assign
jobs to SMDs, but job families to SMDs. In contrast to the job allocation
strategy, the job families are allocated to SMDs before the simulation run
starts. We use characteristics of each job family, such as the cumulative
processing times on the rst (ΣtSMD)and second stage (ΣtAOI)of all jobs
belonging to the same family, as ANN inputs. Furthermore, We consider
due date statistics over all jobs within a family to be particularly
important, as minimizing the total tardiness is our primary objective.
For this reason, we consider the sum, minimum, mean, maximum, the
rst quantile, the second quantile (median) and the third quantile of the
due dates of a family as inputs of the ANN. Although the ANN does not
allocate jobs at simulation run time, we are able to consider dynamic
states as network input. After each allocation decision, we update the
workload and the number of jobs already allocated to each SMD. The
current workload of each SMD and the number of jobs allocated to each
SMD are further used as input parameters of the ANN. The allocation
decisions of job families to SMDs are saved in a dictionary, which the
DES model receives as input when initializing the simulation.
4.3. Joint sequencing and allocation based solution strategies: Competitive
NEAT
The rst four strategies assume that a single ANN constructs job se-
quences or allocates jobs to machines while a simple rule-based
approach does the remaining task. Although the idea of using two
ANNs to solve both problems simultaneously appears obvious, a mean-
ingful implementation of the training process requires some preliminary
conceptual considerations. The most straightforward way to train both
ANNs is to execute NEAT two times in a row to nd rst an ANN for
allocating jobs and second an ANN for sequencing jobs. During the
second execution of NEAT, the algorithm does not use a rule-based
approach for the allocation problem, but uses the ANN representation
that NEAT has determined as the best solution during the rst session.
This process can be also executed vice versa, by which NEAT searches for
an ANN for the sequencing task rst and for an ANN for the allocation
task second. However, we do not expect the best training results from
this strategy, since the rst ANN is trained to solve the THFS problem in
conjunction with a rule-based approach and not together with a second
ANN.
For this reason, we present a competitive version of the NEAT al-
gorithm to synchronize the training of both ANNs. Fig. 12 shows the
ow chart of the competitive NEAT implementation.
The main idea is that two NEAT algorithms compete in a turn-based
manner. In each turn, one of the NEAT algorithms tries to beat the best
tness of the other. At the beginning, we initialize a global tness var-
iable with a high negative value. Then, we execute for the rst time
NEAT for nding an ANN which allocates jobs to SMDs. At this moment,
NEAT cannot utilize any ANN for the sequencing problem, as the NEAT
algorithm for the sequencing task has not been executed yet. Alterna-
tively, we use the FIFO dispatching rule for the sequencing task. Due to
the low initial tness, NEAT will likely nd a solution that exceeds the
global tness value within a single generation. If a better tness value is
found, the execution of the NEAT algorithm for allocation terminates
and the current generation as well as the best genome are saved. Then,
NEAT for sequencing will try to nd an ANN representation that has a
better tness than the best genome of the previous executed NEAT
session. When evaluating the tness value of a sequencing genome, the
best ANN of the previous NEAT session is utilized for the allocation task
and vice versa. As soon as the NEAT algorithm for the sequencing
problem nds a genome that exceeds the tness threshold, the algorithm
saves the current population and the genome with the best tness value
in a local folder before terminating. The NEAT algorithm for the
Fig. 11. Solving the (a) job allocation problem and (b) family allocation problem with an ANN classication model.
S. Lang et al.
Expert Systems With Applications 172 (2021) 114666
13
allocation problem now continues to evolve the last generation of the
previous allocation experiment, instead of creating a new generation
from scratch. This process repeats, until both algorithms cannot improve
the global tness within a user-dened number of generations.
5. Design of experiments
We organize our experiments in three stages. In the rst stage, we
want to show that NEAT is fundamentally suitable as ML training
strategy for the presented problem. Furthermore, we want to show that
the solution quality of an ANN found by NEAT is better than the solution
quality of an ANN that is constructed based on design principles and
initialized with random parameters. To do so, we want to prove that
both NEAT features, the evolution of parameters as well as the evolution
of topologies, contribute to the solution quality. For this purpose, we
want to test each component of the algorithm by conducting three ex-
periments. In each experiment, an ANN is responsible for the sequencing
of jobs, while a heuristic allocates jobs to SMDs according to the mini-
mum workload (this corresponds to the rst experiment in Table 2). For
the rst and second experiment of the rst stage, we manually design an
ANN with four input neurons, a single output neuron and nine hidden
neurons. While the number of input and output neurons correspond to
Fig. 10(a), we select the number of hidden neurons according to Hecht-
Nielsen’s (1987) equation 2n+1, where n corresponds to the number of
input neurons. Hecht-Nielsen proved that any continuous function can
be represented by an ANN with a single hidden layer and precisely 2n+1
hidden neurons (Stathakis, 2009). In the rst experiment of the rst
stage, we initialize 250 instances of our customized ANN with random
parameters. In the second experiment of the rst stage, we do the same,
but consider the 250 instances as a population that we evolve over 50
iterations. However, we disable topological mutations and thus, NEAT
only mutates the parameters of the ANNs. The third experiment of the
rst stage is similar to the second experiment but allows topological
mutations. While the rst experiment represents a baseline scenario, the
second and the third experiment are designed to show the impact of
evolving ANN parameters and topologies on the solution quality. We
conduct 10 observations for each experiment.
Our experiment planning in the second stage focuses on evaluating
different combinations of allocation and sequencing strategies. For this
reason, we conduct and compare 13 experiments, which we summarize
in Table 2. As for the experiments in the rst stage, we conduct 10 ob-
servations for each experiment. When using a dispatching rule for
sequencing, we only consider experiments where the sequencing hap-
pens before the allocation.
Our decision is based on preliminary experiments, in which we
investigated whether sequencing before and after the allocation of jobs
lead to different results. However, our observations indicate that this is
not the case.
The rst experiment of Table 2 corresponds to our experiment setup
in (Lang et al., 2020). However, it was necessary to repeat this experi-
ment, as we made two major changes:
Fig. 12. Process ow chart of competitive NEAT: The blue ow chart elements
represent NEATs search for an ANN representation that solves the allocation
problem, while the orange elements describe the search for an ANN that solves
the sequencing problem.
Table 2
Overview of the conducted experiments.
Experiment Sequencing Allocation Abbrevation
1 Presequencing with
ANN
minWorkload PRE-ANN & MIN-
WL
2 Presequencing with
FIFO
ANN Job-based FIFO & J-ANN
3 Presequencing with
LIFO
ANN Job-based LIFO & J-ANN
4 Presequencing with
EDD
ANN Job-based EDD & J-ANN
5 Presequencing with SPT ANN Job-based SPT & J-ANN
6 Presequencing with
FIFO
ANN Family-
based
FIFO & F-ANN
7 Presequencing with
LIFO
ANN Family-
based
LIFO & F-ANN
8 Presequencing with
EDD
ANN Family-
based
EDD & F-ANN
9 Presequencing with SPT ANN Family-
based
SPT & F-ANN
10 Presequencing with
ANN
ANN Job-based PRE-ANN & J-ANN
11 Postsequencing with
ANN
ANN Job-based POST-ANN & J-
ANN
12 Presequencing with
ANN
ANN Family-
based
PRE-ANN & F-ANN
13 Postsequencing with
ANN
ANN Family-
based
POST-ANN & F-
ANN
S. Lang et al.
Expert Systems With Applications 172 (2021) 114666
14
(1) We modied the allocation rule ‘minWorkload’. The intention of
the rule is to allocate a job to the SMD, where the job is expected
to complete the SMD process at the earliest possible time. How-
ever, in the previous experiment setup, the behaviour of min-
Workload was slightly different. Before allocating a job, the
heuristic checked rst, whether another job of the same family is
already waiting in a queue of one of the SMDs. If the heuristic
found an SMD for which the condition is true, it allocated the job
to this SMD and not the SMD providing the minimum workload.
We implemented this routine as a quick solution for fullling the
constraint that only one SMD at the same time can process a
specic job family. In our new implementation, however, this
routine no longer exists so that the minWorkload heuristic assigns
jobs to SMDs as originally intended. Removing this routine was
possible due to our second change.
(2) We implemented a new feature in our simulation model, which
ne-controls the change of family setup types on each SMD. Our
previous implementation contains no mechanism that controls
the allocation of family setup types to SMDs in situations where
more than one SMD demands the same setup type at the same
time. The prevention of setup violations was only possible with
the previous implementation of the minWorkload heuristic. In
the recent version of the simulation model, we introduce an
additional queuing system for controlling the allocation of family
setup types to SMDs. For each family setup type, there is a cor-
responding queue, which an SMD enters if another SMD is
currently deploying the requested family setup type. The model
serves requests for setups according to the FIFO strategy. SMDs
waiting for a setup are idle until the family setup type is available.
Every time an SMD needs to change the family setup type, it re-
leases the current family setup type and sends a notication to the
corresponding queue that the next SMD can be equipped with the
setup kit.
Table 3 shows the values we chose for the main hyper-parameters
inuencing NEAT’s randomized search strategy.
We use the same hyper-parameter settings for all experiments. In
general, NEAT allows adjusting more than 50 hyper-parameters. A
structured analysis of the hyper-parameters’ sensitivity and inuence to
each other goes beyond the scope of this work. As the list of experiments
in Table 2 indicates, there are already several possibilities and combi-
nations for using ANNs for the allocation and sequencing of jobs, which
are independent of NEAT’s hyper-parameters. Considering that little
research has been done on the application of the NEAT algorithm to
production scheduling problems, we focus in this article on identifying
the best combination of allocation and sequencing strategies under
similar conditions. A detailed investigation of the hyper-parameters of
NEAT, using the best identied combination of allocation and
sequencing strategies, is subject to future research.
Although the settings of many hyper-parameters were either intui-
tive or correspond to default settings, we want to briey explain the
intention of the selected values. In general, our settings are based on the
OpenAI lander example, which is part of the neat-python package
(McIntyre et al., 2019). In the OpenAI lander example, an agent must
control a lunar module in order to land it in the desired position at a
specic location on the moon. We decided for the hyper-parameter
settings of this example as reference because, compared to the other
examples, the number of input and output neurons for controlling the
lunar module (8 inputs and 4 outputs respectively) still has the closest
resemblance to our input/output structure. Furthermore, we assume
that the hyper-parameter settings of the OpenAI lander example were
chosen in order to explore sufciently complex hidden topologies ac-
cording to the size of the input and output layer.
However, we increase the size of the population from 150 to 250 in
order to explore more different topologies per generation, as some of our
strategies consider a signicant higher number of input neurons (see
Fig. 11). By this means, we evaluate 250 different ANNs on dataset 2 and
dataset 3 in each generation, which correspond to 500 simulation runs.
Although NEAT allows to consider different activation functions for
different neurons, we only consider a single activation function for all
neurons, thus to reduce the risk of overtting on the training data. We
select the sigmoid function as neuron activation function, because the
non-negative output value between 0 and 1 suits well for both, calcu-
lating the priority of a job in a sequence and indicating to which ma-
chine a job shall be allocated.
The remaining values of Table 3 correspond to the settings of the
OpenAI lander example. The compatibility disjoint coefcient for neu-
rons cn (see Eq. (2)) and synapses cs (see Eq. (3)) are both equal to 1.
Thus, neuronal and synaptic differences in the ANNs’ topologies are
equally weighted, when deciding about the speciation of the population.
The probabilities for adding new neurons and synapses (15%) are
slightly higher than the probabilities for removing existing ones (10%).
Therefore, ANN topologies tend to grow slowly and gradually.
The output of a neuron n is subject to its activation function actn,
input sn, bias bn and response rn. Considering that we use sigmoid as
activation function, the following equation describes the output of each
neuron:
actn(sn,bn,rn) = 1
1+e− (bn+rn*sn)(8)
Thus, the bias of a neuron is comparable to an intercept value that
shifts the activation function by adding a constant, while the response
value is a coefcient with a value around 1 that amplies/attenuates the
neuron input. Mutations of the neurons’ biases and responses as well as
of synapses’ weights underlie a similar set of hyper-parameters. The
mutation probabilities describe the likelihoods that the values of the
current biases, responses and weights will change. Mutation rates
describe the magnitude of a mutation and are subject to a zero-centred
normal distribution. The mutation rates of biases and weights are high
compared to the mutation rate of response values, as response values are
intended to maintain a value around 1, while biases and weights are the
Table 3
NEAT hyper-parameters.
Parameter Value
Population size 250
Compatibility threshold 3.0
Neuron parameters
Neuron compatibility disjoint coefcient 1.0
Probability of adding a new neuron 0.15
Probability of deleting an existing neuron 0.1
Neuron activation function sigmoid
Initial neuron bias Normal (
μ
=0.0;
σ
=1.0)
Mutation probability of neuron bias 0.8
Mutation rate of neuron bias Normal (
μ
=0.0;
σ
=0.4)
Replacing probability of neuron bias 0.02
Maximum neuron bias 30.0
Minimum neuron bias −30.0
Initial neuron response Normal (
μ
=1.0;
σ
=0.0)
Mutation probability of neuron response 0.1
Mutation rate of neuron response Normal (
μ
=0.0;
σ
=0.01)
Replacing probability of neuron bias 0.0
Maximum neuron response 30.0
Minimum neuron response −30.0
Synapse parameters
Synapse compatibility disjoint coefcient 1.0
Probability of adding a new neuron 0.15
Probability of deleting an existing neuron 0.1
Default synapse state enabled
Mutation probability of synapse state 0.01
Initial synapse weight Normal (
μ
=0.0;
σ
=1.0)
Mutation probability of synapse weight 0.8
Mutation rate of synapse weight Normal (
μ
=0.0;
σ
=0.4)
Replacing probability of synapse weight 0.02
Maximum synapse weight 30.0
Minimum synapse weight −30.0
S. Lang et al.
Expert Systems With Applications 172 (2021) 114666
15
main parameters to explore the solution space for a given network to-
pology. In case of a mutation, NEAT samples a value from the proba-
bility distribution of the mutation rate and adds it to the current value of
the corresponding parameter. However, biases, responses and weights
are bounded by a minimum and maximum value. In addition, there is a
small probability of 2% that biases and weights are replaced by reini-
tialized values. Finally, synapses can be also disabled. By this means, a
downstream neuron will not consume the output of an upstream neuron.
Initially, all synapses are enabled and might change their state with a
probability of 1%.
Due to the many stochastic inuences, the optimization is subject to
statistical uctuations. For this reason, we conduct 10 observations for
each experiment to smooth the effect of statistical uctuations. In
experiment 1 to 9, where only a single ANN is optimized, we run each
observation for about 500 generations. In experiment 10 to 13, where
two ANNs are optimized in a competitive manner, we set the maximum
number of generations for beating the global tness value to 50 gener-
ations. In total, we conduct 130 observations. We run our experiments
on a high-performance workstation with a 128-multicore-processor (2.0
GHz/core) and 256 GB RAM. The total computational time for all
second-stage experiments is about 35 hours.
While the experiment planning on the second stage aims to nd the
best combined allocation and sequencing strategy, the purpose of the
third-stage experiments is to maximize the solution quality of the best
strategy. Our intention is to gure out which solution quality is possible
if a high training time is negligible. For this purpose, we adapt the hyper-
Fig. 13. Analysing the inuence of NEAT’s mutation features on the solution quality based on (a) the average and (b) the best tness of the population as well as (c)
the resulting total tardiness for all datasets. For the rst and second experiment, we predened an ANN structure (d). However, when allowing topological mutations,
NEAT identies a reduced ANN structure with a reduced set of input features as better solution (e).
S. Lang et al.
Expert Systems With Applications 172 (2021) 114666
16
parameters to allow NEAT the exploration of larger solution spaces. In
particular, we double the mutation probabilities for neurons and syn-
apses as well as the value range and mutation rate for synapse weights,
neuron biases and responses. Furthermore, we set the population size
from 250 up to 1000. Beside the adaption of the hyper-parameters in
Table 3, we further increase the number of generations to beat the global
tness value from 50 to 100. We conduct 10 observations for each
experiment to address statistical uctuations.
6. Results and discussion
According to our design of experiments, we want to analyse rst
whether NEAT’s mutation features for the parameters and topologies of
ANNs have a positive impact on the solution quality. Fig. 13 shows for
all three experiments in the rst stage (a) the logarithmic average tness
of the population, (b) the logarithmic tness of the best genome, and (c)
the total tardiness for each dataset over 10 observations. Furthermore,
Fig. 13 illustrates the topology of (d) the manually designed ANN and (e)
the ANN created by NEAT. The results clearly show that the manually
designed ANN with randomly initialized parameters provides a worse
solution quality than both ANNs that NEAT evolved over several gen-
erations. Although the second experiment, in which only ANN param-
eters were evolved, provide the best average population tness, the third
experiment, in which also topologies were evolved, leads in all 10 ob-
servations to the best observed tness value. Looking on the third
experiment, it is noteworthy that all observations provide the same best
tness value. Furthermore, the plot of the total tardiness shows that
allowing both, the mutation of topologies and the mutation of param-
eters, lead to the best and most stable results. The reason why the second
experiment provides a better average tness may become clear, when
analysing the best found topology of the third experiment. While the
ANN of the rst and second experiment incorporate all input features of
Fig. 10(a), the best ANN of the third experiment only considers the AOI
processing times and due dates as input features. In all 10 observations,
NEAT only includes the same two features in the input layer. This gives
rise to the assumption that when using SMD processing times and job
families as input features of the ANN, the predictions of the sequence
priorities become worse. For the third experiment, where topological
evolution is activated, NEAT is expected to maintain genomes within the
population that also or even only consider SMD processing times and job
families as input features. Assuming that both input features impair the
prediction of sequence priorities, it would be reasonable for the average
tness of the third experiment to be worse than the average tness of the
second experiment.
In the following, we want to analyse the experiments of the second
stage. Fig. 14 shows the box plot for every dataset of each experiment.
There are signicant differences in terms of the achieved solution
quality in each experiment. For example, experiment 3 (LIFO & J-ANN)
and experiment 7 (LIFO & F-ANN) perform very poor and we can
conclude that pre-sequencing with LIFO in conjunction with an ANN-
based allocation strategy is not suitable for our objective. Experiments
2 and 6 show that the application of the same dispatching rule on
different datasets can lead to large differences in the quality of the so-
lutions. Both experiments pursue a FIFO dispatching strategy for pre-
sequencing. We conclude that FIFO is not suitable for dataset 4, but
can perform well for other problem instances. Although NEAT did not
know dataset 1 from the training, the best allocation ANNs of experi-
ment 2 and 6 achieve the best total tardiness for dataset 1. This means
that the composition of a job list has a major inuence on the success of
the application of the FIFO rule. Experiments 1 (PRE-ANN & MIN-ML),
experiment 8 (EDD & F-ANN) and experiment 12 (PRE-ANN & F-ANN)
perform best in terms of minimizing the total tardiness. The results
indicate that NEAT performs in general better on the allocation problem,
when it just needs to nd an ANN representation for assigning job
families to SMDs instead of single jobs.
Although we only consider the minimization of the total tardiness as
tness criterion, we were also interested in the makespan of the different
strategy combinations. Fig. 15 shows the makespan for each dataset of
experiment 1–13. Most of the investigated strategies complete the pro-
duction program after 20,000–30,000 min. Only experiment 3 (LIFO &
J-ANN) resulted in a signicant increase of the makespan with values up
to 38,692 min.
For our third-stage experiment planning, we consider experiment 12
(PRE-ANN & F-ANN) as most promising strategy to further improve the
solution quality, since it provides the best total tardiness along with
experiment 1 and 8. In contrast to experiment 1 and 8, however,
experiment 12 is a competitive NEAT experiment and thus has a larger
solutions space, because not a single ANN, but two ANNs must be
optimized. Table 4 shows the total tardiness and the makespan of the ten
observations. As expected, the adaptation of the hyper-parameters has
not only resulted in an increase of the computational time, but also
improved the solution quality. For the rst time, we were able to
construct production schedules with ANNs without any due date viola-
tions in all datasets. However, the uctuations of the results show that
we still need to conduct further parameter studies to increase the cer-
tainty that NEAT converges more often to comparable results.
Finally, we want to compare the performance of the best single NEAT
experiment (PRE-ANN & MIN-WL) and the best competitive NEAT
experiment (PRE-ANN & F-ANN) with other solution methods, which
were already investigated. In addition, we also compare our results with
Fig. 14. Total tardiness box plot for each dataset of experiment 1–13. The experiments are ordered by ascending total tardiness.
S. Lang et al.
Expert Systems With Applications 172 (2021) 114666
17
the dispatching rules EDD and SPT in combination with our minWork-
load heuristic to allocate jobs to SMDs. In the past, Aurich et al. (2016)
and Nahhas et al. (2016) already tested the metaheuristics ‘simulated
annealing’, ‘tabu search’ and ‘genetic algorithm’ to directly compute
solutions for the THFS without using any ANN. Furthermore, the authors
introduced an ‘Integrated Simulation-Based Optimization’ (ISBO) heu-
ristic, which is tailored to the THFS scheduling problem. Currently, the
company behind our case study applies a job family production strategy
that minimizes the number of major setups. The ISBO and the meta-
heuristics solve the allocation problem on the rst stage. In particular,
the metaheuristics solve the allocation problem on family level. By this
means, they assign each job family to a specic SMD before the simu-
lation starts Table 5 compares the different solution approaches in terms
of total tardiness and makespan. The comparison includes the results of
our previous implementation of the PRE-ANN & MIN-WL experiment,
which we presented in Lang et al. (2020).
We can derive several interesting observations from the table.
Compared to Lang et al. (2020), the modication of the strategy PRE-
ANN & MIN-WL lead to a reduced total tardiness for all datasets. On
the other hand, the makespan has signicantly increased on all datasets.
So far, it is not clear whether the increase of the makespan is due to the
modication of the PRE-ANN & MIN-WL strategy or subject to sto-
chasticity. In general, the PRE-ANN & MIN-WL strategy performs equal
to or better than the heuristic ISBO in terms of minimizing the total
tardiness. Furthermore, PRE-ANN & MIN-WL dominates the job family
production strategy and the dispatching rules EDD and SPT over all
datasets. Looking on the competitive NEAT strategy, it is striking that
the PRE-ANN & F-ANN strategy achieves a total tardiness of zero on all
datasets. Thus, the strategy is competitive to the metaheuristics. How-
ever, compared to the metaheuristics, PRE-ANN & F-ANN also achieves
a lower makespan on average across all datasets.
Next, we want to compare the different approaches in terms of
computational effort for determining a solution. In order to make a fair
comparison, we evaluate the different solution methods according to the
number of simulation runs that were necessary to achieve the results of
Table 5. A comparison based on computational time would not be
meaningful, since Aurich et al. (2016) and Nahhas et al. (2016) modeled
the problem in a different simulation framework. Furthermore, the au-
thors developed the simulation annealing algorithm and the tabu search
algorithm as external applications in different programming languages,
which means they require considerably more computation time due to
the time-consuming communication between the metaheuristics and the
simulation model. Table 6 compares the number of simulation runs for
ISBO, simulated annealing, tabu search, and genetic algorithm with the
number of simulation runs for training ANNs with NEAT and competi-
tive NEAT as well as for generating schedules with a trained ANN.
The ISBO only requires a single simulation run to compute the nal
solution, as the heuristic reschedules job families during the simulation,
whenever a job is at risk of being delayed. The number of simulation
runs for simulated annealing and tabu search are subject to their hyper-
parameter settings. For simulated annealing, the authors set the initial
temperature to 20, the number of iterations before changing the tem-
perature to 15, and the cooling rate to 0.2. The authors congured the
tabu search algorithm to run 30 iterations and to perform 105 moves in
each iteration. The number of simulation runs of the genetic algorithm is
subject to uctuations, as its termination depends on a convergence
Fig. 15. Makespan box plot for each dataset of experiment 1–13. The experiments are ordered by ascending makespan.
Table 4
Computational time, total tardiness and makespan of the ve observations of the second-stage experiment (all values in minutes): Observation 4 provides the best
results.
No. Dataset 1 (Testing) Dataset 2 (Training) Dataset 3 (Training) Dataset 4 (Testing)
T CMax T CMax T CMax T CMax
1 135,966 25,414 0 23,633 3 20,265 1745 20,032
2 0 21,368 0 24,286 4 21,617 12,974 18,928
3 438 18,247 74 23,977 0 22,174 10,194 20,531
4 0 17,863 0 24,912 0 22,405 0 18,975
5 5828 19,923 0 23,795 131 20,830 0 19,755
6 4932 19,477 57 26,654 105 20,378 6025 19,732
7 0 18,557 0 24,467 0 21,878 0 19,115
8 1171 21,437 4 24,699 0 20,987 1416 22,187
9 1871 22,011 41 24,518 0 21,480 0 19,741
10 469 20,247 66 24,806 0 21,663 45,441 24,742
S. Lang et al.
Expert Systems With Applications 172 (2021) 114666
18
function and a maximum number of generations. The convergence
function was set to the distance between the tness of the best genome
and the average tness of the population, while the maximum number of
generations was set to 100. The genetic algorithm terminates, whenever
the average tness of the population was greater than or equal to 97% of
the best observed tness or when the maximum number of generations
was reached. The population size was set to 50. By this means, the ge-
netic algorithm evaluates 50 solution candidates in each generation. For
determining the number of simulation runs for the strategy PRE-ANN &
MIN-WL, we dened a tness threshold of zero as termination criteria
and conducted 10 additional observations. Considering a population
size of 250 genomes, NEAT requires a minimum of 3 generations and a
maximum of 14 generations to nd an ANN that achieves a total tardi-
ness of zero on both training sets. To determine the number of simula-
tion runs for the PRE-ANN & F-ANN strategy, we use the hyper-
parameters of our third experiment stage with which we achieved the
results in table 5 (population size =1000; number of generations to beat
the global tness value =100). The extremely high number of simula-
tion runs clearly indicates a much lower efciency compared to the
other solution techniques. However, we must admit that our competitive
NEAT strategies probably still have great potentials for improvement.
Because of the poor starting conditions (due to the randomly generated
ANNs for pre-sequencing and family allocation) and the low rate of
convergence (due to the low initial tness to beat), the current imple-
mentation requires comparatively many generations until it identies a
promising area in the solution space.
As for the ISBO, we only require a single simulation run to construct
and evaluate a schedule, when deploying a trained ANN on the test
datasets. However, if we analyse the number of calculation steps that are
required for the scheduling of all jobs, we can show that the ISBO and
the ANNs differ in terms of time complexity. For the ISBO, the number of
calculation steps depends on the number of rescheduling events. Every
time a job completes the SMD stage, the ISBO checks for the job family
that owns the job with the lowest due date and that is not currently being
produced, whether the due date of this job is at the risk of being violated.
For this purpose, the ISBO sorts all families in ascending order
depending on the unscheduled jobs that provide the lowest due date. If
the condition is true, the ISBO reschedules the complete family to the
SMD that has become available.
In order to approximate the worst-case complexity of the ISBO, let us
assume that we have J jobs, where each job has a unique family. In this
case, the number of different job families corresponds to the number of
jobs J. For sorting the families, we assume a worst case complexity of
OJ2. However, we are dealing with a gradually decreasing number of
jobs, because rescheduling events occur whenever a job is completed.
Thus, the number of jobs to sort j varies between 1 and J. Therefore, the
worst case complexity of the ISBO is O(J
j=1j2).
When deploying an ANN for scheduling decisions, the number of
calculation steps is subject to the number of jobs to be scheduled and the
number of matrix multiplications for forward propagating input features
through the network. However, the structure of an ANN and thus the
Table 5
Performance of the competitive NEAT implementation in comparison to other
solution approaches (Aurich et al., 2016; Nahhas et al., 2016) and the NEAT
implementation of Lang et al. (2020).
Total tardiness
(minutes)
Makespan
(minutes)
Dataset 1
Job Family Production Strategy 198,783 23,513
EDD & MinWorkload 6646 24,491
SPT & MinWorkload 120,794 23,445
ISBO 148 19,354
Simulated Annealing 0 21,930
Tabu Search 0 19,669
Genetic Algorithm 0 17,786
NEAT PRE-ANN & MIN-WL (Lang
et al., 2020)
124 17,768
Improved NEAT PRE-ANN & MIN-WL 0 20,663
Competitive NEAT PRE-ANN & F-ANN 0 17,863
Dataset 2
Job Family Production Strategy 271,700 25,447
EDD & MinWorkload 5513 26,813
SPT & MinWorkload 143,511 27,993
ISBO 148 19,354
Simulated Annealing 0 21,930
Tabu Search 0 19,669
Genetic Algorithm 0 22,458
NEAT PRE-ANN & MIN-WL (Lang et al
2020)
303 20,916
Improved NEAT PRE-ANN & MIN-WL 0 22,938
Competitive NEAT PRE-ANN & F-ANN 0 24,912
Dataset 3
Job Family Production Strategy 31,372 23,626
EDD & MinWorkload 11,832 26,653
SPT & MinWorkload 188,743 27,302
ISBO 0 21,819
Simulated Annealing 0 23,108
Tabu Search 0 25,142
Genetic Algorithm 28 22,671
NEAT PRE-ANN & MIN-WL (Lang et al
2020)
0 20,584
Improved NEAT PRE-ANN & MIN-WL 0 24,272
Competitive NEAT PRE-ANN & F-ANN 0 22,405
Dataset 4
Job Family Production Strategy 257,376 23,539
EDD & MinWorkload 28 23,196
SPT & MinWorkload 111,470 22,937
ISBO 536 19,979
Simulated Annealing 0 23,059
Tabu Search 0 22,507
Genetic Algorithm 0 22,569
NEAT PRE-ANN & MIN-WL (Lang et al
2020)
0 18,771
Improved NEAT PRE-ANN & MIN-WL 0 20,886
Competitive NEAT PRE-ANN & F-ANN 0 18,975
Average over all datasets
Job Family Production Strategy 162,510 24,031
EDD & MinWorkload 6005 25,288
SPT & MinWorkload 141,130 25,419
ISBO 331 19,990
Simulated Annealing 0 22,243
Tabu Search 0 22,439
Genetic Algorithm 7 21,371
NEAT (Lang et al 2020) 107 19,510
Improved NEAT PRE-ANN & MIN-WL 13 22,190
Competitive NEAT PRE-ANN & F-ANN 0 21,039
Table 6
Number of simulation runs required for NEAT, competitive NEAT, and the
resulting ANNs during inference in comparison to the solution techniques of
Aurich et al. (2016), Nahhas et al. (2016).
Solution technique Number of simulation runs
ISBO 1
Simulated Annealing 1500
Tabu Search 3150
Genetic Algorithm 6000–8000
NEAT PRE-ANN & MIN-WL 750–3500 (mean 1725)
Competitive NEAT PRE-ANN & F-ANN 220,000–360,000 (mean
310,000)
Deployment of ANNs trained by any NEAT
strategy
1
Solution technique Number of simulation runs
ISBO 1
Simulated Annealing 1500
Tabu Search 3150
Genetic Algorithm 6000–8000
NEAT PRE-ANN & MIN-WL 750–3500 (mean 1725)
Competitive NEAT PRE-ANN & F-ANN 220,000–360,000 (mean
310,000)
Deployment of ANNs trained by any NEAT
strategy
1
S. Lang et al.
Expert Systems With Applications 172 (2021) 114666
19
number of matrix multiplications are constant for a trained ANN and
therefore negligible for the determining the complexity. For this reason,
the complexity of each deployed ANN can be reduced to O(n)and is
therefore linear. We can further quantify the difference in complexity
using the computational time for a simulation experiment. Thus, an ANN
created and parameterized by NEAT requires about 0.1 to 0.4 s,
depending on the dataset, to generate and evaluate a complete schedule.
The ISBO, on the other hand, requires 8 to 18 s to perform the same task.
However, the time difference is not only caused by the different
complexity of ANN and ISBO, but also due to the use of different
frameworks for implementing the simulation models.
7. Summary and conclusions
In this article, we present a novel approach to solve a two-stage
hybrid ow shop scheduling problem with family setup times: Neuro-
Evolution of Augmenting Topologies (NEAT) in conjunction with
discrete-event simulation (DES) to generate articial neural networks
(ANN), which are able to create job schedules. We further introduce ve
different implementation strategies, which differ in terms of how the
ANN solves the problem (as an allocation problem, as a sequencing
problem or as a joint allocation-sequencing problem) and how the ANN
interacts with the DES model (integrated or separated). In 14 experi-
ments, we evaluate and compare the performance and computational
efciency of different combinations of ANN-based allocation and
sequencing strategies. Furthermore, we compare our best performing
strategy with other solution approaches like simulated annealing and
tabu search.
NEAT proves that the application of ANNs for ow shop scheduling
can compete with state-of-the-art metaheuristics in terms of solution
quality. Once trained, the ANNs are able to determine production
schedules in real-time, as they compute solutions based on the charac-
teristics of the jobs to be scheduled and the current state of the system
instead of randomly searching and trying out solutions. The runtime for
computing a solution with an ANN grows linearly with the problem
complexity, i.e. the number of forward propagations of an ANN to
generate a complete production schedule corresponds to the number of
jobs to be scheduled.
In the following, we want to review our initial hypotheses from
section 1, which we used to justify our motivation for investigating
NEAT. Our rst hypothesis (i), that ANNs determine scheduling de-
cisions in real-time during inference, turns out to be true. As stated in the
last section, ANNs that are created and parameterized by NEAT require
only 0.1 to 0.4 s to generate and evaluate a complete schedule for 143 to
175 jobs. Thus, our approach competes with dispatching rules, such as
EDD and SPT, and outperforms the ISBO as well as the compared met-
aheuristics in terms of computational efciency. In our second hypoth-
esis (ii), we claimed that even the runtime of NEAT for converging to a
suitable ANN might be less than the runtime of a metaheuristic to
converge to a solution for a given problem instance. We consider this
hypothesis to be partly true. In fact, the runtime of NEAT strongly de-
pends on the required ANN topology for computing sufcient allocation
and sequencing decisions. On the one hand, our strategy PRE-ANN &
MIN-WL requires only 750–3500 simulation experiments to identify a
suitable ANN for the sequencing task. Considering the best observation,
NEAT requires far fewer simulation experiments than each of the
compared metaheuristics. In any case, taking all observations into ac-
count, NEAT still requires far fewer simulation runs compared to the
genetic algorithm for computing a specic solution for a problem
instance. On the other hand, our best competitive NEAT strategy PRE-
ANN & F-ANN provides the worst computational efciency, as it
currently requires several hundreds of thousands of simulation runs to
synchronize the topology and parameters of two ANNs for the allocation
and sequencing of jobs. Our third hypothesis (iii) was that ANNs
generate better solutions as conventional dispatching rules (e.g. EDD,
SPT), because ANNs incorporate information from their environment to
make decisions. Based on the results in table 5, this hypothesis can be
considered to be true, since the three favoured NEAT strategies clearly
dominate the family production strategy of the company as well as the
dispatching rules EDD and SPT in terms of both, total tardiness and
makespan. The fourth hypothesis (iv), claiming that the effort required
to implement NEAT for a given scheduling problem is lower than for
many problem-tailored heuristics and DRL methods, is difcult to verify.
In our very personal opinion, the effort for implementing a specic
NEAT strategy is low, when the developer has already implemented a
single and/or a competitive NEAT strategy as reference. In terms of
implementation effort, we could not nd much difference between
NEAT and conventional metaheuristics. Our fth hypothesis (v) was
that, compared to other machine learning methods, NEAT reduces the
human effort for engineering, as NEAT automates the design process for
ANNs. According to our current knowledge, the last hypothesis seems be
true, since our experiments in the rst stage have shown that NEAT nds
a better ANN topology compared to the application of a static design
rule. Going beyond our hypotheses, we were very surprised by NEAT’s
solution quality. In particular, we did not expect in advance that some of
the NEAT strategies will be able to compute solutions on both, the
training and test sets, that are competitive with the ISBO and the met-
aheuristics, while outperforming the same in terms of computational
efciency.
We did not investigate how trained ANNs will perform on a larger
number of problem instances that show signicant differences in their
statistical properties. It is necessary to conduct further experiments to
answer this question. However, our experiments revealed that some
solution strategies provide a better solution quality as the compared
metaheuristics and require at the same time a much lower runtime for
training compared to the runtime of the compared metaheuristics for
solving a single problem instance. Thus, even if future experiments
should show that learned ANNs cannot generalize to a larger number of
problem instances, NEAT can be still a very efcient solution strategy to
quickly approximate high-quality solution strategies for specic prob-
lem instances.
Declaration of Competing Interest
The authors declare that they have no known competing nancial
interests or personal relationships that could have appeared to inuence
the work reported in this paper.
Acknowledgements
The authors acknowledge the nancial support by the Federal Min-
istry of Education and Research of Germany (BMBF) within the frame-
work of the SENECA project (grant number: 01IS20019A).
References
Aurich, P., Nahhas, A., Reggelin, T. & Tolujew, J. (2016). Simulation-based optimization
for solving a hybrid ow shop scheduling problem. In T. M. Roeder, P. I. Frazier, R.
Szechtman, & E. Zhou (Eds.), Proceedings of the 2016 Winter Simulation Conference
(WSC’16) (pp. 2809–2819). IEEE. https://doi.org/10.1109/WSC.2016.7822317.
Blum, C., & Roli, A. (2003). Metaheuristics in combinatorial optimization: Overview and
conceptual comparison. ACM Computing Surveys, 35(3), 268–308. https://doi.org/
10.1145/937503.937505
Burke, L. I., & Ignizio, J. P. (1992). Neural networks and operations research: An
overview. Computers & Operations Research, 19(3–4), 179–189. https://doi.org/
10.1016/0305-0548(92)90043-5
Çalis
¸, B., & Bulkan, S. (2015). A research survey: Review of AI solution strategies of job
shop scheduling problem. Journal of Intelligent Manufacturing, 26(5), 961–973.
https://doi.org/10.1007/s10845-013-0837-8
Chandra, J., & Talavage, J. (1991). Intelligent dispatching for exible manufacturing.
International Journal of Production Research, 29(11), 2259–2278. https://doi.org/
10.1080/00207549108948082
Chiou, C.-W., Chen, W.-M., Liu, C.-M., & Wu, M.-C. (2012). A genetic algorithm for
scheduling dual ow shops. Expert Systems with Applications, 39(1), 1306–1314.
https://doi.org/10.1016/j.eswa.2011.08.008
S. Lang et al.
Expert Systems With Applications 172 (2021) 114666
20
Csaszar, P., Tirpak, T. M., & Nelson, P. C. (2000). Optimization of a high-speed
placement machine using tabu search algorithms. Annals of Operations Research, 96
(1/4), 125–147. https://doi.org/10.1023/A:1018911821102
Dahl, O.-J., & Nygaard, K. (1966). SIMULA: An ALGOL-based simulation language.
Communications of the ACM, 9(9), 671–678. https://doi.org/10.1145/
365813.365819
Emmons, H., & Vairaktarakis, G. (2013). Flow shop scheduling: Theoretical results,
algorithms, and applications. International series in operations research &
management science: Vol. 182. Springer.
Fernandez-Viagas, V., Costa, A., & Framinan, J. M. (2020). Hybrid ow shop with
multiple servers: A computational evaluation and efcient divide-and-conquer
heuristics. Expert Systems with Applications, 153, 113462. https://doi.org/10.1016/j.
eswa.2020.113462
Gomez, F. J., Burger, D., & Miikkulainen, R. (2001). A neuro-evolution method for
dynamic resource allocation on a chip multiprocessor. In Ijcnn’01: Proceedings:
International Joint Conference on Neural Networks: Washington, D.C., July 15–19,
2001 (pp. 2355–2360). IEEE. https://doi.org/10.1109/IJCNN.2001.938732.
Gong, Z., Zhong, P., & Hu, W. (2019). Diversity in machine learning. IEEE Access, 7,
64323–64350. https://doi.org/10.1109/ACCESS.2019.2917620
Guinet, A. G. P., & Solomon, M. M. (1996). Scheduling hybrid owshops to minimize
maximum tardiness or maximum completion time. International Journal of Production
Research, 34(6), 1643–1654. https://doi.org/10.1080/00207549608904988
Guinet, A., Solomon, M. M., Kedia, P. K., & Dussauchoy, A. (1996). A computational
study of heuristics for two-stage exible owshops. International Journal of
Production Research, 34(5), 1399–1415. https://doi.org/10.1080/
00207549608904972
Haykin, S. (1999). Neural networks: A comprehensive foundation (2nd Ed.). Pearson
Education.
Hecht-Nielsen, R. (1987). Kolmogorov’s mapping neural network existence theorem. In
S. Grossberg, T. Kohonen, & B. Kosko (Eds.), Proceedings of the IEEE rst annual
international conference on neural networks. IEEE.
Hoogeveen, J., Lenstra, J., & Veltman, B. (1992). Minimizing makespan in a multiprocessor
owshop is strongly NP-hard.: Working paper. Netherlands: Eindhoven University of
Technology.
Irpan, A. (2018). Deep Reinforcement Learning Doesn’t Work Yet. https://www.
alexirpan.com/2018/02/14/rl-hard.html.
Klug, F. (2017). Das Perlenkettenprinzip der stabilen Auftragsfolge in der
Automobillogistik. In I. G¨
opfert, D. Braun, & M. Schulz (Eds.), Automobillogistik:
Stand und Zukunftstrends (3rd Ed., pp. 137–160). Springer Gabler.
Lang, S., Reggelin, T., Behrendt, F. & Nahhas, A. (2020). Evolving Neural Networks to
Solve a Two-Stage Hybrid Flow Shop Scheduling Problem with Family Setup Times.
In Proceedings of the 53rd Hawaii international conference on system sciences
(HICSS 2020). Symposium conducted at the meeting of University of Hawaii at
Manoa, Wailea, HI. http://scholarspace.manoa.hawaii.edu/bitstream/10125/
63899/1/0129.pdf.
LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444.
https://doi.org/10.1038/nature14539
Lin, C.-C., Deng, D.-J., Chih, Y.-L., & Chiu, H.-T. (2019). Smart Manufacturing Scheduling
With Edge Computing Using Multiclass Deep Q network. IEEE Transactions on
Industrial Informatics, 15(7), 4276–4284. https://doi.org/10.1109/TII.2019.2908210
Looi, C.-K. (1992). Neural network methods in combinatorial optimization. Computers &
Operations Research, 19(3–4), 191–208. https://doi.org/10.1016/0305-0548(92)
90044-6
Luo, S. (2020). Dynamic scheduling for exible job shop with new job insertions by deep
reinforcement learning. Applied Soft Computing, 91, 106208. https://doi.org/
10.1016/j.asoc.2020.106208
Mao, X., ter Mors, A., Roos, N. & Witteveen, C. (2007). Using neuro-evolution in aircraft
deicing scheduling. In K. Tuyls, S. de Jong, M. Ponsen & K. Verbeeck (Eds.), ALAMAS
2007: Adaptive and learning agents and multi-agent systems (pp. 138–145). https://
dke.maastrichtuniversity.nl/nico.roos/wp-content/uploads/2016/01/
MMRW07ALAMAS.pdf.
McIntyre, A., Kallada, M., Miguel, C. G. & Feher da Silva, C. (2019). Neat-python.
https://github.com/CodeReclaimers/neat-python.
Nahhas, A., Aurich, P., Reggelin, T. & Tolujevs, J. (2016). Heuristic and metaheuristic
simulation-based optimization for solving a hybrid ow shop scheduling problem. In
A. G. Bruzzone, F. de Felice, C. Frydman, F. Longo, M. Massei & A. Solis (Eds.), The
15th international conference on modeling and applied simulation (MAS 2016) (pp.
95–103). Dime Universit`
a di Genova.
Pan, Q.-K., Gao, L., Wang, L., Liang, J., & Li, X.-Y. (2019). Effective heuristics and
metaheuristics to minimize total owtime for the distributed permutation owshop
problem. Expert Systems with Applications, 124, 309–324. https://doi.org/10.1016/j.
eswa.2019.01.062
Pinedo, M. L. (Ed.). (2016). Scheduling. Cham: Springer International Publishing.
Potvin, J. -Y. & Smith, K. A. (2003). Articial neural networks for combinatorial
optimization. In F. Glover & G. A. Kochenberger (Eds.), International series in
operations research & management science: Vol. 57. Handbook of metaheuristics
(pp. 429–455). Kluwer Academic Publishers.
Reeves, C. R. (2010). Genetic Algorithms. In M. Gendreau & J.-Y. Potvin (Eds.),
International series in operations research & management science: Vol. 146.
Handbook of Metaheuristics (2nd Ed., pp. 109–140). Springer Science +Business
Media.
Ruiz, R., & V´
azquez-Rodríguez, J. A. (2010). The hybrid ow shop scheduling problem.
European Journal of Operational Research, 205(1), 1–18. https://doi.org/10.1016/j.
ejor.2009.09.024
Shiue, Y.-R., Lee, K.-C., & Su, C.-T. (2018). Real-time scheduling for a smart factory using
a reinforcement learning approach. Computers & Industrial Engineering, 125,
604–614. https://doi.org/10.1016/j.cie.2018.03.039
Smith, K. A. (1999). Neural networks for combinatorial optimization: A review of more
than a decade of research. INFORMS Journal on Computing, 11(1), 15–34. https://doi.
org/10.1287/ijoc.11.1.15
Stanley, K. O. & Miikkulainen, R. (2002a). Efcient evolution of neural network
topologies. In Proceedings of the 2002 Congress on Evolutionary Computation,
CEC’02: May 12–17, 2002, Hilton Hawaiian Village Hotel, Honolulu, Hawaii (pp.
1757–1762). IEEE Service Center. https://doi.org/10.1109/CEC.2002.1004508.
Stanley, K. O. (2004). Efcient Evolution of Neural Networksthrough Complexication
[Dissertation]. Austin, TX: The University of Texas.
Stanley, K. O., & Miikkulainen, R. (2002b). Evolving neural networks through
augmenting topologies. Evolutionary Computation, 10(2), 99–127. https://doi.org/
10.1162/106365602320169811
Stathakis, D. (2009). How many hidden layers and nodes? International Journal of Remote
Sensing, 30(8), 2133–2147. https://doi.org/10.1080/01431160802549278
Stricker, N., Kuhnle, A., Sturm, R., & Friess, S. (2018). Reinforcement learning for
adaptive order dispatching in the semiconductor industry. CIRP Annals, 67(1),
511–514. https://doi.org/10.1016/j.cirp.2018.04.041
Syed, M. N., & Pardalos, P. M. (2013). In Handbook of combinatorial optimization (pp.
2027–2093). New York, NY: Springer New York. https://doi.org/10.1007/978-1-
4419-7997-1_65.
T’kindt, V. & Billaut, J. -C. (2006). Multicriteria scheduling: Theory, models and
algorithms (2nd Ed.). Springer.
van der Ham, R. (2018). Salabim: Discrete event simulation and animation in python.
Journal of Open Source Software, 3(27), 767–768. https://doi.org/10.21105/
joss.00767
van Wassenhove, L. N., Potts, C. N., Sevast’janov, S. V., Strusevich, V. A. & Zwaneveld, C.
M. (1992). The two-stage assembly scheduling problem: complexity and
approximation: INSEAD Working paper 92/53/TM.
Wang, H., Sarker, B. R., Li, J., & Li, J. (2020). Adaptive scheduling for assembly job shop
with uncertain assembly times based on dual Q-learning. International Journal of
Production Research, 1–17. https://doi.org/10.1080/00207543.2020.1794075
Waschneck, B., Reichstaller, A., Belzner, L., Altenmüller, T., Bauernhansl, T., Knapp, A.,
& Kyek, A. (2018). Optimization of global production scheduling with deep
reinforcement learning. Procedia CIRP, 72, 1264–1269. https://doi.org/10.1016/j.
procir.2018.03.212
Whiteson, S. (2005). Improving reinforcement learning function approximators via
neuroevolution. In M. Pechoucek, D. Steiner, & S. Thompson (Eds.), Proceedings of
the fourth international joint conference on Autonomous agents and multiagent
systems (pp. 1386–1387). ACM. https://doi.org/10.1145/1082473.1082794.
Wilson, G. V., & Pawley, G. S. (1988). On the stability of the travelling salesman problem
algorithm of Hopeld and tank. Biological Cybernetics, 58(1), 63–70. https://doi.org/
10.1007/BF00363956
Zang, Z., Wang, W., Song, Y., Lu, L., Li, W., Wang, Y., & Zhao, Y. (2019). Hybrid deep
neural network scheduler for job-shop problem based on convolution two-
dimensional transformation. Computational Intelligence and Neuroscience, 2019, 1–19.
https://doi.org/10.1155/2019/7172842
Zhao, F., Liu, Y., Zhang, Y., Ma, W., & Zhang, C. (2017). A hybrid harmony search
algorithm with efcient job sequence scheme and variable neighborhood search for
the permutation ow shop scheduling problems. Engineering Applications of Articial
Intelligence, 65, 178–199. https://doi.org/10.1016/j.engappai.2017.07.023
Zhao, F., Qin, S., Yang, G., Ma, W., Zhang, C., & Song, H. (2019). A factorial based
particle swarm optimization with a population adaptation mechanism for the no-
wait ow shop scheduling problem with the makespan objective. Expert Systems with
Applications, 126, 41–53. https://doi.org/10.1016/j.eswa.2019.01.084
Zhao, F., Zhang, L., Zhang, Y., Ma, W., Zhang, C., & Song, H. (2020). A hybrid discrete
water wave optimization algorithm for the no-idle owshop scheduling problem
with total tardiness criterion. Expert Systems with Applications, 146, 113166. https://
doi.org/10.1016/j.eswa.2019.113166
S. Lang et al.