A NeuroGenetic Approach for Multiprocessor Scheduling

Abstract

In this study we combine two metaheuristic search techniques, the Augmented Neural Networks and Genetic Algorithms approach to create a hybrid metaheuristic called the NeuroGenetic approach. We apply this hybrid approach to a multiprocessor scheduling problem, the job-shop scheduling problem to test if the hybridization helps improve the solution. The hybridization of AugNN and GA is achieved by interleaving the two approaches. Since the GA approach is better at diversification or global search whereas AugNN is better at intensification or local search, the combination provides improved solutions than either GA or AugNN search with the same number of iterations. Computational results showed that such hybridization provided improvements in the solutions, than if each technique was used alone. Given the encouraging results, more research needs to be done in this area. Such hybrid techniques can be applied to other scheduling problems and also on the job shop scheduling problem by applying other GA approaches that have performed well in the literature. The AugNN technique can also be hybridized with other non GA techniques such Tabu Search and Simulated Annealing approaches, which tend to give good results for the job-shop scheduling problem.

Full text

Page 1

7
A NeuroGenetic Approach for Multiprocessor
Scheduling
Anurag Agarwal
Department of Information Systems and Operations Management, Warrington College of
Business Administration, University of Florida
USA
1. Abstract
This chapter presents a NeuroGenetic approach for solving a family of multiprocessor
scheduling problems. We address primarily the Job-Shop scheduling problem, one of the
hardest of the various scheduling problems. We propose a new approach, the NeuroGenetic
approach, which is a hybrid metaheuristic that combines augmented-neural-networks
(AugNN) and genetic algorithms-based search methods. The AugNN approach is a non-
deterministic iterative local-search method which combines the benefits of a heuristic search
and iterative neural-network search. Genetic algorithms based search is particularly good at
global search. An interleaved approach between AugNN and GA combines the advantages
of local search and global search, thus providing improved solutions compared to AugNN
or GA search alone. We discuss the encoding and decoding schemes for switching between
GA and AugNN approaches to allow interleaving. The purpose of this study is to
empirically test the extent of improvement obtained by using the interleaved hybrid
approach instead of applied using a single approach on the job-shop scheduling problem.
We also describe the AugNN formulation and a Genetic Algorithm approach for the Job-
Shop problem. We present the results of AugNN, GA and the NeuroGentic approach on
some benchmark job-shop scheduling problems.
2. Introduction
Multiprocessor scheduling problems occur whenever manufacturing or computing
operations are to be scheduled on multiple machines, processors or resources. A variety of
such scheduling problems are discussed in the literature. The most general scheduling
problem is the resource-constrained project-scheduling problem; this problem has received
a lot of attention in the literature Herroelen et al. (1998), Kolisch (1996). The open-shop,
flow-shop, job-shop and task scheduling problems can be considered special cases of the
resource-constrained project-scheduling problem. While smaller instances of the various
types of scheduling problems can be solved to optimality in reasonable computing time
using exact solution methods such as branch and bound, most real-world problems are
unsolvable in reasonable time using exact methods due to the combinatorial explosion of the
feasible solution space. For this reason, heuristics and metaheuristics are frequently
employed to obtain satisfactory solutions to these problems in reasonable time. In this
Source: Multiprocessor Scheduling: Theory and Applications, Book edited by Eugene Levner,
ISBN 978-3-902613-02-8, pp.436, December 2007, Itech Education and Publishing, Vienna, Austria
Open Access Database www.i-techonline.com

Page 2

Multiprocessor Scheduling: Theory and Applications
122
paper, we propose a new hybrid metaheuristic approach called the NeuroGenetic approach
for solving one family of multiprocessor scheduling problems – the job-shop scheduling
problem. The NeuroGenetic approach is a hybrid of the Augmented Neural Networks
(AugNN) approach and the Genetic Algorithms (GA) approach. The AugNN approach
provides a mechanism for local search, while the GA approach provides a mechanism for
global search. An interleaving of the two approaches helps guide the search to better
solutions.
In this chapter, we focus on the job-shop scheduling problem (JSSP). In JSSP, there are n
jobs, each having m operations and each operation requires a different machine, so there are
m machines. For each job, the order in which operations require machines is fixed and is
independent of the order of machine requirement on other jobs. So in a 2x3 job shop-
problem, for example, say job 1 requires machines in the order 2, 3 and 1, job 2 may require
the machines in a different order, say 1, 3 and 2 or 1, 2 and 3 or 3, 1 and 2 or it could be the
same i.e., 2,3 and 1. In a flow-shop problem (FSP), which is special case of the job-shop
problem, the order in which machines are needed for each operation is assumed to be the
same for each job. An FSP is therefore, a special case of the JSSP. In both JSSP and the FSP,
there is only one machine of each type, and a machine can only process one operation at a
time. The problem is to find a precedence and resource feasible schedule for each operation
for each job with the shorted possible makespan. In general, preemption is not allowed, i.e.
operations must proceed to completion once started.
A job-shop scheduling problem can be considered a special case of the resource-constrained
project scheduling problem (RCPSP). In the RCPSP, a PERT chart of activities can be drawn
just like for a JSSP. The RCPSP is more general because it allows multiple successors for an
operation, whereas a JSSP allows only one successor. Also, while in RCPSP an activity may
require multiple units of multiple resource types, in JSSP activities require only one unit of
one resource type. Task scheduling problem is also a special case of RCPSP, in that only one
type of resource is required for all activities. In task scheduling there can be multiple
successors for an operation, like in an RCPSP.
In the next section, we review the literature primarily on JSSP. In the following section, the
AugNN formulation for a JSSP is described. Section 4 outlines the GA approach for solving
the JSSP. Section 5 describes the Neurogenetic approach and discusses how the AugNN and
GA approaches can be interleaved. Section 6 provides the computational results of several
benchmark problems in the literature. Section 7 summarizes the paper and offers
suggestions for future research. This study contributes to the literature of job-shop
scheduling by proposing for the first time an AugNN architecture and formulation for the
JSSP and also proposing a hybrid of AugNN and GA approach.
3 Literature Review
The JSSP has been recognized as an academic problem for over four decades now. Giffler
and Thompson (1960) and Fisher and Thompson (1963) were amongst the first to address
this problem. Exact solution methods have been proposed by Carlier and Pinson (1989),
Applegate and Cook (1991) and Brucker et al. (1994). A number of heuristic search methods
have also been proposed, for example, Adams et al. (1988) and Applegate and Cook (1991).
A variety of metaheuristic approaches have also been applied to the JSSP, such as Neural
Networks (Sabuncuoglu and Gurgun, 1996), Beam Search (Sabuncuoglu and Bayiz, 1999),
Simulated Annealing (Steinhofel et al. 1999), Tabu Search (Barnes and Chambers, 1995;

Page 3

A NeuroGenetic Approach for Multiprocessor Scheduling
123
Nowicki and Smutnicki, 1996; Pezzella and Merelli, 2000; Zhang et al. 2008), Genetic
Algorithms (Falkenauer and Bouffoix, 1991; Storer et al, 1995; Aarts et al., 1994; Bean, 1994;
Croce et al., 1995), Evolutionary Algorithms (Mesghouni and Hammadi, 2004), Variable
Neighborhood Search (Sevkli and Aydin, 2007), Global Equilibrium Search technique
(Pardalos and Shylo, 2006). Jain and Meeran (1999) provide a good survey of techniques
used for the JSSP. For the RCPSP, a number of heuristic and metaheuristic approaches have
been proposed in the literature. For a good review of the heuristics, see Herroelen et al.,
1998.
4. Augmented Neural Network Formulation
The AugNN approach was first introduced by Agarwal et al. (2003). They applied the
AugNN approach to the task scheduling problem and offered an improved approach for
using AugNN approach in Agarwal et al. (2006). In this approach, a given scheduling
problem is converted into a neural network, with input layer, hidden layers and output
layer of neurons or processing elements (PEs). The connections between the PEs of these
layers are assigned weights. Input, activation and output functions are then designed for
each node in such a way that a single-pass or iteration from the input to the output layer
produces a feasible solution using a heuristic. An iteration, or a pass, consists of calculating
all the functions of the network from the input up to the output layer. A search strategy is
then applied to modify the weights on the connections such that subsequent iterations
produce neighboring solutions in the search space.
We now describe, with the help of an example, how to convert a given JSSP problem into a
neural network. We will assume a simple 3x2 JSSP instance of Figure 1 for this purpose.
Job ??
1
1 (4)
2 (3)
2
2 (5)
1 (4)
3
1 (3)
2 (6)
Machine (Proc Time)
Figure 1. An Example 3x2 Job Shop Scheduling Problem
In this 3x2 problem, there are 3 jobs, each with 2 operations, for a total of 6 operations (O11,
O12, O21, O22, O31 and O32). Job 1 requires 4 units of time (ut) on machine 1 (O11) followed by
3 ut on machine 2 (O12). Job 2 requires 5 ut on machine 2 (O21) followed by 4 ut on machine
1 (O22). Job 3 requires 3 ut on machine 1 (O31) followed by 6 ut on machine 2 (O32). The
problem is how to schedule these six operations such that the makespan is minimized.
Figure 2 shows a neural network for this problem.
There are two operation layers, corresponding to the two operations for each job. Each
operation layer has three nodes corresponding to each job. Note that for a more general nxm
case, there will be m operation layers, each with n nodes. Following each operation layer is a
machine layer with 3 nodes each. Each of the three operation nodes is connected to a machine
which is determined by the given problem. So, for example, given our 3x2 problem of Figure
1, O11is connected to machine 1 and O12 is connected to machine 2; O21 is connected to machine
2 and O22 is connected to machine 1, and so on. For a more general n x m case, there will be n
machine nodes in each of the m machine layers. An input layer is designed to provide a signal
to the first operation layer to start the scheduling process. There is also an output layer with
one PE labeled OF for “final operation”, which is a dummy operation with zero processing
time and no resource requirement. The operation and the machine layers can be regarded as

Page 4

Multiprocessor Scheduling: Theory and Applications
124
hidden layers of a neural network. Connections between operation nodes and machine nodes
are characterized by weights. These weights are all the same for the first iteration, but are
modified for subsequent iterations. There are also connections between machine nodes and
subsequent operation nodes, which are not characterized by any weights. These connections
serve to pass signals from one layer to the next to trigger some functions.
Figure 2: AugNN Architecture to solve a 3x2 Job Shop Scheduling Problem
The output of the operation nodes (OO) becomes input to the machine nodes. There are
three types of outputs from each machine node. One output (OMF) goes to the next
operation node (or to the final node). This signals the end of an operation on that machine.
The second type of output (OMM) goes to the machine node of its own type. For example,
machine 1 sends an output to all other machine 1 nodes. Similarly, machine 2 sends an
O11
O21
O31
M1,11
O12
O22
O32
I
OF
Operation Layer
Operation Layer
Machine Layer
Machine Layer
Output Layer
Input Layer
W12
W22
W32
W11
W21
W31
OI
OI
OI
OO1
OMF1,11
OMF2,12F
OMM
OMR1,1
M2,21
M1,31
M2,12
M1,22
M2,32
II
II

Page 5

A NeuroGenetic Approach for Multiprocessor Scheduling
125
output to all other machine 2 nodes. These signals are used to enforce the constraint that the
same machine cannot process more than one operation at the same time. The third output
(OMR) is in the reverse direction, back to the operation node. Whenever an operation is
assigned to a machine, the machine node sends a signal back to the operation node,
indicating that it has been assigned. This signal changes the state of the operation node and
triggers other functions.
We now describe the input, activation and output functions for each node and the search
strategy for the weights. We will need the following notation to describe our functions:
n
Number of jobs
m
Number of machines
C
J
Set of jobs = {1,..,n}
Ji
Job i, i ∈ J
M
Set of machines = {1,..,m}
Mk
Machine k, k ∈ M
O
Set of operations
Oij
ijth operation node, i ∈ J, j∈ M
Mk,ij
Node for machine k, connected from Oij, i ∈ J, j ∈ M, k ∈ M
ωij
Weight on the link from Oij to machine node, i ∈ J, j ∈ M
ωm
Large weight on the link between machine nodes.
α
Search coefficient
εc
Error in iteration c
OF
Final Dummy operation node
STijk
Start time of Oij on Mk, i ∈ J, j ∈ M, k ∈ M
PTij
Processing Time of Jion Mj, i ∈ J, j ∈ M
Winij
Winning status of Job Jion Machine Mj, i ∈ J, j ∈ M
Current iteration
Following are all functions of elapsed time t :
t
Elapsed time
II(t)
Input function value of the Initial I node.
IOij(t)
Input function value of Operation node Oij, i ∈ J, j ∈ M
IOF(t)
Input function value of Operation node OF
IMk,ij(t)
Input function value of Machine node k from operation Oij, i ∈ J, j ∈ M, k∈M
OI(t)
Output function value of the Initial I node.
OOij(t)
Output function value of Operation node Oij, i ∈ J, j ∈ M
OOF(t)
Output function value of Operation node OF
OMFk,ij(t)
Output of Mc. node Mk,ij to the operation node in forward direction, i∈ J, j∈
M, j ? m, k∈M
OMFk,ijF(t)
Output of Machine node Mk,ij to OF in the forward direction, i ∈ J, j=m, k∈ M
OMRk,ij(t)
Output of Machine node Mk,ij to Oij in reverse direction, i ∈ J, j ∈ M, k∈ M
OMMk(t)
Output of Machine node Mk* to Mk* k∈ M
θOij(t)
Activation function of Operation node Oij, i ∈ J, j ∈ M
θMk,ij(t)
Activation function of Machine node Mk,ij, i ∈ J, j ∈ M, k∈M
assignijk(t)
Operation Oij assigned to Machine Mk
S(t)
Set of operations that can start at time t. S(t) = {Oij| OOij(t)= 1}