ANNSA: a hybrid artificial neural network/simulated annealing algorithm for optimal control problems
ABSTRACT This paper introduces a numerical technique for solving nonlinear optimal control problems. The universal function approximation capability of a threelayer feedforward neural network has been combined with a simulated annealing algorithm to develop a simple yet efficient hybrid optimisation algorithm to determine optimal control profiles. The applicability of the technique is illustrated by solving various optimal control problems including multivariable nonlinear problems and free final time problems. Results obtained for the different case studies considered agree well with those reported in the literature.

Article: Multistep optimal control of complex process: a genetic programming strategy and its application
[Show abstract] [Hide abstract]
ABSTRACT: In many industrial processes, especially chemistry and metallurgy industry, the plant is slow for feedback and data test because of complex and varying factors. Considering the multiobjective feature and the complex problem of production stability in optimal control, this paper proposed an optimal control strategy based on genetic programming (GP), used as a multistep state transferring procedure. The fitness function is computed by multistep comprehensive evaluation algorithm, which provides a synthetic evaluation of multiobjective in process state based on single objective models. The punishment to process state variance is also introduced for the balance between optimal performance and stability of production. The individuals in GP are constructed as a chain linked by a few relation operators of time sequence for a facilitated evolution in GP with compact individuals. The optimal solution gained by evolution is a multistep command program of process control, which not only ensures the optimization tendency but also avoids violent process variation by adjusting control parameters step by step. An optimal control system for operation direction is developed based on this strategy for imperial smelting process in Shaoguan. The simulation and application results showed its effectiveness for production objects optimization in complex process control.Engineering Applications of Artificial Intelligence 08/2004; 17(5):491500. · 1.96 Impact Factor  SourceAvailable from: Gregory Francois[Show abstract] [Hide abstract]
ABSTRACT: The aim of this paper is to present an approach to dynamic offline optimization of batch emulsion polymerization reactors using a stochastic optimizer. The control objective is to find the optimal temperature profile that minimizes the final batch time constrained by the final conversion and molecular weight. In this study, we evaluate the applicability of MSIMPSA, a simulatedannealingbased algorithm, to solve the optimal control problem. Two cases are studied: first, a simple case without energy balances and, second, a more realistic case using energy balances and constraints on heat transfer. In addition, an SQP optimizer was applied to perform a local optimization around the best results obtained by MSIMPSA. The following conclusions can be drawn from the results: (i) MSIMPSA can be applied in an easy and straightforward manner (blackbox approach) to such optimal control problems. (ii) Even though MSIMPSA is a stochastic algorithm, the best obtained solution is so good that it cannot be further improved by local optimization methods.Industrial & Engineering Chemistry Research 10/2004; 43(24). · 2.24 Impact Factor  [Show abstract] [Hide abstract]
ABSTRACT: In the present work, it was developed an adaptive model predictive control algorithm to control a semi batch pyrolysis reactor. An 8L reactor and a separation system were assembled for this purpose. The reactor temperature control was carried out through a digital control system implemented for this process. The model used to infer about the process was a multilayered neural network completely recursive. To avoid offset problems an adaptive algorithm was applied, performing online weights actualization. The neural network was used to explicitly predict the process output (reactor temperature) through a predefined prediction horizon. Through optimization, this output vector was used to estimate the process input (heat power supply). A qualitative analysis of the products and the total time of operation for a fast pyrolysis, sustained for ten minutes in the set point temperature, had pointed out, in this case, a superior performance to the proposed controller when compared with the classical feedback controller. Besides, the temperature stabilizes without overshoots and offsets. The developed control algorithm was able to compensate the strong disturbances that occur during the partial discharge of pyrolysis products, due to reactor pressure relief. A performance index based on ISA criteria was used and again, exhibits consistent improvement of the adaptive control over a classical feedback algorithm.
Page 1
Chemical Engineering Science 58 (2003) 3131–3142
www.elsevier.com/locate/ces
ANNSA: a hybrid arti?cial neural network/simulated annealing
algorithm for optimal control problems
Debasis Sarkar, Jayant M. Modak∗
Department of Chemical Engineering, Indian Institute of Science, Bangalore 560012, India
Received 15 July 2002; received in revised form 12 November 2002; accepted 26 March 2003
Abstract
This paper introduces a numerical technique for solving nonlinear optimal control problems. The universal function approximation
capability of a threelayer feedforward neural network has been combined with a simulated annealing algorithm to develop a simple yet
e?cient hybrid optimisation algorithm to determine optimal control pro?les. The applicability of the technique is illustrated by solving
various optimal control problems including multivariable nonlinear problems and free ?nal time problems. Results obtained for the di?erent
case studies considered agree well with those reported in the literature.
? 2003 Elsevier Ltd. All rights reserved.
Keywords: Neural networks; Simulated annealing; Optimisation; Control; Bioreactors; Chemical reactors
1. Introduction
Optimal control problems appear frequently in many dif
ferent industries and the determination of optimal control
trajectories is extremely important for better performance of
process industries. Typical examples in chemical engineer
ing include temperature pro?le determination for batch re
actors, substrate feed rate pro?les for fedbatch bioreactors,
and catalyst distribution in packed bed reactors. Since the
objective is to determine the trajectories of the optimal con
trol variables as a function of time or length of the reactor,
these problems fall under the realm of variational calculus.
In general, the solution to such problems is often a di?cult
and timeconsuming task even for a single control variable
problem. There are several methods available to solve opti
mal control problems and they can be classi?ed broadly into
two major categories.
One class of optimisation methods uses variational
approach based on the optimal control theory and thus re
quires the solution of multipoint boundary value problems
with initial conditions for state variables and terminal con
ditions for adjoint (costate) variables. Analytical solutions
can be determined using the optimality conditions only
when the number of state variables is few and the numerical
∗Corresponding author. Tel.: +91803942768; fax: +91803608121.
Email address: modak@chemeng.iisc.ernet.in (J. M. Modak).
solutions are usually unavoidable. Several numerical tech
niques using the optimality conditions have been reported in
the literature. These include among others multiple shooting
methods for multipoint boundary value problems (Oberle
& Sothmann, 1999), boundary condition iterative methods
(Ray, 1981), and gradient methods (Stutts, 1983). These are
the most elegant and precise methods to solve optimal con
trol problems and the characteristic properties of the opti
mal solutions can be clearly identi?ed here. However, these
methods require very good initial guesses for either con
trol variable trajectory or unknown boundary conditions in
order to achieve convergence. Additionally, the switching
structure (sequence of maximum, minimum, and interme
diate control) has to be guessed correctly in advance in
some cases. The other class of optimisation methods dis
cretises the independent variable (time for batch/fedbatch
reactors or axial position for packed bed reactors) horizon
into ?nite subintervals and piecewise constant or linear
control functions in the discretized intervals are determined
using di?erent search techniques (Shukla & Pushpavanam,
1998; Banga, IrrizarryRivera, & Seider, 1998; Roubos, van
Straten,&vanBoxtel,1999;Luus,2000;Na,Chang,Chung,
& Lim, 2002). Usually, these techniques are more robust in
terms of convergence. Moreover, the switching structure of
the optimal solution need not be guessed at all.
An alternate strategy to avoid discretisation can be the
representation of the control variable pro?le as a single
00092509/03/$see front matter ? 2003 Elsevier Ltd. All rights reserved.
doi:10.1016/S00092509(03)001684
Page 2
3132D. Sarkar, J. M. Modak/Chemical Engineering Science 58 (2003) 3131–3142
nonlinear function of the independent variable. However,
such a function must be capable of capturing the character
istic nonlinearities associated with the optimal control prob
lems. For example, in the case of singular control problems,
so called because control variable appears linearly in the
state and/or performance index, the control variable trajec
tory will have bangbang and/or singular arcs. The con
tinuous function chosen to represent the optimal control
pro?le must be capable of displaying the sharp discontinu
ities. However, constructing such a nonlinear function may
be di?cult in practice.
There has been considerable interest recently in the vari
ous possibilities o?ered by Neural Networks and they have
been successfully applied for various interesting applica
tions (Baughman & Liu, 1995). A key feature of neural net
works is their ability to approximate any arbitrary nonlinear
function and this provides an excellent means of generating
a continuous control pro?le in combination with any global
optimisation technique. Simulated annealing (SA), a prob
abilistic optimisation technique with a potential of ?nding
global optimal solution, is based on the ideas from statistical
mechanics(Kirkpatrick,Gelatt,&Vecchi,1983).Inthe?eld
of chemical engineering, SA has been successfully applied
in various problems like optimisation of heat exchanger
networks (Dolan, Cummings, & Le Van, 1989), separation
sequence synthesis (Floquet, Pibouleau, & Domenech,
1994), optimisation of reactive distillation processes
(Cardoso, Salcedo, de Azevedo, & Barbosa, 2000), optimi
sation of batch distillation processes (Hanke & Li, 2000),
biotechnical processes (Simutis & Lubbert, 1997), etc.
The purpose of the present study is to exploit nonlinear
function approximation capability of a neural network to
develop a simple yet e?cient hybrid optimisation strategy
by combining a multilayer feedforward neural network with
a Simulated Annealing algorithm. The e?ectiveness and the
?exibility of the proposed hybrid Arti?cial Neural Network
Simulated Annealing (ANNSA) approach is demonstrated
with four di?erent optimal control problems taken from the
literature.
2. Optimisation problem formulation
Letusconsiderasystemdescribedbythefollowingvector
di?erential equation:
dx
dt= f(x;u);(1)
with the initial conditions speci?ed as x(0). The state vector
x is an (n × 1) vector and u is an (m × 1) control vector
bounded by
umin6ui6umax;i = 1;2;:::;m: (2)
The objective is to determine the optimal control policy u(t)
in the time interval 06t 6tfwhich will maximise a scalar
index of performance (PI) that depends on the ?nal outcome
of the process:
Maximise
u(t)
PI = ?(x(tf));(3)
where tfis the ?nal time that may be speci?ed a priori or
free.
3. Neural network for the time/control relationship
In the present study, the role of a neural network is to ap
proximate the control variable which is a continuous func
tion of time. A neural network with a single hidden layer
has been proved to approximate any arbitrary function with
any arbitrary precision, provided there are su?cient num
ber of neurons in the hidden layer (Cybenko, 1989; Hornik,
Stinchcombe, & White, 1989). This ability of arti?cial neu
ral networks (ANN) to approximate any arbitrary nonlinear
functions can be exploited to determine the control function
of time in optimal control problems.
Since neural network computations are usually performed
by scaling input and output quantities in the range of 0–1,
let us consider a scaled control vector ˆ u(?) where ? is the
scaledtime.Accordingtothefunctionapproximationtheory,
there should be an approximating function F(?;P) having
a ?xed number of realvalued parameters P∈R which will
e?ectively approximate the function ˆ u(?):
ˆ u(?) = F(?;P);(4)
where F is the ANN model and P represents the vector of
tunable parameters of the network, namely, its weights and
the biases. Thus, for a choice of a speci?c F (i.e. ANN),
the problem is to ?nd the set of parameters P that provides
the best possible approximation of ˆ u(?). The problem of de
termining the optimal control pro?le, therefore, reduces to
a nonlinear programming problem where the decision vari
ables are the appropriate weights and biases of the chosen
neural network:
Maximise
P
and it can, in general, be solved using any nonlinear
optimisation technique. In the present study, we choose sim
ulated annealing for this purpose because of its simplicity
and ability to produce global optimal solutions for complex
problems which outweigh its relatively large computational
requirements.
One of the most important tasks in using neural networks
is to decide on the architecture of the network. For the prob
lems we study here, we chose a threelayer feedforward
network (Fig. 1) with one neuron in the input layer corres
ponding to the time, a single hidden layer of NH neurons,
and an output layer of m neurons corresponding to the num
ber of control variables. A simple sigmoidal function is used
as an activation function for all the neurons in hidden and
output layers. We refer to this network as 1−NH−m ANN.
PI = ?(x(tf))(5)
Page 3
D. Sarkar, J. M. Modak/Chemical Engineering Science 58 (2003) 3131–3142 3133
θj
b
bj
τ
Bias unit
Hidden layer
Bias unit
Input layer
Wij
Output layer
out i,
(m)
(1)
u^ u^
1m
( NH)
Fig. 1. A threelayer feedforward ANN architecture.
With this network, the control vector ˆ u = [ ˆ u1ˆ u2::: ˆ um]Tis
represented as follows:
1
1 + e−neti(?);
where neti(?) is
NH
?
j=1
ˆ ui(?) =i = 1;2;:::;m; (6)
neti(?) =Wij’(?j? + bj) + bi;out:(7)
Wijis the weight connecting jth hidden layer neuron to the
ithoutputneuron,?jistheweightconnectinginputneuronto
the jth hidden layer neuron, bjis the bias for the jth hidden
layer neuron, bi;outis the bias for the ith output neuron, and
’(?j?+bj) is the output of the jth hidden layer neuron upon
application of sigmoidal activation function:
1
1 + e−(?j?+bj);
For a 1−NH−m ANN used in this study, there are 2NHcon
nection parameters (?j;bj) between input and hidden layers,
and m(NH+ 1) connection parameters (Wij;bi;out) between
hidden and output layers. Thus, the total number of param
eters in set P = {Wij;?j;bj} is N = 2NH+ m(NH+ 1).
One of the key features of an optimal control pro?le,
particularly for singular control problems, is the sharp jumps
between maximum, minimum, and intermediate values for
the control variable. As we use an ANN to approximate the
control pro?le, the ANN must be able to predict these sharp
edges when the control variable switches from one level to
’(?j? + bj) =j = 1;:::;NH:(8)
other. Since ANN is a universal function approximator, it
can indeed predict these sharp changes in the pro?le, pro
vided there are enough neurons in the hidden layer as dis
cussed below.
Let us consider a typical bangbang policy ˆ u1(?) with ns
switches as shown in Fig. 2A for ns= 3. To approximate
this bangbang policy with nsswitches, ˆ u1(?) must have the
same value at nsand di?erent values of ? as shown by the
dotted line in Fig. 2A. In view of the unimodal sigmoidal
activation function used (Eq. (6)), the same value of net1(?)
for the nsdi?erent values of ? must be predicted by the ANN
model. In other words,
net1(?) = c (9)
must have nsroots, where c is any arbitrary constant. For
the 1−NH−1 ANN model described above (Eqs. (6)–(8)),
Eq. (9) can be written as
W11
1 + ?1
1 + ?2
1 + ?NH
where ?jis e−(?j?+bj). Eq. (10) can be simpli?ed and ex
pressed as a sum of exponential functions in the following
form:
+
W12
+ ··· +
W1NH
= c;(10)
c1e−?1?+ c2e−?2?+ ··· + cle−?l?= 0;
where ciand ?i, i = 1;2;:::;l are real constants de?ned in
terms of parameter set P, and l=2NH. It is known that a sum
of l real exponential functions, with real coe?cients, has at
most (l−1) real zeros, where equal zeros are counted with
full multiplicity (Pontryagin, Boltyanskii, Gamkrelidze, &
Mischenks, 1962). Thus, Eq. (11) has at most (l − 1), i.e.
2NH− 1 real roots. This leads to the conclusion that to ap
proximate a bangbang policy with nsswitches, the follow
ing must hold true:
(11)
2NH− 1¿ns:
It may be pointed out that Eq. (12) gives us the bare min
imum number of hidden neurons that is necessary and for
a reasonably good approximation of the control policy we
may need more number of neurons in the hidden layer than
that predicted by the equality sign of the above Eq. (12).
Fig. 2b–d show the results of approximation of the
bangbang policy given by Fig. 2A by a 1 − NH− 1 ANN
trained by backpropagation algorithm with various number
of hidden neurons (NH). In view of Eq. (12), the minimum
number of neurons necessary in the hidden layer is 2. It can
be seen from the ?gures that NH= 1 can predict only one
switch, and therefore fails to approximate the policy (Fig.
2b). NH = 2 satis?es Eq. (12) and predicts the required
number of switches. As a result, there is a qualitative match
between the predicted and desired policy (Fig. 2c), but the
quantitative agreement is not very good. It can be expected
that the introduction of additional neurons in the hidden
layer will improve the prediction capability of the ANN
model. This can be clearly seen in Fig. 2d which shows the
prediction of ANN model with NH= 3.
(12)
Page 4
3134 D. Sarkar, J. M. Modak/Chemical Engineering Science 58 (2003) 3131–3142
0 0.10.2 0.30.4 0.5
Time ?
0.60.7 0.80.91
0
0.2
0.4
0.6
0.8
1
0 0.10.20.30.4 0.5
Time ?
0.60.7 0.80.91
0
0.2
0.4
0.6
0.8
1
0 0.10.2 0.3 0.40.5
Time ?
0.6 0.7 0.80.91
0
0.2
0.4
0.6
0.8
1
0 0.1 0.20.3 0.40.5
Time ?
0.6 0.7 0.80.91
0
0.2
0.4
0.6
0.8
1
(a)
(b)
(c)
(d)
Fig. 2. (a) Bangbang control policy with three switches. (b–d) Prediction of control policy in (a) with various architectures of ANN. (b) 111, (c)
121, and (d) 131.
4. Implementation details of the proposed hybrid
algorithm
The SA part of our hybrid optimisation algorithm is
similar to the implementation of simulated annealing by
Go?e, Ferrier, and Rogers (1994). The details of our hybrid
optimisation algorithm ANNSA are outlined as follows:
Step 1: The SA algorithm starts at some speci?ed “high”
temperature T0with a single trial solution vector P (weights
and biases of chosen ANN) of dimension N. The elements
of P are initially generated randomly within a suitable upper
and a lower limit. The objective function (PI) for the opti
mal control problem in Eq. (5) is now evaluated by solving
the di?erential equations describing the process (Eq. (1))
with the control pro?le u(t) generated by the chosen ANN
with the above weights and biases (P) in the speci?ed time
interval 06t 6tf. The IMSL subroutine DGEAR, which
is based on variable step size Gear’s method for the solv
ing sti? di?erential equations (Gear, 1971), is used for the
solution of the di?erential equations. A maximum time step
of 0:01 h is used to obtain numerical solutions.
Step 2: A new candidate solution P?is now generated by
means of random perturbation of the current solution,
P?
i= Pi+ r?i;(13)
where r ∈[ − 1;1] is a uniformly distributed random num
ber and ?i is the ith element of the step length vector ?
of dimension N. The new objective function value PI?
is then evaluated for this new solution as discussed in
Step 1.
Step 3: If PI?¿PI, the new solution P?is always ac
cepted, P is set to P?and the algorithm moves uphill. But if
PI?6PI, the new solution P?is accepted with a probability
Page 5
D. Sarkar, J. M. Modak/Chemical Engineering Science 58 (2003) 3131–31423135
Table 1
SA parameters used for optimisation
Initial temperature, T0
Temperature reduction factor, ?
Feasible solution space
Step length vector, ?
NS
NT
NC
?
1.0
0.95
−100 to 100
100
20
20
4
10−6
of (Metropolis criterion)
P = exp(−?PI=T);
where ?PI=PI?−PI. If the new solution P?is accepted, P
is set to P?, and the algorithm moves downhill. Otherwise,
P?is rejected.
Step 4: After NS×N function evaluations, the step length
vector ? is so adjusted that 50% of all moves are accepted.
In order to simulate the thermal equilibrium at every temper
ature, the temperature is reduced after NT times the above
loop (NT× NS× N function evaluations) according to the
following decrement function:
(14)
T?= ?T; (15)
where ? is a constant temperature reduction factor somewhat
less than unity.
Step 5: The algorithm ends by comparing the last NC
values of the largest function values from the end of each
temperature reduction with the most recent one and the op
timum function value. If all these di?erences are less than a
prespeci?ed small value ?, the algorithm terminates.
Table 1 summarises the values of the SA parameters that
have been used in this study unless mentioned otherwise. It
may be pointed out here that not much e?ort has been made
in ?ne tuning these parameters for better performance of the
algorithm.
5. Optimisation results and discussion
5.1. Case study I: Optimal production of secreted protein
in a fedbatch bioreactor
This optimal control problem deals with the maximisation
of the production of secreted heterologous protein by a yeast
strain in a fedbatch bioreactor with the feed ?ow rate as the
onlycontrolvariable.ThiswasoriginallyformulatedbyPark
and Ramirez (1988) and the required details of the model
can be found in their work. The objective is to maximise the
total secreted protein in the bioreactor at the end of fedbatch
operation for a speci?ed ?nal time, tf (=15 h). The feed
rate is constrained as 06u610 l=h.
SA alone can be used to solve this optimal control prob
lem if the control function is approximated by a piecewise
constant or piecewise linear functions. Fig. 3 shows the
0510 15
0
0.5
1
1.5
2
2.5
3
Time (h)
Feed Rate (L/h)
Fig. 3. Optimal feed rate pro?le for secreted protein (Case Study I):
Simulated annealing with piecewise constant feed.
optimal feed rate pro?le obtained for this protein secre
tion process when the time horizon is divided into 50 equal
intervals and a piecewise constant feed is used in each
interval. Although the pro?le yields a good performance
index (PI = 32:759055), it can be seen from the ?gure that
the pro?le shows high activity with lots of abrupt jumps
in the values of feed rates. This behaviour is characteristic
of many algorithms which approximate the control function
by piecewise constant functions and some ?ne tuning of
the algorithm such as proper choice of number of intervals,
introduction of ?lters etc. is required to generate smoother
pro?les (Tholuder & Ramirez, 1997).
The proposed ANNSA algorithm was run with various
architectures of ANN and the results of the optimisation
studies are presented in Table 2. The pro?les for the control
variable (substrate feed rate) obtained for the various cases
are shown in Fig. 4a–e and Fig. 4f shows the optimal sub
strate feeding policy that is obtained through the application
of optimal control theory (Park & Ramirez, 1988). Even
with as few hidden neurons as 3 (Fig. 4a), the predicted feed
rate policy displays all the characteristics of the optimal pol
icy except the presence of the batch period in the time inter
val 9.65–10:70 h (Fig. 4f). With ?ve neurons in the hidden
layer, ANNSA prediction of the feed pro?le agrees quali
tatively with the optimal solution, that is, feeding sequence
is correctly predicted as singular, batch, singular, and max
imum feed rates. The insensitivity of the control pro?le on
the yield (PI) is clearly seen from Table 2 as even a subop
timal pro?le also yields a high performance index. The best
performance index obtained is 32.633647 (case E in Fig. 4)
and this is in good agreement with the values reported in the
literature for this bioreactor. Fig. 5 shows the convergence
pro?le for the case where we use a 161 ANN (Fig. 4e).
Page 6
3136 D. Sarkar, J. M. Modak/Chemical Engineering Science 58 (2003) 3131–3142
Table 2
Optimal production of secreted protein
Case ANN
model
No. of decision
variables
Performance
index
Nevala
Final
temperature
×103
0.0045
0.2461
0.0017
0.2221
0.0232
A
B
C
D
E
121
131
141
151
161
7 30.894534
32.359190
32.414095b
32.630342
32.639583
674801
652001
6734001
1056001
1588401
10
13
16
19
aNumber of function evaluations.
bSA parameters: NT=100, feasible solution space: −200 to +200, ? = 200.
05 1015
0
2
4
6
8
10
Time (h)
Feed Rate (L/h)
05 1015
0
2
4
6
8
10
Time (h)
Feed Rate (L/h)
05 10 15
0
2
4
6
8
10
Time (h)
Feed Rate (L/h)
05 10 15
0
2
4
6
8
10
Time (h)
Feed Rate (L/h)
05 1015
2
4
6
8
10
Time (h)
Feed Rate (L/h)
05 1015
0
2
4
6
8
10
Time (h)
Feed Rate (L/h)
(a)
(b)
(c)
(d)
(e) (f)
Fig. 4. Optimal feed rate pro?les for secreted protein (Case Study I): ANNSA with various architectures of ANN: (a) 121, (b) 131, (c) 141, (d)
151, (e) 161, and (f) feed pro?le using optimal control theory.
Page 7
D. Sarkar, J. M. Modak/Chemical Engineering Science 58 (2003) 3131–31423137
100
101
102
103
104
105
106
0
5
10
15
20
25
30
35
Number of function evaluations
Performance index
Fig. 5. Convergence pro?le of ANNSA algorithm for secreted protein
problem (Case Study I) with 161 ANN.
The nonlinearity of the optimal feed rate policy (Fig. 4f)
results from: (i) the time varying feed rate rates during the
singular intervals and (ii) the sequence of feed rate among
maximum, minimum, and singular interval feed rate with
sudden switching among them. It is instructive to examine
as to how an ANN model predicts these two aspects of time
varying feed rate policy. This is achieved by examining the
variation of output of each neuron in the hidden layer (Eq.
(8)) with time. Fig. 6 shows the output of each neurons
in the hidden layer for the 161 ANN (case E in Fig. 4).
Because of the sigmoidal activation function, the output of
theseneuronsareboundedbetween0and1.Acertaindegree
of order can be seen from the contributions of the individ
ual neurons. It is interesting to note that not all the neurons
are “active” (that is, nonzero contribution) throughout the
entire operating period of 15 h. Furthermore, most of the
neurons remain either “on” with unity output or “o?” with
zero output. For example, the 1st neuron has a constant con
tribution of unity till 9:6 h followed by rapidly decreasing
contribution till 10:8 h and no contribution thereafter. The
time interval of 9.6–10:8 h corresponds to the switching of
feed rate from 9.65–10:70 h (Fig. 4f). In other words, the
sudden switches in the optimal feed rate is emulated by the
on/o? nature of the neurons. Except for the 4th neuron, all
the neurons in the hidden layer showed this type of be
haviour. The output of the 4th neuron varies continually
throughout the operating period and this neuron is mainly
responsible for the prediction of feed rate variation with time
during the singular intervals.
5.2. Case study II: induced foreign protein production by
recombinant bacteria
Lee and Ramirez (1994) have developed the optimal
nutrient and inducer feeding strategy for the fedbatch
production of induced foreign protein using recombinant
bacteria. They used the optimal control theory and showed
the existence of singular control arcs for this system. There
after, this fedbatch system with two control variables has
been widely used for optimal control studies by various
algorithms (Carrsco & Banga, 1997; Roubos et al., 1999;
Mekarapiruk & Luus, 2000; Jayaraman, Kulkarni, Gupta,
Rajesh, & Kusumaker, 2001). The details of the model can
be found in Lee and Ramirez (1994). The objective is to
maximise the pro?tability of the process for a speci?ed
?nal time of fedbatch operation. This can be described
mathematically by the following performance functional:
Maximise
u(t)
PI = x4(tf)x1(tf) − Q
?tf
t0
u2(t)dt; (16)
wherex4istheforeignproteinconcentration,x1isthereactor
volume, and Q is the ratio of the cost of the inducer to the
value of the protein product. The fermentation time is ?xed
at 10 h. The nutrient (glucose) feed rate (u1) and the inducer
feed rate (u2) values are constrained as 06ui61:0, i=1;2.
Two di?erent cases have been considered for this
twocontrol variable problem: (i) the cost of the inducer
can be neglected, Q = 0, and (ii) the cost of the inducer
cannot be neglected, Q = 5. Three di?erent ANN architec
tures (132, 142, and 152) were used for this problem
and the results of our optimisation studies are presented in
Table 3. Fig. 7 shows the optimal control pro?les that are
obtained with a 142 ANN for both the cases when Q = 0
and 5. It can be seen that when the cost of the inducer can
be neglected (Q = 0) both the feed rates have to be varied
for maximum performance, whereas when the cost of the
inducer cannot be neglected (Q = 5) only the inducer feed
rate has to be varied, as the nutrient feed rate is maintained
at zero for the entire period of operation. Table 4 shows the
performance index reported in the literature for this system
using di?erent methods. It can be seen that the results of
our ANNSA algorithm match favourably with the reported
results.
5.3. Case study III: isothermal CSTR with complex
reactions
This problem considers a nonlinear system consisting of
?ve simultaneous reactions taking place in an isothermal
CSTR. There are four control variables and thus it serves
well to test the e?ciency of the proposed algorithm. This
optimal control problem has been studied by Luus (1990)
to test the e?ciency of the dynamic programming algorithm
and the required details of the model can be found in the
above reference. The objective is to maximise the economic
bene?t of the CSTR by choosing u1;u2;u3, and u4in the
Page 8
3138 D. Sarkar, J. M. Modak/Chemical Engineering Science 58 (2003) 3131–3142
05 10 15
0
0.2
0.4
0.6
0.8
1
Time (h)
1st Neuron
05 1015
0
0.2
0.4
0.6
0.8
1
Time (h)
2nd Neuron
05 10 15
0
0.2
0.4
0.6
0.8
1
Time (h)
3rd Neuron
05 10 15
0
0.2
0.4
0.6
0.8
1
Time (h)
4th Neuron
05 1015
0
0.2
0.4
0.6
0.8
1
Time (h)
5th Neuron
05 10 15
0
0.2
0.4
0.6
0.8
1
Time (h)
6th Neuron
Fig. 6. Contribution of each hidden neuron for 161 ANN for feed rate in Fig. 4e.
Table 3
Induced foreign protein production
CaseANN
model
No. of decision
variables
Q Performance
index
Nevala
Final
temperature
×105
0.0748
0.0448
0.0404
A
B
C
132
142
152
14
18
22
0
0
0
1.009750
1.009796
1.009783
1545601
2059201
2534401
D
E
F
132
142
152
14
18
22
5
5
5
0.816620
0.816699
0.816713
1338401
2023201
2120801
0.4991
0.0578
0.4504
aNumber of function evaluations.
Page 9
D. Sarkar, J. M. Modak/Chemical Engineering Science 58 (2003) 3131–31423139
02468 10
0
0.2
0.4
0.6
0.8
1
1.2
Time (h)
u1 (L/h)
02468 10
0
0.2
0.4
0.6
0.8
1
1.2
Time (h)
u2 (L/h)
024
Time (h)
68 10
0
0.2
0.4
0.6
0.8
1
1.2
u1 (L/h)
024
Time (h)
68 10
0
2
4
6
8
10
12x 10
3
u2 (L/h)
(a)(b)
(c) (d)
Fig. 7. Optimal feed rate (u1and u2) pro?les for induced foreign protein production (Case Study II): ANNSA with 142 ANN. (a), (b): Q = 0; (c),
(d): Q = 5.
time interval 06t 6tf. The ?nal time tf is speci?ed as
0:2 h a priori. The constraints on the control variables are:
06u1620, 06u266, 06u364, and 06u4620.
Luus (1990) reported the results of two sets of optimisa
tion studies for this problem. In the ?rst case, u4is kept ?xed
at 6.0 and the optimisation is carried out on u1;u2, and u3.
Rao and Luus (1972) also solved this threecontrol problem
with an improved control vector iteration procedure. In the
second case, all the four control variables are taken free. In
order to compare our results with those reported in the litera
ture, we ?rst solved this problem by keeping u4 ?xed at
6.0. The ANN used to determine the optimal trajectories for
u1;u2, and u3is 131. Fig. 8
pro?les obtained for the control variables u1;u2; and u3. This
results in a performance index of 20.083093 which agrees
well with that reported by Luus (1990) using dynamic pro
gramming (20.10 with 40 stages). The control pro?les ob
tained from our ANNSA are also in excellent agreement,
both qualitatively and quantitatively, with the pro?les ob
tained by Rao and Luus (1972) using control vector iteration
who reported a maximum performance index of 20.09. Next,
the problem is solved by the ANNSA algorithm for the case
when all the control variables are free. Table 5 summarises
the results of our optimisation study for various architec
presents the optimal control
Table 4
Induced foreign protein production: a comparative study
Method Performance index
Q = 0Q = 5
Optimal control theory
(Lee & Ramirez, 1994)
Adaptive stochastic algorithm
(Carrsco & Banga, 1997)
Evolution strategy
(Roubos et al., 1999)
Iterative dynamic programming
(Mekarapiruk & Luus, 2000)
Ant colony algorithm
(Jayaraman et al., 2001)
Proposed ANNSA with 142 ANN
1.0012 0.7988
1.00970.8165
Not reported0.8149
1.0096040.816480
1.00194 0.8095
1.0097960.816699
tures of ANN. Fig. 9 presents the optimal control pro?les
for all the four variables for the 154 ANN. The pro?les are
in good agreement with those reported by Luus (1990). The
performance indices obtained in this study (21.78, Table 5)
also match well with the value reported by Luus (21.76 with
11 stages). The optimal control policy for four control prob
lems is completely di?erent from the case with three control
Page 10
3140 D. Sarkar, J. M. Modak/Chemical Engineering Science 58 (2003) 3131–3142
0 0.050.1 0.150.2
0
2
4
6
8
10
12
14
16
18
20
Control
Time (h)
u1
u2
u3
Fig. 8. Optimal control pro?les for isothermal CSTR with three control
variables (Case Study III): ANNSA with 133 ANN.
variables. Also, the total ?ow rate (u1+ u2+ u4) is zero
in the time interval 0:0926t 60:122 and this makes the
CSTR semicontinuous under the optimal operating condi
tions. A similar observation has also been reported by Luus
(1990). It may be pointed out that the gain in performance
index by allowing u4to be a free variable is about 8%.
5.4. Case study IV: biphasic growth of yeast
Pyun, Modak, Chang, and Lim (1989) reported the opti
misation of fedbatch growth of yeast for various initial con
ditions of the bioreactor using optimal control theory. We
consider here caseA from their study where the initial sub
strate (glucose) and cell concentrations are low. The optimal
control problem is to determine the optimal glucose feed
ing policy that maximises the pro?t de?ned as the di?erence
Table 5
Isothermal CSTR with complex reactions
CaseANN
model
No. of decision
variables
Performance
index
Nevala
Final
temperature
×104
0.0188
0.3689
0.0582
A
B
C
134
144
154
22
28
34
21.160195
21.785739
21.789844
2270401
2240001
8024001
aNumber of function evaluations.
between the product value (cell mass) and the operating
cost:
Maximise
u(t)
PI = (XV)tf− ?tf; (17)
where X is the concentration of cells and V is the fermentor
volume. The ? is a cost factor, a composite operating cost
per unit time per unit yeast cell mass price. The ?nal time tf
is not speci?ed a priori and has to be determined optimally.
The required details of the model equations can be found in
the original reference (Pyun et al., 1989).
The proposed ANNSA algorithm can be easily ex
tended to handle this free ?nal time problem simply by
including the ?nal time tf as an additional decision vari
able to be optimally determined. Thus, in this approach,
the number of decision variables is N + 1, where N is
the number of weights and biases of ANN chosen to
approximate the control function u(t). The ANN cho
sen here is 131 which results in 11 decision variables.
Fig. 10 presents the optimal control pro?le obtained for
this problem. The feeding sequence is maximumbatch–
singularbatch and this is what can be expected from the
application of optimal control theory. The performance in
dex obtained is 0.216218 and the ?nal time predicted is
12.65. The use of 151 ANN resulted in a performance
index of 0.216942 with a ?nal time of 12:65 h. These val
ues compare favourably with the results reported by Pyun
et al. (1989). In fact, the performance index obtained in
this study is signi?cantly higher than the reported results
(PI = 0:20660;tf= 12:63).
6. Conclusion
This paper proposes a simple yet e?ective hybrid Ar
ti?cial Neural Network Simulated Annealing (ANNSA)
algorithm for the solution of optimal control problems that
appear in chemical engineering applications. The ability of
a feedforward neural network with a single hidden layer to
approximate any arbitrary function has been exploited and
the optimal control problem is transformed into a nonlin
ear programming problem where the decision variables are
the weights and biases of the networks. Simulation studies
Page 11
D. Sarkar, J. M. Modak/Chemical Engineering Science 58 (2003) 3131–31423141
0 0.050.1 0.15 0.2
0
2
4
6
8
10
12
14
16
18
20
Time (h)
u1
0 0.05 0.10.150.2
0
2
4
6
8
10
12
14
16
18
20
Time (h)
u2
0 0.05 0.10.150.2
0
2
4
6
8
10
12
14
16
18
20
Time (h)
u3
0 0.050.1 0.150.2
0
2
4
6
8
10
12
14
16
18
20
Time (h)
u4
Fig. 9. Optimal control pro?les for isothermal CSTR with four control variables (Case Study III): ANNSA with 154 ANN.
show that not many neurons in the hidden layer are neces
sary for a reasonably good approximation of the control pro
?les. The applicability of the proposed ANNSA method has
been demonstrated by solving four wellknown challeng
ing optimal control problems from the literature. The results
obtained for the case studies considered are in excellent
agreement with the published results. The computational
di?culties associated with the solution of singular optimal
control problems can be very easily avoided as the ANNSA
method can clearly identify the singular arcs in the opti
mal control problems. Also, the e?ciency of the method is
not dependent on the choice of initial guesses of the deci
sion variables. The present approach, as demonstrated in this
study, is very ?exible and can easily solve both multicontrol
and free ?nal time problems.
Notation
bi;out
bj
F
m
N
Neval
NH
NS;NT;NC SA parameters
ns
number of switches in the control policy
PI performance index
P
vector of weights and biases for ANN
r uniform random number in [ − 1;1]
t time, h
bias for ith output neuron
bias for jth hidden neuron
function represented by ANN
number of control variables/output neurons
number of weights and biases of ANN
number of function evaluations
number of hidden layer neurons in ANN
Page 12
3142 D. Sarkar, J. M. Modak/Chemical Engineering Science 58 (2003) 3131–3142
02468 101214
0
0.05
0.1
0.15
0.2
Time (h)
Feed Rate (L/h)
Fig. 10. Optimal feed rate pro?le for biphasic growth of yeast (Case
Study IV): ANNSA with 131 ANN.
T0
u
ˆ u
Wij
initial temperature of SA
control variable vector
scaled control variable vector
weight connecting ith output neuron to jth
hidden neuron
state variable vectorx
Greek letters
?
?
?
?
’
?
?j
temperature reduction factor
SA convergence parameter
step length vector
performance index
sigmoidal activation function
scaled time
weight connecting input neuron to jth hidden
neuron
Subscripts
f
max
min
?nal
maximum
minimum
References
Banga, J. R., IrrizarryRivera, R., & Seider, W. D. (1998). Stochastic
optimisation for optimal and modelpredictive control. Computers &
Chemical Engineering, 22(4/5), 603–612.
Baughman, D. R., & Liu, Y. A. (1995). Neural networks in bioprocessing
and chemical engineering. San Diego, CA: Academic Press.
Cardoso, M. F., Salcedo, R. L., de Azevedo, S. F., & Barbosa, D.
(2000). Optimization of reactive distillation processes with simulated
annealing. Chemical Engineering Science, 55, 5059–5078.
Carrsco, E. F., & Banga, J. R. (1997). Dynamic optimisation of
batch reactors using adaptive stochastic algorithms. Industrial and
Engineering Chemistry Research, 36(6), 2252–2261.
Cybenko, G. (1989). Continuous valued neural networks with two hidden
layers are su?cient. Mathematics of Control, Signals, and Systems,
2, 303–314.
Dolan, W. B., Cummings, P. T., & Le Van, M. D. (1989). Process
optimisation via simulated annealing: Application to network design.
A.I.Ch.E. Journal, 35(5), 725–736.
Floquet, P., Pibouleau, L., & Domenech, S. (1994). Separation sequence
synthesis: How to use simulated annealing procedure?. Computers and
Chemical Engineering, 18(11/12), 1141–1148.
Gear, C. W. (1971). Numerical initial value problems in ordinary
di?erential equations. Englewood Cli?s, NJ: PrenticeHall, Inc.
Go?e, W. L., Ferrier, G. D., & Rogers, J. (1994). Global optimisation of
statistical functions with simulated annealing. Journal of Econometrics,
60, 65–99.
Hanke, M., & Li, P. (2000). Simulated annealing for the optimisation
of batch distillation processes. Computers and Chemical Engineering,
24, 1–8.
Hornik, K., Stinchcombe, M., & White, H. (1989). Multilayer feedforward
networks are universal approximators. Neural Networks, 2, 359–366.
Jayaraman, V. K., Kulkarni, B. D., Gupta, K., Rajesh, J., & Kusumaker,
H. S. (2001). Dynamic optimisation of fedbatch bioreactors using the
ant algorithm. Biotechnology Progress, 17, 81–88.
Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983). Optimisation by
simulated annealing. Science, 220, 671–680.
Lee, J., & Ramirez, W. F. (1994). Optimal fedbatch control of induced
foreign protein produced by recombinant bacteria. A.I.Ch.E. Journal,
40, 899–907.
Luus,R. (1990). Application
highdimensional nonlinear optimal control problems. International
Journal of Control, 52(1), 239–250.
Luus, R. (2000). Iterative dynamic programming. London, UK: Chapman
& Hall/CRC.
Mekarapiruk, W., & Luus, R. (2000). Optimal control by iterative dynamic
programming with deterministic and random candidates for control.
Industrial and Engineering Chemistry Research, 39, 84–91.
Na, J. G., Chang, Y. K., Chung, B. H., & Lim, H. C. (2002). Adaptive
optimisation of fedbatch culture of yeast by using genetic algorithms.
Bioprocess and Biosystems Engineering, 24, 299–308.
Oberle, H. J., & Sothmann, B. (1999). Numerical computation of optimal
feed rates for a fedbatch fermentation model. Journal of Optimisation
Theory and Applications, 100(1), 1–13.
Park, S., & Ramirez, W. F. (1988). Optimal production of secreted protein
in fedbatch reactors. A.I.Ch.E. Journal, 34(9), 1550–1558.
Pontryagin, L. S., Boltyanskii, V. G., Gamkrelidze, R. V., & Mischenks,
E. F. (1962). The mathematical theory of optimal processes. English
translation by K.N. Trirogo?. New York: Wiley.
Pyun, Y. R., Modak, J. M., Chang, Y. C., & Lim, H. C. (1989).
Optimisation of biphasic growth of saccharomyces carlsbergenesis in
fedbatch culture. Biotechnology and Bioengineering, 33, 1–10.
Rao, S. N., & Luus, R. (1972). Evaluation and improvement of control
vector iteration procedures for optimal control problems. The Canadian
Journal of Chemical Engineering, 50, 777–784.
Ray,W.H.(1981).Advancedprocesscontrol.NewYork:McGrawHillInc.
Roubos, J. A., van Straten, G., & van Boxtel, A. J. B. (1999). An
evolutionary strategy for fedbatch bioreactor optimisation; concepts
and performance. Journal of Biotechnology, 67(2/3), 173–187.
Shukla, P. V., & Pushpavanam, S. (1998). Optimisation of biochemical
reactors—ananalysisof di?erent
operation. Chemical Engineering Science, 53(2), 341–352.
Simutis, R., & Lubbert, A., (1997). A comparative study on random search
algorithms for biotechnical process optimization. J. Biotechnology, 52,
245–256.
Stutts, B. E. (1983). An improved algorithm for singular control problems
with applications to the optimisation of the fedbatch penicillin fermen
tation. Ph.D. thesis, Purdue University, West Lafayatte, Indiana, USA.
Tholuder, A., & Ramirez, W. F. (1997). Obtaining smoother singular arc
policies using a modi?ed iterative dynamic programming algorithm.
International Journal of Control, 68(5), 1115–1128.
of dynamic programmingto
approximationsoffedbatch