Effects of learning rate on the performance of the population based incremental learning algorithm.
ABSTRACT The effect of learning rate (LR) on the performance of a newly introduced evolutionary algorithm called populationbased incremental learning (PBIL) is investigated in this paper. PBIL is a technique that combines a simple genetic algorithm (GA) with competitive learning (CL). Although CL is often studied in the context of artificial neural networks (ANNs), it plays a vital role in PBIL in that the idea of creating a prototype vector in learning vector quantization (LVQ) is central to PBIL. In PBIL, the crossover operator of GAs is abstracted away and the role of population is redefined. PBIL maintains a realvalued probability vector (PV) or prototype vector from which solutions are generated. The probability vector controls the random bitstrings generated by PBIL and is used to create other individuals through learning. The setting of the learning rate (LR) can greatly affect the performance of PBIL. However, the effect of the learning rate in PBIL is not yet fully understood. In this paper, PBIL is used to design power system stabilizers (PSSs) for a multimachine power system. Four cases studies with different learning rate patterns are investigated. These include fixed LR; purely adaptive LR; fixed LR followed by adaptive LR; and adaptive LR followed by fixed LR. It is shown that a smaller learning rate leads to more exploration of the algorithm which introduces more diversity in the population at the cost of slower convergence. On the other hand, a higher learning rate means more exploitation of the algorithm and hence, this could lead to a premature convergence in the case of fixed LR. Therefore, in setting the LR, a tradeoff is needed between exploitation and exploration.

Conference Paper: Optimization of power system stabilizer parameters using populationbased incremental learning
[Show abstract] [Hide abstract]
ABSTRACT: Populationbased incremental learning (PBIL) has been recently applied to a range of optimization problems in controller designs with promising results. It combines aspects of genetic algorithm with competitive learning. The learning rate in the standard PBIL is generally fixed which makes it difficult for the algorithm to explore the search space effectively. In this paper, the standard PBIL is improved by using a combination of adaptive and fixed learning rate that varies according to the generation. The adaptivefixed (AF) algorithm can adjust the learning rate automatically according to the degree of evolution of the search. The objective of the power system stabilizer (PSS) design is to achieve adequate stability over a wide range of power system operating conditions. The proposed controller is compared with conventional PBIL with fixed learning rate (PBIL) and tested under various operating conditions. Simulation results show that the AFPBIL based PSS provides a more efficient search capability and gives a better damping and adequate dynamic performance of the system than the conventional PBIL based PSS.Energy Conversion Congress and Exposition (ECCE), 2012 IEEE; 01/2012  [Show abstract] [Hide abstract]
ABSTRACT: A portfolio selection problem is about finding an optimal scheme to allocate a fixed amount of capital to a set of available assets. The optimal scheme is very helpful for investors in making decisions. However, finding the optimal scheme is difficult and timeconsuming especially when the number of assets is large and some actual investment constraints are considered. This paper proposes a new approach based on estimation of distribution algorithms (EDAs) for solving a cardinality constrained portfolio selection (CCPS) problem. The proposed algorithm, termed PBILCCPS, hybridizes an EDA called populationbased incremental learning (PBIL) algorithm and a continuous PBIL (PBILc) algorithm, to optimize the selection of assets and the allocation of capital respectively. The proposed algorithm adopts an adaptive parameter control strategy and an elitist strategy. The performance of the proposed algorithm is compared with a genetic algorithm and a particle swarm optimization algorithm. The results demonstrate that the proposed algorithm can achieve a satisfactory result for portfolio selection and perform well in searching nondominated portfolios with high expected returns.Technologies and Applications of Artificial Intelligence (TAAI), 2010 International Conference on; 12/2010  [Show abstract] [Hide abstract]
ABSTRACT: This paper introduces two improved forms of population based incremental learning (PBIL) algorithm applied to proportional integral derivative (PID) controller and Smith predictor design. Derivative free optimization methods, namely simplex derivative pattern search (SDPS) and implicit filtering (IMF) are used to intensify search mechanism in PBIL algorithm with improved convergence than that of the original PBIL. Although the idea of combining local methods and global methods is not new, this paper focuses application of hybrid heuristics to the vast field of control design especially, control of systems having deadtime. The effectiveness of the controller schemes arrived using the developed algorithms namely simplex derivative pattern search guided population based incremental learning (SDPSPBIL) and implicit filtering guided population based incremental learning (IMFPBIL) are demonstrated using unit step set point response for a class of deadtime systems. The results are compared with some existing methods of controller tuning.Optimization and Engineering · 0.83 Impact Factor
Page 1
Proceedings of International Joint Conference on Neural Networks, Atlanta, Georgia, USA, June 1419, 2009
Effects of Learning Rate on the Performance of the Population
Based Incremental Learning Algorithm
Komla A. Folly and Ganesh K. Venayagamoorthy
Abstract
performance of a newly introduced evolutionary algorithm
calledpopulationbased incremental
investigated in this paper. PHIL is a technique that combines a
simple genetic algorithm (GA) with competitive learning (CL).
Although CL is often studied in the context of artificial neural
networks (ANNs), it plays a vital role in PHIL in that the idea of
creating a prototype vector in learning vector quantization
(LVQ) is central to PHIL. In PHIL, the crossover operator of
GAs is abstracted away and the role of population is redefined.
PHIL maintains a realvalued probability vector (PV) or
prototype vector from which solutions are generated. The
probability vector controls the random bitstrings generated by
PHIL and is used to create other individuals through learning.
The setting of the learning rate (LR) can greatly affect the
performance of PHIL. However, the effect of the learning rate
in PHIL is not yet fully understood. In this paper, PHIL is used
to design power system stabilizers (PSSs) for a multimachine
power system. Four cases studies with different learning rate
patterns are investigated. These include fixed LR; purely
adaptive LR; fixed LR followed by adaptive LR; and adaptive
LR followed by fixed LR. It is shown that a smaller learning
rate leads tomore exploration of the
introduces more diversity in the population at the cost of slower
convergence. On the other hand, a higher learning rate means
more exploitation of the algorithm and hence, this could lead to
a premature convergence in the case of fixed LR. Therefore, in
setting the LR, a tradeoff is needed between exploitation and
exploration.
The effectof learningrate (LR) on the
learning(PHIL) is
algorithm which
I.
INTRODUCTION
T
HE past decade has witnessed a flurry of interest in the
application of Genetic Algorithms (GAs) to design
power system controllers [1] [2]. Genetic algorithms are
randomized parallel search method modeled on natural
selection [3][4].GAs have
applications in solving global optimization problems [4].
Theyoperateona population of potential
(chromosomes) applying a sequence of operators to the
population based on the relative fitness of the members [5].
recentlyseenextensive
solutions
Manuscript received January 15, 2008. This work was supported in part
by the TESP, the THRIP and the NSF EFRI grants.
K. A. Folly is with the University of Cape Town, Department of
Electrical Engineering, Private Bag, Rondebosch 7701, Cape Town, SA
phone: 27216504490; fax: 27216503465; (email: Komla.Folly@
uct.ac.za.). He is currently on Sabbatical leave and is Fulbright Scholar
with the RealTime Power and Intelligent Systems Laboratory, Missouri
University of Science and Technology, Rolla, MO 65409, USA.
G. K. Venayagamoorthy is with the RealTime Power and Intelligent
Systems Laboratory, Missouri University of Science and Technology,
Rolla, MO 65409, USA, phone: 15733416641; fax: 15733414532; (e
mail: gkumar@ieee.org.)
9781424435531/09/$25.00 ©2009 IEEE
The operators typically involve selection, crossover, and
mutation. The goal of the evolutionary process
continually improve the fitness ofthe best solution vector, as
well as the average population fitness, until some termination
criteria is satisfied. One important feature of GAs is their
implicit parallelism, i.e., the ability to search the function
space from multiple points in parallel [3][5]. Therefore,
GAs are more likely to locate the global optima than
traditional techniques, because they are much less likely to
get stuck at the local optima [5]. However, GAs have several
shortcomings. For example, the convergence of a GA is
usually slower than traditional optimization techniques.
Furthermore, the problem ofgenetic drift can lead to the lost
of diversity in the population. Once the diversity is lost, the
crossover operator becomes ineffective in exploring the
search space. Although mutation can be used to introduce
diversity in the population, its effect is limited.
To cope with GAs' limitations, several researchers have
recently proposed a family of new algorithms
Estimation of Distribution Algorithms (EDA) [6][9]. Like
GAs, EDA work with a population of individuals. However,
one of the important features of EDA is that they avoid the
'blindness' of crossover by finding how the problem space
distributes and use this information to guide individuals to
explore better space areas during the search. One of the
algorithms that belong to the family of EDA is the so called
PopulationBased Incremental Learning (PBIL) which was
originally proposed by Baluja [7][8]. PBIL is a technique
that combines simple GAs with competitive learning. In
PBIL, the crossover operator of GAs is abstracted away and
the role of population is redefined. PBIL maintains a real
valued probabilityvector
generated. The probability vector controls the random
bitstrings generated by PBIL and is used to create other
individuals through learning. Learning in PBIL consists of
using the currentprobability distribution to create
individuals. These individuals are evaluated according to the
objective function. The best individual is used to update the
probability vector, increasing the probability of producing
solutions similar to the current best individuals.
is to
called
from which solutionsare
N
Many authors have shown the effectiveness of PBIL in
solving many difficult optimization problems [6][12]. One
of the key aspects of PBIL is the learning. The learning
process in PBIL ensures that individuals can adapt easily to a
new environment. In PBIL, the learning rate plays a crucial
role in finding the optimal solution of a problem. However,
the effect ofthe learning rate in PBIL is not well understood.
861
Authorized licensed use limited to: King Abdul Aziz City For Science & Technology. Downloaded on August 9, 2009 at 07:20 from IEEE Xplore. Restrictions apply.
Page 2
After the training, the weights are considered as prototype
vectors. The idea of creating a prototype vector is central to
PBIL as will be discussed in the next section.
Since the output units compete with one another to tum on,
they are called "winnertake all" units. During training, the
weights of the winning output unit are moved closer to the
presented point by adjusting the weights according to the
following rule [7]
(2)
(1)
Llw.. == LR x iinput,  w.. )
1)
r
1
1)
III.OVERVIEW OF POPULATIONBASED INCREMENTAL
LEARNING
PBIL is a technique that combines aspects of genetic
algorithms and simple competitive learning from ANN.
PBIL has the following features [9][12]:
• It has no crossover and fitness proportional operators.
• Instead of representing the entire genetic population
using myriads of chromosomes, the population is
represented through a probability vector (number in
range 01). The probability vector can be considered as
a prototype for high valuation vectors for the function
space being explored. This probability vector controls
the random bitstrings generated by PBIL and is used to
create other individuals through learning.
• In PBIL, there is no need to store all solutions in the
population. Only two solutions are stored: the current
best solution and the solution being evaluated.
feedforward network usually implements an
Hebbian learning rule. This consists of increasing the
influence of the input cell that persistently participates in
firing an output cell [13]. An example of supervised
competitive learning is Learning Vector Quantization (LVQ)
proposed in [14]. In learning vector quantization, it is
assumed that there is a set of reference vectors represented
by the number of weights wij connecting input i to output j.
These weights are initially chosen randomly by the user. The
activation of the output units is calculated by the following
formula [7]
excitatory
where:
LR: the learning rate parameter
1\: small variation
Excitatory
connections
Inhibitory
connections
Input s (1...4)
Fig. 1 A basic competitive learning network
The effect of the learning rate (LR) on the performance of a
PBIL used to design power system stabilizers (PSSs) for a
multimachine power system is investigated in this paper.
Four types of learning rates are investigated. These includes
the fixed learning rate where the learning rate is fixed and
does not change during the run; the purely adaptive learning
rate where the learning rate varies with the generations, and a
combination of fixed and adaptive learning rate where the
learning rate is fixed for the first portion of generations and
becomes adaptive for the other portion and viceversa. It is
shown that a smaller learning rate leads to more exploration
of the algorithm which introduces more diversity in the
population at the cost of slower convergence. On the other
hand, a higher learning rate means more exploitation of the
algorithm and hence,this could
convergence in the case of fixed learning rate. Therefore, a
tradeoff is needed between exploitation and exploration
when setting the learning rate.
lead to a premature
II. COMPETITIVE LEARNING IN ANN
Adaptation involves a progressive modification of some
structure or structures [1]. Without adaptation, no human or
animal species can survive. In PBIL, adaptation is provided
to the evolving chromosomes through competitive learning
[6]. Competitive learning (CL) is often studied in the context
of Artificial Neural Networks (ANNs) [7]. A common goal
in competitive learning is to distribute a certain number of
vectors in a possibly highdimensional space. Competitive
learning is often used to cluster a number of unlabeled data
into distinct groups. The objective is to group the data such
that the inputs in the same cluster are in some sense similar
[13]. A basic competitive learning network is shown in
Figure 1. It has one layer of input neurons and one layer of
output neurons. It consists of the feedforward excitatory
network(s) and the lateral inhibitory network(s). There are as
many output nodes as there are classes and each output node
represents a pattern category.
Outnuts (1 ... 1)
The inhibitory between output units ensure that only one
output is turned on at a time. The output unit that is turned
on is the one which has the largest net input. The
The three main operators of PBIL used in this paper are:
probability vector (PV), Learning rate (LR) and the mutation
(i.e., forgetting factor, FF).
862
Authorized licensed use limited to: King Abdul Aziz City For Science & Technology. Downloaded on August 9, 2009 at 07:20 from IEEE Xplore. Restrictions apply.
Page 3
Each probability value in the sequence represents the
probability that 1 or 0 can be generated at the gene position.
The learning rate is used in the updating rule of the
probability vector. It affects both the speed at which the
probability vector is shifted to resemble the best solution
vector and the portion of the search space that will be
explored. Like in GAs, the mutation is used to maintain the
diversity in the population. In [9][10], Baluja employed a
"mutation" operator similar to that used in GAs. In this
paper, a slightly different "mutation" operator is used. That
is, a forgetting factor is used to relax the probability vector
toward neutral value of0.5 [11][12].
A summary ofthe PBIL algorithm used in the paper is given
below [9][12], [15][16]:
Step 1. Initialize elements ofthe probability vector (PV) to
0.5 to ensure uniformlyrandom bitstrings.
Step 2. Generate a population (i.e., sample solutions) of
uniformlyrandombitstrings
elementbyelement with the PV. Wherever an
element ofthe PV is greater than the corresponding
random element, a "1' is generated, otherwise a '0'
is generated.
Step 3. Interpret each bitstring as a solution to the problem
and evaluate its merit in order to identify the "Best".
Step 4. Adjust PV by slightly increasing PV (i) to favor the
generation of bitstrings which resemble "Best", if
Best (i) = 1 and decrease PV(i) ifBest(i) = o.
Step 5. Perform mutation on the probability vector PV
Step 6. Generate a new population reflecting the modified
distribution. Stop if satisfactory solution is found.
Otherwise, go to step 2.
andcomparing it
This algorithm is much easier to implement than the
conventional GAs (which involve crossovers, mutations,
reproductions, etc.), and yet effective. It has been shown that
PBIL outperforms standard GAs approaches on a variety of
optimization problems including commonly used benchmark
problems [7][9]. Experience in executing GAs and PBIL
shows that the overhead for GA operations is significantly
higher than for PBIL [12].
IV. OPERATORSOFPBIL
A. Probability Vector (PV)
One important feature of GAs is their implicit parallelism,
i.e., the ability to search the function space from multiple
points in parallel [7]. However, as the search progresses, this
parallelism is not easily maintained in the latter generations
of GAs. Therefore, the idea behind PBIL is to represent the
entire genetic population through a probability vector rather
than a myriad of chromosomes. This probability vector
should be considered as a prototype for high evaluation
vectors for the function space being explored [7][8]. This
concept is central to PBIL. Each probability value in the
sequence represents the probability that a 1 or 0 can be
generated at the gene position. For example, the probability
vector [0.5, 0.5, 1, 1] can be represented by the following
population of4 bits
0,0,1,1
0, 1, 1, 1
1,0, 1, 1
1, 1, 1, 1
Note that the size ofthe above population is 4.
The probability that a 1 or 0 will be generated in the first
two positions is equal (i.e., 50/50). The probability of
generating 1 in the 3rd and 4thpositions is 1.
Unlikethe mechanisms
operations are defined on the population; in BPIL, the
operations take place directly on the probability vector.
During the search, the values in the probability vector are
updated to represent those in high evaluation vectors. It
should also be noted that besides from specifying the
prototype based upon the high evaluations of the sample
solutions, the probability vector also guides the search,
which produces the next sample point from which learning
take place [7].
Initially, the values of the probability vector are set to
0.5 to ensure that the probability of generating 0 or 1 is
equal. As the search progresses, the values in the probability
vector move away from 0.5, towards either 0.0 or 1.0.
It has been argued that because PBIL uses a single
probability vector, it may be less powerful than GAs because
a largenumber
ofpoints
simultaneously. However, this argument is only true at the
beginning of the search space. Because of the sampling
errors, the population will converge to one point at the latter
portion of the search and GA will not be able to maintain
multiple dissimilar points [7][8].
inherentto GAs,where
cannotbe represented
B. Learning Rate (LR)
The probability update rule is similar to the weight update
rule in a competitive learning of ANN as given in (2). The
following probability update rule based on the competitive
learning is used:
PV(i) ==PV(i)x(l.OLR)+(LRxV(i))
where
PV(i): the probability ofgenerating 1 in bit position i.
V(i): the ith position in the solution vector towards which
the probability vector is moved.
(3)
The learning rate has a greater effect in PBIL as compared to
the standard competitive learning. This is because the
probability vector is used
solutions. Like in competitive learning, the learning rate
affects the speed at which the probability vector is shifted to
resemble the best solution vector. It also affects the portion
to generatefuturesample
863
Authorized licensed use limited to: King Abdul Aziz City For Science & Technology. Downloaded on August 9, 2009 at 07:20 from IEEE Xplore. Restrictions apply.
Page 4
of the search space that will be explored. Therefore, it has a
direct impact on the tradeoff between exploration (i.e., the
ability of the algorithm to search the function
thoroughly) and exploitation (i.e., the ability ofthe algorithm
to use the information it has gained about the function space
to narrow its future search) ofthe search space [7].
space
As shown later in the simulations, a small learning rate
means less exploitation of the information gained through
previous search and more exploration of the search space to
search for diverse and possibly better solutions. As the
learning rate is increased, the amount of exploitation
increases, and the ability to search large portions of the
search space diminishes. That is, the exploration capability
becomes less.
Two types of learning are used in PBIL algorithm[7][8].
The positive learning which was discussed previously, and
the negative learning. For the negative learning, the
probability vector is shifted away from the worst vector. In
this paper, only positive learning is used.
C. Mutation Operator: Forgettingfactor (FF)
To maintain diversity in PBIL, a "mutation" operator
similar to that used in GAs was employed by Baluja [7][8].
In this paper, a slightly different "mutation" operator is used.
That is, a forgetting factor is used to relax the probability
vector toward neutral value of 0.5 [9][11]. There are two
methods that can be used to perform mutation. The first
method is to perform mutation directly on the sample vectors
generated (i.e., population). The second method is to
perform mutation on the probability vector. The PBIL used
in this paper adopted the latter method. The formula used for
the implementation ofthe mutation is given by
PV(i) ==PV(i)FFx(PV(i)O.5)
(4)
where the state variables are x, the system output is y and
the signal u represents the control input. Ao,Bo, Co, Do are
constant matrices ofappropriate dimensions.
L
L. 4
Fig. 2 Power system configuration
Several operating conditions have been considered during
the design of the PSSs, this includes, the nominal operating
condition, light and heavy load conditions. In addition, the
load conditions mentioned above are considered under weak
transmission system. Note that weak transmission system in
our contest means that the nominal transmission line data are
doubled. In this paper, the PSSs are designed using fives
operating conditions. The nominal operating condition (case
1), the light load condition under nominal transmission line
system (case 2), the heavy load condition under nominal
transmission line system (case 3), the light load condition
under weak transmission line (case 4) and the heavy load
condition under weak transmission system (case 5).
These cases are listed in Table I (generation) and Table II
(loads). The eigenvalues of the openloop nominal system
(without PSS) are listed in Table III.
(5)
where
FF: the forgetting factor
V.
SYSTEM MODEL AND OPERATING CONDITIONS
The power system model considered in this paper is a
threemachine ninebus power system as shown in Fig. 2
[15]. Each machine is represented by the twoaxis model
(fourth order). The machines are equipped with a simple
AVR, which is modeled by a second order differential
equation [11]. The dynamics ofthe system are described by a
set of nonlinear differential equations. However, for the
purpose of controller design these equations are linearized
around the nominal operating conditions.
The linearized equation ofthe system is given by [2][3]
X == Aox+Bou
y==Cox+Dou
Gen.
No.
Case
1
2
3
4
5
TABLE I
POSSIBLE OPERATING CONDITIONS
GlG2 G3
r,
Qe
r,
Qe
r,
Qe
0.7160.321 1.630 0.001
0.850
0.118
0.505
0.0051.100 0.241
0.300
0.307
2.115
0.523
0.877
0.674
1.900
1.100
0.392
0.637
1.240
0.300
0.281
0.523
2.193
0.5001.900
0.123 1.2400.086
All the values are given in perunit
864
Authorized licensed use limited to: King Abdul Aziz City For Science & Technology. Downloaded on August 9, 2009 at 07:20 from IEEE Xplore. Restrictions apply.
Page 5
TABLE II
LOAD PARAMETERS
Load.
Case
ABC
PL
QL
PL
QL
PL
QL
I
1.250
0.500 0.900 0.300
1.000
0.350
2&4
0.750 0.300
0.540 0.180
0.600
0.210
3&5
1.750 0700
1.6200.540
1.8000.630
All thevalues aregiven inperunit.
TABLE III
OPENLOOP EIGENVALUES (WITHOUT PSS)
Case
A
S(%)
3.5
3.2
10.4
12.8
2.9
0.4
4.2
2.8
0.6
4.3
0.345±j9.787
0.440:tj13.822
1.006±j9.620
1.786±j13.856
0.277±j9.544
0.053±j13.621
0.368±j8.691
0.355±j12.523
0.053±j8.317
+0.053±j12.329
2
3
4
5
It can be seen from Table III that the system has two
electromechanical oscillation modes [16][17]. It should be
mentioned that theseoscillation modes change as the
operating conditions are varied. The effects of learning rate
on the performance of the PBILPSSs eigenvalues for the
nominal operating condition are illustrated in the simulation
results section.
VI.
PROBLEM FORMULATION
The objective ofthe study is to optimize the parameters of
the PSSs such that controllers simultaneously stabilize the
family of the power system models as described previously.
It was found that a double stage leadlag network with time
constants T1T4and gain Kpis sufficient to provide adequate
damping to the multimachine system shown in Fig.2 [11].
The speed input PBILPSS is ofthe form given in (6)
where, Kp is the gain, T1T4 represent suitable time
constants. T; is the washout time constant needed to prevent
steadystate offset ofthe voltage.
Since the electromechanical modes are generally poorly
damped and dominate the time response of the system, it is
expected that by maximizing the minimum damping ratio
over the entire family of the system models, the closedloop
systems could be simultaneously stabilized over a wide range
of operating conditions [11],[15]. The following objective
function is used in PBIL to achieve the above requirements:
(7)
where i= 1,2 ... n, andj= 1,2, ... m
a..
and Si j" = ~
,
a .. +m..
l,j
l,J
is the damping ratio ofthe ith
22
l,j
eigenvalue in the jth operating condition. Gij is the real part
of the eigenvalue and the C4j is the frequency. n denotes the
total number eigenvalues and m denotes the number of
operating conditions.
VII. PSS DESIGN FOR AMULTIMACHINE POWER
SYSTEM
There are in total 15 PSS parameters (five for each
generator) that need to be optimized. It should be noted that
the reset time constant T; shown in (2) was not considered in
the optimization process. This is because T; is not critical.
Its value was fixed to 10 sec.
The configuration ofthe PBIL is as follows:
•
•
•
•
•
Length ofchromosome: 15 bits
Population: 10
Generations: 200
Learning rate (LR): 0.1 (default)
Forgetting factor (mutation): 0.005
The parameter domain for the PBILPSS was set to:
o~
o<T1, T 3 ~
0.010 <T2, T4<0.5
Kp~ 20
1
A modified MATLAB software [9], [11] was used for the
design.
The challenge in PSS design problem (as opposed to
other problems) is to guaranty adequate stability of the
closedloop system despite uncertainties in the
(parameter variations, load change, faults, etc). This task is
complicated by the fact that power systems are nonlinear,
complex, and highly multivariable. Therefore, the search
space is typically multimodal and uncertain. There is a
potentialdanger that the algorithm will
suboptimal solutions. The controller is designed using a
linear model but it expected to give good performance under
nonlinear conditions.
system
bestuck at
VIII. SIMULATION RESULTS
A. Test cases
Fixed LR: The learning rates considered are: LR=O.Ol, LR
=0.0125, LR = 0.167, LR=0.05, LR= 0.1 (default), LR = 0.2,
LR = 0.4, LR = 0.6, LR=0.8 and LR =1.
865
Authorized licensed use limited to: King Abdul Aziz City For Science & Technology. Downloaded on August 9, 2009 at 07:20 from IEEE Xplore. Restrictions apply.
Page 6
Purely Adaptive LR: The simulation is started with a very
small value of LR0), and this value is increased
linearly with the generation until the final value of LR is
reached. For example, ifthe final value learning rate is LR =
0.2, the run is started with LR
LR = 0.05, after 100 generations, LR = 0.1 and so on.
( L R ~
~
0, and after 50 generation,
Combined Fixed LR and Adaptive LR (Adaptive to Fixed
LR and Fixed to Adaptive LR) :
• Adaptive to Fixed LR
For the first 100 generations, adaptive learning rate
is used until LR = 0.1 and for the second halfofthe
generations (100 generations), fixed learning rate of
LR =0.1 is used.
• Fixed to Adaptive LR:
In the first halfofthe generations (100 generations),
a fixed learning rate of LR = 0.1 is used and an
adaptive learning rate is used for the second half of
the generation.
B. Convergence rate
Figs. 36 show the convergence rate of the average of the
best fitness values over 50 trials for different learning rates.
Fig. 3 shows the average of fitness values over 50 trials,
for selected fixed learning rates (LR= 0.01, LR = 0.1, LR =
0.2, and LR = 0.8, LR = 1). It can be seen that LR = 1 gives
consistentlythelowest fitness
generations. This is because of the premature convergence
problem. By setting a high value of the learning rate, the
ability of the algorithm to explore the search space is
reduced. As a result, the algorithm converges to a local
maximum instead of a global one. Setting LR to be small
(i.e., LR = 0.01) means that we put much weight on
exploration than exploitation. The algorithm will need more
time to explore the search space and it will take longer time
to converge to a good solution and more generations than the
200 generations used here. From our experience, we have
observed that the larger the learning rate the quicker the
convergence (i.e., less time is used) and the higher the
possibility to converge to local maxima because of the loss
of this diversity in the population. It can be said that
diversity in the population is maintained longer if the
learning rate is sufficiently small. Diversity is very critical
for adaptation. If there is no diversity, natural selection
cannot takeplace. Onthe
exploration of the search space when the optimal solution
has already been found will waste valuable time. What is
needed is a tradeoff between exploitation and exploration.
The results in Figure 3 show that LR = 0.1 provides the best
tradeofffollowed by LR = 0.2.
valueoverthe 200
other hand, unnecessary
Fig. 4 shows the average offitness value over 50 trials, for
adaptive learning rates (LR= 0.01, LR = 0.1, LR = 0.2, and
LR = 0.8, LR = 1). It can be seen that LR = 1 is now among
the best three learning rate to provide a good average fitness
value. In fact, the fitness value of LR = 1.0 was the best for
the first 80 generations. The reason why LR = 1.0 for
adaptive LR is better than fixed LR is because of the
adaption that was introduced. In fact the run was started with
smaller learning rate, which mean the algorithm has time to
explore the search space before exploiting it at the latter
stage. As a result, this produces a better solution as
compared to the fixed learning rate. On the other hand, LR =
0.01, consistently gives the worse fitness value starting from
50 generations up to 200 generations. In both the fixed and
adaptive cases, LR = 0.01 did not perform well. The main
reason for this is because there was virtually no exploitation
with this learning rate and the algorithm was busy exploring
the search space before the run was stopped. It can be seen
that the final best fitness value comes from LR = 0.6. The
second best is LR = 0.2. LR = 0.2 produces the fastest
increase in fitness during the run. It seems from Fig. 4, that if
we were to increase the number of generations, LR = 0.2
may eventually outperform LR = 0.6.
It can be seen from Fig. 5 (Adaptive to fixed LR ) that for
the first 100 generations, when the learning rate is adaptive,
the fitness values are much more spread. This suggests that
the different learning rates were exploring different locations
of the search space. LR =1.0, consistently give the worst
solution until about 150 generations, when it starts to
improve. Starting from about 25 generations, LR= 0.2 gives
the best fitness value and for the rest ofthe run.
Fig. 6 shows the average fitness curves when the learning
rate was fixed to LR = 0.1 for the first 100 generations and
then changes to adaptive in the second halfofthe generation.
It can be seen that in contrast to Fig. 5 the fitness values for
all the learning rate are very similar from the beginning until
the end of the run. This suggests a lack of diversity in the
population.
C. Fitness value
Tables IV and V show the learning rate and the average
of the best fitness values for each learning rate under the
various test cases.
It can be seen from Table IV that the fixed learning rate
(column 2 of Table IV), LR = 0.1 gives the 'best' average
value of the fitness, the second best is LR = 0.2. The worst
value comes from LR = 1, followed by LR = 0.01. The worst
performance for LR=larises because of a premature
convergence problem. Diversity is lost earlier in the run. For
LR= 0.01, the algorithm was still exploring the search space
when it was stopped. So, it could not found a good solution.
866
Authorized licensed use limited to: King Abdul Aziz City For Science & Technology. Downloaded on August 9, 2009 at 07:20 from IEEE Xplore. Restrictions apply.
Page 7
TABLE IV
BESTFITNESS VALVES OVER50 TRIALS (FIXED& ADAPTIVE)
LR
Savgbest (%)
 Fixed
0.0100
0.1071
0.0125 0.1095
0.0167 0.1151
0.02500.1274
0.0500
0.1489
0.1000 0.1652
0.20000.1626
0.4000 0.1536
0.60000.1247
0.8000
0.1114
1.00000.1007
Savgbest (%) 
Adaptive
0.0923
0.0920
0.0967
0.1069
0.1258
0.1443
0.1635
0.1659
0.1691
0.1637
0.1569
TABLEv
LEARNINGRATE ANDBESTFITNESSVALVES OVER 50 TRIALS
LR
Savgbest (%) 
AdaptiveFixed
0.20000.1641
D.4000
0.1447
0.60000.1430
0.8000
0.1304
1.00000.1339
Savgbest (%) 
FixedAdaptive
0.1595
0.1528
0.1531
0.1536
0.1499
Fixed LR
0.25
0 .2
~
0 .1 5
j
j
."
f
0 . 1
L R = 0 .01
L R = 0 . 1
 L R = 0 .2
L R = 0 .8
 L R = 1.0
0. 1 r
0 20 4 06080 10 0 120 14 0 16 018 0 200
Generation
Fig. 3 Fixed learning rate
A daptive LR
0 .25 
I
~
""
j
~
0.15
0 . 1
11
j
 L R = O.0 1
L R = 0 . 1
 L R = 0 .2
L R = 0.6
 L R = 1.0
0. 10
20 4 0
60 8010 0 120 140 16 018 0200
Generation
Fig. 4 Adaptive Icaming rate
The best overall fitness value for the four test cases is
provided by the adaptive learning rate of LR = 0.6 (0.1691)
and the second best by LR = 0.4 (0.1659). The third best was
provided by the fixed learning rate of LR =0.1 (0.1652).
This problem could be solved by increasing the number of
generations.
For the adaptive learning rate (column 3 of Table IV),
LR= 0.6 gives the 'best' average value of the fitness, the
second best is LR = 0.4. The worst value comes from LR =
0.01, followed by LR = 0.0125. Again here, the worst
performance for LR = 0.01 arise because the algorithm is
still exploring the search space and has not yet converged.
Compared to the fixed learning rate, the adaptive scheme
performs poorly for LR < 0.2. However, for LR>0.2, the
adaptive scheme outperforms the fixed scheme. In particular,
for LR =1, which gave the worse fitness value for the fixed
learning rate. Furthermore, all the fitness values for LR>0.2
all bigger than 0.15. This suggests that in the case of
adaptive learning rate, premature convergence did not occur,
this is because adaptation was included. Therefore, the
algorithm was able to explore the search space at the
beginning of the run before exploiting it at the later stage. As
a result, this leads to a better solution eompared to the fixed
learning rate. The results suggest that for low learning rate
(i.e., LR<0.2) it is better to use fixed learning rate. However,
for higher learning rates, adaptive learning rate is preferred.
The best fitness values are obtained for LR between 0.2 and
0.8.
Table V shows the learning rate and the average of the
best fitness values over 50 trials when both fixed and
adaptive learning rate are combined.
Column 2 of Table V shows the fitness values when
adaptive learning rate is first used at the start ofthe run until
100 generations, after this a fixed learning rate (LR = 0.1) is
used. It ean be seen that the best average fitness value is
obtained when LR = 0.2. As the learning rate is increasing
from LR = 0.2 to LR = 0.8, the fitness value deereases, and
increases again slightly at LR =1.0. Compared to the case
where only fixed learning rate is used, the fitness values for
this case are slightly higher. Compared to the case where
purely adaptive learning rate is used, most of the fitness
values are smaller except for LR =0.2.
Column 3 ofthe Table V shows the fitness values when fixed
learning rate of LR = 0.1 is first used at the start of the run
until 100 generations, and then it was switehed to adaptive
learning rate. It can be seen that all of the fitness values are
higher than the case where adaptive learning rate is first used
and then it was switched to fixed learning rate (exeept for LR
= 0.2).
867
Authorized licensed use limited to: King Abdul Aziz City For Science & Technology. Downloaded on August 9, 2009 at 07:20 from IEEE Xplore. Restrictions apply.
Page 8
0 .2
VII. CONCLUSION
The effect of the learning rate on the performance of
PBILalgorithm has been investigated.
results clearly show that the learning rate has a significant
impact on the performance of the algorithm. When the
learning rate was set to very high values (i.e., LR = 081) for
fixed learning rate, diversity was lost in the population and
the algorithmconvergesprematurely in less than 80
generations. It was found that for the same high values, the
adaptive learning rate performs better. Smaller learning rates
lead to more exploration of the algorithm and hence this
introduces more diversity in the population than high
learning rates. However, it takes longer for the algorithms to
converge. Therefore, a tradeoff between exploitation and
exploration is needed to obtain the desired results within a
reasonabletime. Atpresent,
straightforward. It requires the designer to experiment on
several values of the learning rate before choosing a suitable
one. The simulations with the adaptive learning rate yields
more or less consistent fitness of higher values for LR
greater than 0.2. Test cases 3 and 4 with combinations of
adaptive and fixed learning rates show that setting the
learning rate to be initially adaptive introduces diversity.
This is not case when the learning rate was initially fixed
before switching to adaptive learning rate.
The simulations
thistradeoffis not
However, these results are true for our particular problem
of power system stabilizer design where the search space is
[15] KA Folly, "Robust Controller Based on a Combination ofGenetic
Algorithms and Competitive Learning", IJCNN 2007, Orlando,
Florida, USA, No. I793.ISBN: 142441380X, August 1217,
2007.
REFERENCES
[I] Y.L. Abdel M. A. Abido, and a. H. Mantawy, Simultaneous
Stabilization ofMultimachine
Algorithms, IEEE Trans on Power Systems. 14 (4), 1999, 1428
1438.
K. Sundareswaran and S. Razia Begum, Genetic Tuning ofa Power
System Stabilizer, Euro. Trans. Electr 14 , 2004, 151160.
J. H., Holland, Adaptation in nature and artificial systems. The
University ofMichigan Press, 1975.
M. Mitchell, An Introduction to genetic Algorithms (The MIT Press
1996).
Z. Michalewicz, Genetic Algorithms+Data Structure = Evolution
Programs (3'd Ed., SpringerVelag, 1996).
C. Gonz'alez J.A. Lozano P. Larranaga, The convergence behavior of
PHIL algorithm: a preliminary approach, Intelligent Systems Group
Computer Science and Artificial Intelligence. The University ofthe
BasqueCounty,
http://www.sc.ehu.es/ccwbayes/postscript/kzaaik0399.ps
S. Baluja, PopulationBased Incremental Learning: A method for
Integrating Genetic Search Based Function
Competitive Learning. Technical Report CMUCS94163, Carnegie
Mellon University, 1994.
S. Baluja and R. Caruana, "Removing the Genetics from the
Standard Genetic Algorithm", Technical Report CMUCS95141,
Carnegie Mellon University, 1995.
J. R. Greene. Simulated and Adaptive Search in Engineering Design
Experience at the University of Cape Town, Invited key paper at
world Conference on soft Computing WSC3, Springer Verlag, 1997.
K.A. Folly, Stability Enhancement of Power Systems Using a PBlL
Based Power System Stabilizer, 6thAfricon Conferencein Africa,
2002,953956
K. A. Folly, "Design of Power System: A Comparison Between
Genetic Algorithms(GAs) and
Learning (PBIL)", to be presented at IEEE. PES 2006. General
Meeting, Canada.
F. Southey and K Karray, "Approaching Evolutionary Robotics
Through PopulationBased Incremental Learning, in Proc. of the
1999 International Conference on Systems. Man. and Cybernetics
(SMC'99).
D. Andina and I. A. Vega, Artificial Neural Network, available at
http://www.gc.ssr.upm.es/inves/neural/annl /anntutor.htm
Power Systems viaGenetic
[2]
[3]
[4]
[5]
[6]
availableat
[7]
Optimization and
[8]
[9]
[10]
[II]
PopulationBasedIncremental
[12]
[13]
[14]T. Kohonen, Improved version oflearning vector quantization, Proc.
ofthe International Joint Con! on Neural Networks, 1990 vol I,
545550.
multimodal. In order to generalize this statement, more
work still needs to be done on wide range ofproblems. Also
it is necessary to investigate the effect of generation and
population on the learning rate. These issues will be looked
at in the future.
80 10 0 120 140 16 018 02 00
Genera l io n
•rr
8010 0120 14 0
Generatio n
F ixed to a d a pt i..e L R
60
60
Fig.5 Adaptive to fixed learning rate
Fig. 6 Fixed to adaptive learning rate
20 4 0
204 0
0 .1 
0 .2 5
0 .2
'!ig;
11
oE
j
~
0 . 1 5
0 . 1
0 .05
"5j
0
0.05
0 . 1
~
0
!!l
~
0 . 1 5
j
868
Authorized licensed use limited to: King Abdul Aziz City For Science & Technology. Downloaded on August 9, 2009 at 07:20 from IEEE Xplore. Restrictions apply.
View other sources
Hide other sources
 Available from Komla Folly · May 21, 2014
 Available from edu.sa