A performance comparison of multi-objective optimization algorithms for solving nearly-zero-energy-building design problems

Abstract

Integrated building design is inherently a multi-objective optimization problem where two or more conflicting objectives must be minimized and/or maximized concurrently. Many multi-objective optimization algorithms have been developed; however few of them are tested in solving building design problems. This paper compares performance of seven commonly-used multi-objective evolutionary optimization algorithms in solving the design problem of a nearly zero energy building (nZEB) where more than 1.610 solutions would be possible. The compared algorithms include a controlled non-dominated sorting genetic algorithm with a passive archive (pNSGA-II), a multi-objective particle swarm optimization (MOPSO), a two-phase optimization using the genetic algorithm (PR-GA), an elitist non-dominated sorting evolution strategy (ENSES), a multi-objective evolutionary algorithm based on the concept of epsilon dominance (evMOGA), a multi-objective differential evolution algorithm (spMODE-II), and a multi-objective dragonfly algorithm (MODA). Several criteria was used to compare performance of these algorithms. In most cases, the quality of the obtained solutions was improved when the number of generations was increased. The optimization results of running each algorithm 20 times with gradually increasing number of evaluations indicated that the PR-GA algorithm had a high repeatability to explore a large area of the solution-space and achieved close-to-optimal solutions with a good diversity, followed by the pNSGA-II, evMOGA and spMODE-II. Uncompetitive results were achieved by the ENSES, MOPSO and MODA in most running cases. The study also found that 1400-1800 were minimum required number of evaluations to stabilize optimization results of the building energy model.

Full text

© 2016, the authors. This manuscript version is made available under the CC-BY-NC-ND 4.0 license
http://creativecommons.org/licenses/by-nc-nd/4.0/
Please site this article as: Hamdy, M., Nguyen, A.T. and Hensen, J.L., 2016. A performance comparison of multi-objective optimization
algorithms for solving nearly-zero-energy-building design problems. Energy and Buildings, 121, pp.57-71
Status: Accepted manuscript
Official publication can be found at: http://dx.doi.org/10.1016/j.enbuild.2016.03.035
A performance comparison of Multi-objective optimization algorithms for
solving nearly-zero-energy-building design problems
Mohamed Hamdy1, 2, 3, Anh-Tuan Nguyen4, Jan L. M. Hensen1
1 Department of the Built Environment, Building Physics and Services, Eindhoven University of
Technology, P.O. Box 513, 5600 MB Eindhoven, The Netherlands.
2 Department of Mechanical Power Engineering, Helwan University, P.O. Box 11718 Cairo, Egypt.
3 Department of Energy Technology, Aalto University School of Engineering, P.O. Box 14400, FI-
00076 Aalto, Finland.
4 Faculty of Architecture, Danang University of Science and Technology, 54 Nguyen Luong Bang,
Lien Chieu, Danang, Vietnam.
ABSTRACT
Integrated building design is inherently a multi-objective optimization problem where two or more
conflicting objectives must be minimized (e.g., primary energy) and/or maximized (e.g., cost-
effectiveness and thermal-comfort) concurrently. Recently, many multi-objective optimization
algorithms have been developed; however few of them are tested in solving building design cases.
This paper compares performance of seven commonly-used multi-objective evolutionary
optimization algorithms in solving a large-scale cost-optimality nearly zero energy building (nZEB)
design problem where more than 1.610 solutions would be possible. The compared algorithms include
a controlled non-dominated sorting genetic algorithm with a passive archive (pNSGA-II), a multi-
objective particle swarm optimization (MOPSO), a two-phase optimization using the genetic
algorithm (PR_GA), an elitist non-dominated sorting evolution strategy (ENSES), a multi-objective
evolutionary algorithm based on the concept of epsilon dominance (evMOGA), a multi-objective
differential evolution algorithm (spMODE-II), and a multi-objective dragonfly algortihm (MODA.
The performance is compared in terms of convergence to the Pareto front, repeatability, computational
time, number of trade-off solutions, and their contributions to the best obtained solutions.
The optimization results of running each algorithm 20 times with gradually increasing number of
evaluations indicated that the PR_GA algorithm had a high repeatability to explore a large area of the
solution-space and achieved close-to-optimal solutions with a good diversity. Uncompetitive results
were achieved by the ENSES, MOPSO and MODA in most running cases.
Keywords: multi-objective optimization; evolutionary optimization algorithms; simulation-
based optimization approach; building energy analysis; comparison

2/33
Contents
ABSTRACT 1
Contents 2
1. Background and objectives of the study 4
2. An overview on the performance of optimization algorithms in building energy analysis
4
2.1. Performance comparison of single objective optimization algorithms 5
2.2. Performance comparison of MOOAs 6
3. Research methodology 7
3.1. Test algorithms 7
3.2. Test models 12
3.3. Objectives to be minimized 14
3.4. Criteria for performance comparison 15
4. Performance of the optimization algorithms 19
4.1. Convergence to the best Pareto front and diversity of obtained solutions 20
4.2. Execution time 23
4.3. Number of solutions on the Pareto-optimal set 24
4.4. Performance ranking and some observations 25
5. Discussion and conclusion 27
References 28

3/33
NOMENCLATURE AND ABREVIATIONS
BOP
Building optimization problem
DHW
Domestic hot water
DMn
Diversity Metric
e
Escalation rate of energy price [%]
ENSES
Elitist Non-dominated Sorting Evolution Strategy
EPBD
Energy performance of buildings directive
evMOGA
Multi-objective evolutionary algorithm based on the concept of epsilon dominance
FIT
Feed-in tariff [€/kWh]
GA
Genetic algorithm
GDn
Generational Distance
HVAC
Heating, ventilation and air-conditioning
IGDn
Normalized Inversed Generational Distance
LCC
Life cycle cost
MODA
Multi-objective dragonfly algorithm
MOEA
Multi-objective evolutionary algorithm
MOOA
Multi-objective optimization algorithm
MOPSO
Multi-objective particle swarm optimization
NoPsolution
Number of solutions on the Pareto-optimal set
NSGA
Non-dominated sorting genetic algorithm
nZEB
nearly zero energy building
PEC
Primary energy consumption
pNSGA-II
Non-dominated sorting genetic algorithm-II with a passive archive
PR_GA
Two-phase optimization using the genetic algorithm
PSO
Particle swarm optimization
spMODE-II
Multi-objective differential evolution algorithm

4/33
1. Background and objectives of the study
Today, simulation-based optimization has become an efficient design approach to
satisfy several stringent requirements in designing high performance buildings (e.g. low-
energy buildings, passive houses, green buildings, net zero-energy buildings, zero-carbon
buildings…) [1]. In real-world building design problems designers often have to deal with
conflict design criteria simultaneously such as minimum energy consumption versus
maximum thermal comfort, minimum energy consumption versus minimum construction
cost… Multi-objective optimization is therefore, in many cases, more relevant than the single-
objective approach. This has led to the application of multi-objective optimization algorithms
(MOOAs) that identify the Pareto optimum trade-off between conflicting design objectives
(e.g. [2, 3, 4, 5, 6]).
It is found that efficient MOOAs are essential to find the optimal solutions without a
need for numerous time consuming simulations. However, performance of MOOAs on
building optimization problems has not been well understood due to the lack of research-
based evidences. This study is among the first efforts that comprehensively investigate
performance of seven evolutionary optimization algorithms in solving multi-objective
optimization problems by using the simulation-based optimization approach. The test
problem is a building energy model, which has a discrete solution space of energy saving
measures and energy supply systems options, including renewable energy sources (RES). The
major aims of this study are:
- To compare performance of different evolutionary algorithms in optimizing
building energy models;
- To understand behavior of these evolutionary algorithms in solving a multi-
objective optimization problem.
In this study, we provide an overview on performance comparison of optimization
algorithms in building energy analysis. The investigated MOOAs and their performance
criteria are described in the next section, which is followed by a description of the nZEB
optimization problem (that is in accordance with the implementation of the new EPBD-2010).
The results of this study were reported to support the performance criteria.
2. An overview on the performance of optimization algorithms in building energy
analysis
Due to the large amount of design variables of building energy models as well as their
discrete, non-linear, and highly constrained characteristics, simulation results are generally
multi-modal and discontinuous, generating discontinuities or noise in the objective functions
in building optimization problems (BOPs). As a result, optimization algorithms that require
smoothness were found not efficient [7, 8]. In many cases, stochastic population-based
MOOAs (evolutionary optimization, swarm intelligence…) that do not require smoothness

5/33
are able to handle the discontinuity of the search space. It is clear that performances of
different algorithms are not equal. An algorithm may perform well by one criterion but fails
by other criteria. Thus, performance of the optimization algorithms in solving BOPs is really
an attractive research question that needs to be investigated.
2.1. Performance comparison of single objective optimization algorithms
The performance is often considered the major criterion for selecting an optimization
algorithm. There have been several studies on the performance of different optimization
algorithms on single-objective BOPs. Wetter and Wright [9] compared the performance of a
Hooke-Jeeves algorithm and a GA in optimizing building energy consumption. Their result
indicated that the GA outperformed the Hooke-Jeeves algorithm and the latter was attracted
by a local minimum. Wetter and Wright [7] compared the performance of eight algorithms
(Coordinate search algorithm, Hooke-Jeeves algorithm, particle swarm optimization (PSO),
PSO that searches on a mesh, hybrid PSO-Hooke-Jeeves algorithm, simple GA, simplex
algorithm of Nelder and Mead, discrete Armijo gradient algorithm) in solving simple and
complex building models using a low number of cost function evaluations. Performance
criteria include number of iterations, and optimal objective values. They found that the GA
consistently got close to the best minimum and the Hybrid algorithm achieved the overall best
cost reductions (although with a higher number of simulations than the simple GA). When
the discontinuities in the cost function were small, the Hooke–Jeeves algorithm achieved good
performance with a small number of iterations. Performances of other algorithms were not
stable and the use of simplex algorithm and discrete Armijo gradient algorithm were not
recommended. In GenOpt [10], Wetter introduced an improved hybrid algorithm PSO -
Hooke-Jeeves in which the PSO performs the search on a mesh, significantly reducing the
number of function evaluations called by the algorithm. Kampf et al. [11] examined the
performance of two hybrid algorithms (PSO - Hooke-Jeeves and CMA-ES/HDE) in
optimizing five standard benchmark functions (Ackley, Rastrigin, Rosenbrock, sphere
functions and a highly-constrained function) and real-world problems using EnergyPlus
simulation program. The results indicated that the CMA-ES/HDE performed better than the
PSO - Hooke-Jeeves in solving the benchmark functions with 10 dimensions or less.
However, if the number of dimensions is larger than 10, the hybrid PSO - Hooke-Jeeves gave
better solutions. Both these algorithms performed well with the real-world BOPs using
EnergyPlus models. Lee et al. [12] compared the performance of the differential evolution
algorithm with that of a PSO, a GA and the Lagrangian method by solving the optimal chiller
loading problem for reducing energy consumption. They found that the proposed differential
evolution algorithm could give similar results as the PSO did, but obtained better average
solutions. The differential evolution algorithm outperformed the GA in finding optimal
solutions and also overcame the divergence problem caused by the Lagrangian method
occurring at low demands.

6/33
Tuhus-Dubrow and Krarti [13] examined the performance of a GA against the PSO
and the sequential search method in building envelope design to minimize lifecycle cost. The
result reveals that the GA was more efficient than the two remaining methods in the complex
cases (more than 10optimization parameters were included in the building model). Moreover,
the GA optimization method could define the optimal solution with an accuracy of 0.5%, and
required only a half of the number of iterations needed by the PSO and the sequential search
method.
Bichiou and Krarti [14] compared a GA with the PSO and the sequential search in
optimizing building envelops and HVAC system design, in terms of computational time and
cost reduction. They found that the GA and the PSO required typically less computational
time to obtain optimal solutions than the sequential search. The optimal results given by the
three algorithms were almost similar.
From these studies, it can be seen that the stochastic population-based optimization
algorithms (e.g. evolutionary algorithms, swarm intelligence…) generally outperform the
others and are likely suitable for BOPs. However, it is worthy of note that the cost reduction
by an algorithm not only depends on the natures of the algorithm, but also depends on the
settings of algorithm parameters [7, 11]. According to the so-called ‘no free lunch theorem’
[15], there is no single best algorithm for all optimization problems. Hence, algorithm
selection and settings might involve trial and error [1].
2.2. Performance comparison of MOOAs
In contrary to the case of single-objective optimization algorithm, performance of
MOOAs on BOPs has not been well understood due to the lack of research-based evidences.
Nassif et al. [16] compared the NSGA and the NSGA-II in solving simplified variable air
volume HVAC design problems. The comparison criteria includes the distance to the true
Pareto front and the spread of optimal solutions. They went to conclusion that the NSGA-II
performed better than the NSGA, both in terms of distance and spread.
Hofpe [17] compared performance of two MOOAs: the NSGA-II and the SMS
EMOA, by comparing their solutions on the Pareto fronts. She found that the performance of
the NSGA-II was not satisfied in all tests, while the SMS EMOA gave more competitive
results but the number of simulations was much higher. The failure of the NSGA-II was still
unknown.
Brownlee et al. [18] compared the performance of five multi-objective algorithms
(IBEA, MOCell, NSGA-II, SPEA2 and PAES) along with the random search method in
solving a multi-objective problem concerning window placement. The assessment criteria
included the size of hypervolume and the spread established by the Pareto front found after
5000 evaluations. They found that the NSGA-II outperformed the others, both in terms of the
size of hypervolume and the spread of optimal solutions.
Hamdy et al. [2] evaluated performance of three multi-objective algorithms, including
the NSGA-II, aNSGA-II (NSGA-II with active archive) and pNSGA-II (NSGA-II with a

7/33
passive archive strategy), on a BOP and 2 benchmark test problems. They reported that the
aNSGA-II has a better repeatability in finding high-quality solutions close to the true Pareto
front with fewer evaluations and high convergence than the original NSGA-II and pNSGA-
II.
Hamdy et al. [19] proposed two new approaches in HVAC design optimization by
using combination of different algorithms. They compared performance of the original
genetic algorithm implemented in Matlab with the “so called” PR_GA (Running the Fmincon
solver and Fminimax function in Matlab optimization toolbox to prepare a good initial
population for the first generation of the GA) and the GA_RF (a combination between the
genetic algorithm in Matlab and the sequential quadratic programming (SQP) method to
improve or refine some of GA Pareto points as well as to enhance the Pareto front with
additional refined solutions). For nearly equal number of simulation runs, Hamdy et al. found
that the PR_GA and the GA_RF approaches provided much better Pareto fronts and
contributed more Pareto solutions than the original GA while maintaining reasonable
computational time (less than the GA). The PR_GA approach was included in the test of this
study. Multi-objective evolutionary algorithms (MOEAs) were used for several purposes,
e.g. in designing cost-effective environmentally-friendly buildings [4, 20], low-energy
highly-comfort buildings [3] or improving the energy consumption of net-zero energy solar
homes [21]. Elitist MOEAs (e.g., NSGA-II) offer great potential for implementations in this
field. However and because of their stochastic operators, they could occasionally fail to get
close to the Pareto-optimal front, particularly if a low number of iterations is implemented as
the stopping criterion [22, 23, 6]. Researchers, therefore, need more detailed guidelines more
comprehensive comparative tests so as to fully understand the natures of MOOAs in solving
BOPs. Also, an advanced test method is essential to fully examine the performance of these
algorithms. This is also the major objective of this paper.
3. Research methodology
3.1. Test algorithms
We conducted a comprehensive review on optimization methods applied to BOPs [1].
The result indicates that the evolutionary algorithms and the swarm intelligence algorithms
were the most commonly-used methods (see Figure 1). In another study [24], the author also
found that evolutionary algorithms (GA, NSGA-II, swarm intelligence algorithms…) were
among the most popular optimization methods. These findings allow us to identify the
potential algorithms to be investigated in this study.
Testing all evolutionary algorithms is well beyond the scope of this paper because of
a large number of existing evolutionary optimization methods. After considering all

8/33
possibilities, we decided to test performance of 7 popular MOOAs, including: a controlled
non-dominated sorting genetic algorithm with a passive archive (a variant of the NSGA-II),
a multi-objective particle swarm optimization (MOPSO), a two-phase optimization GA
(PR_GA), a multi-objective evolutionary algorithm based on the concept of epsilon
dominance (evMOGA), an elitist non-dominated sorting evolution strategy (ENSES), a
multi-objective differential evolution algorithm (spMODE-II), and a multi-objective
dragonfly algorithm (MODA). These algorithms were developed along 2002 to 2014. Their
codes were exploited from MathWorks/from the authors and then implemented into Matlab
optimization toolbox for use. The natures of these MOOAs are briefly introduced in the
following sub-sections.
3.1.1. Non-dominated Sorting Genetic Algorithm II with a passive archive (pNSGA-II)
The NSGA-II is one of the most commonly-used multi-objective optimization
algorithms, first proposed by Deb et al. [25]. The pNSGA-II is an improved version of the
NSGA-II, in which an archive was added. Candidate solutions that would be rejected by the
NSGA-II are stored in this archive for use. Thus the passive archive is simply the storage of
evaluated solutions. As the last step of the algorithm, the non-dominated solutions are
identified from the archive and are added to the final population. The NSGA-II with such kind
of archive strategy was applied to a number of buildings and HVAC design optimization
8
1
1
1
1
1
2
2
2
2
4
5
5
10
13 40
0 5 10 15 20 25 30 35 40
Other
Harmony search algorithms
Coordinate search algorithms
Simplex algorithms
Discrete Armijo Gradients algorithms
Tabu search
Branch and Bound methods
Other direct search algorithms
Ant colony algorithms
Simulated annealing
Other evolutionary algorithms
Hooke-Jevees algorithms
Linear programing methods
Hybrid algorithms (all types)
Particle swarm optimization
Genetic algorithms
Use frequency (times)
Figure 1: Use frequency of different optimization algorithms, the result was derived from
more than 200 building optimization studies given by SciVerse Scopus of Elsevier

9/33
problems [3] and preliminary tests show its improved performance, compared with the
NSGA-II [26].
3.1.2. Multi-objective particle swarm optimization
Eberhart and Kennedy [27] originally proposed the particle swarm algorithm for
optimization. It is a swarm-based search algorithm which is relied on the simulation of the
social behavior of bird flocking or fish schooling. In the particle swarm optimization (PSO)
algorithm, potential solutions, called particles, move through the solution space by following
the current optimal particles. The PSO has been found to be very effective in a wide variety
of applications, being able to produce good optimal solutions at an acceptable computational
cost [28]. The original PSO was only applicable to single-objective continuous optimization
problems in which the decision variables are real numbers. Several modified PSO algorithms
have been introduced using different approaches which make PSO capable in solving multi-
objective discrete optimization problems [29]. In this study, the multi-objective PSO
(MOPSO) proposed by Heris [30] was used. The algorithm of MOPSO can be retrieved from
[31].
3.1.3. Two-phase optimization using Genetic algorithm (PR_GA)
The PR_GA was first proposed by Hamdy et al. [19]. PR_GA is not an optimization
algorithm, but it is a combination of a preparation phase and main optimization one done by
the genetic algorithm. The preparation phase improves the efficiency of optimization process
by prepare a good initial population for the genetic algorithm.
By default, the multi-objective GA generates the initial population randomly using a
creation function. The population of the next generation is prepared by comparing fitness of
candidate solutions of the current generation using their non-dominated ranks and their
distance measure. By this way, the next generation will track relatively the first generation.
Random creation for the initial population and the dependence on this random behavior
usually need a large number of trials in order to achieve good results [19].
By setting a good initial population, we could produce best fitness at each generation
and little diversity for the algorithm. In addition, it allows the GA to focus on specified areas
of the search space, which is close to the optimal Pareto front. For these reasons, a preliminary
optimization run is done by deterministic algorithms in order to prepare a good initial
population for the GA before running the multi-objective genetic algorithm. The deterministic
algorithms for the preparation process of the PR_GA were Fmincon solver and Fminimax
function from Matlab 2008a optimization toolbox. In this study, a fast stopping criterion for
these algorithms was implemented since the quality of the results is not the goal of this step.

10/33
3.1.4. Multi-objective evolutionary algorithm based on the concept of epsilon dominance
(evMOGA)
The evMOGA is an elitist multi-objective genetic algorithm based on the concept of
ε-dominance which is used to control the content of the archive where the solutions are stored.
The evMOGA was first introduced in 2009 by Martinez-Iranzo et al. [32]. Different from the
original MOGA, the evMOGA tries to search for a good approximation to the Pareto front
with smart distributions (i.e. the density of solutions belonging to the Pareto front depends on
its rate of change). By this way, the evMOGA is able to characterize the Pareto front with a
small number of solutions, thereby reducing computational cost. The evMOGA also extends
the limits of the Pareto front dynamically. Details on the ε-dominance and the evMOGA can
be found in [33, 32].
3.1.5. Elitist non-dominated sorting evolution strategy (ENSES)
The ENSES algorithm is a variance algorithm based on the NSGA-II of Deb et al.
[25]. The difference between the ENSES and the NSGA-II is that the ENSES uses Evolution
Strategies (ES) instead of the genetic algorithm (GA) as the evolutionary algorithm for multi-
objective optimization. Similar to the GA, evolution strategy also uses mutation,
recombination and selection applied to a population of individuals containing candidate
solutions in order to evolve iteratively into better and better solutions. The ENSES is
implemented with an effective sorting method based on individual ranking, including a non-
dominated sorting and a crowded distance metric sorting, which is similar to the sorting
method used in the NSGA-II. It also shows good performance in providing widely distributed
Pareto optimal solutions in multi-objective optimization [34].
3.1.6. Multi-objective differential evolution algorithm with spherical pruning based on
preferences – an improved version of the spMODE algorithm (spMODE-II)
The DE is a very simple population-based, stochastic optimization algorithm,
developed in 1994 - 1996 by Storn and Price [35]. The DE is a parallel direct search method
which utilizes NP D-dimensional parameter vectors as a population for each generation G.
The main concept of the DE is the scheme for generating trial parameter vectors, which is
crucial for generating vectors of the next generation. Basically, the DE adds the weighted
difference between two population vectors to a third vector. By this way no separate
probability distribution has to be used which makes the scheme completely self-organizing
[35]. The multi-objective differential evolution (MODE) was first introduced in [36]. This
MOOA was presented as a variant of the original DE, in which the best individual is adopted
to generate the offspring. A Pareto-based approach is introduced to perform the selection of
the best individual. Similar to the NSGA-II, MODE also uses (+) selection, Pareto ranking

11/33
and crowding distance in order to produce and maintain well distributed solutions. In [37], a
spherical pruning with preferences MODE (called the spMODE-II) was presented and its code
was kept open on Matlab Central (http://www.mathworks.com/matlabcentral), from which
spMODE-II was implemented into this study.
3.1.7. MODA: Multi-Objective Dragonfly Algorithm
Mirjalili [38] proposed a new swarm-based algorithm - the dragonfly algorithm (DA)
- that is capable in solving single-objective, discrete, and multi-objective problems. The major
mechanism of the DA relies on the static and dynamic swarming behavior of dragonflies in
nature, which is similar to the two main phases of optimization using meta-heuristics:
exploration and exploitation. In the DA, these two phases are implemented by mimicking the
social interaction of dragonflies in navigating, searching for foods, and avoiding enemies
when swarming dynamically or statistically. To solve discrete problems, the author
implemented the binary version of the DA, called binary DA. A multi objective dragonfly
algorithm (MODA) was also introduced for use in multi-objective optimization. All details
and source codes of the MODA can be found in [38] and on Mathworks.
3.1.8. Parameter settings for the algorithms
The performance of the optimization algorithms not only depends on the optimization
problems but also on the settings of the algorithms, which affects solution quality and
processing time. However, due to the lack of scientific information on parameter settings for
building energy models, we did not tune the default settings of the algorithm parameters
because we assumed the default settings as general optimal settings for each algorithm. Only
the effects of changing the population size and the number of generations are investigated.
Population size is an important parameter affecting the performance of population-
based optimization methods. The population size depends on many parameters: natures of the
problem, landscape of the search space, dimensionality of the problem and the search
algorithm itself. A small population size may prevent the algorithm to fully explore the search
space, but a large population size may have bad effects on the final result if parameter settings
prevent the algorithm to fully converge or when the computational budget is fixed. In other
words, if we increase the population size while the computational budget is fixed, the number
of iterations must be decreased, resulting in early termination [39]. In this study, the
population sizes were twice / 4 times as many as the number of variables as recommended for
population-based search methods [40] [41] (for small scale problem, the population size
should be larger than the dimensionality of the problem). Thus two different choices were
used in the performance tests as follows:
- Each generation has 64 individuals;
- Each generation has 32 individuals.
A detailed explanation of all parameters of these algorithms is beyond the scope of
this paper. For further details of these parameter settings, we refer the reader to [42] for the

12/33
NSGA-II and ENSES; to [30] for the MOPSO; to [19] for the PR_GA; to [32] for the
evMOGA; to [37] for the spMODE-II; and to [38] for the MODA.
3.2. Test models
The studied house, Figure 2, is located in Helsinki, Finland, and has two floors with a net
floor area of 143 m2. The house is divided into 12 zones (living room, technical room, kitchen,
sauna, and toilet in the ground floor; four bedrooms, a common area, and toilet in the upper
floor; as well as a staircase with opened doors between the ground and upper floor). Operable
windows are installed in each zone for natural ventilation when the outdoor temperature is
lower than the summer threshold 25 °C and higher than the heating set-point temperature of
21 °C. Wind pressure coefficients are used from Jokisalo (2009) to take account of the wind
effect on infiltration and the natural ventilation of the house. No mechanical cooling is used.
Internal shading control is assumed to apply the shading when the schedule is 'on' and the
incident light exceeds 100 W/m2 on the inside of the glass. Beside the natural cooling and
internal shading, recess shading by the embrasure is implemented to minimize the summer
overheating risk. In addition, overhangs are used for the south zones on the upper floor.
Exhaust-air heat recovery is an essential feature in a cold climate. The thermal performance
of the house is evaluated by using the IDA-ICE 4.6 energy simulation program. To optimize
the simulation results, a combination between simulation and optimization is first created by
combining IDA ICE program with Matlab 2008a optimization toolbox. The weather file used
in the simulations was the reference year weather data (Vantaa_TRY2012) which was
previously developed for energy simulation in Finland [43].
As a reference design, the house is assumed to be connected to the district heating grid
with no on-site energy generation systems. The reference house is assumed to have minimum
energy performance requirements in according with Finnish building regulations. The district
heating plus the minimum energy saving measures (ESM) cost is about 21 k€. The reference
design has an energy performance level of 150 kWh/m2a, and the total primary energy use
that is a bit lower than the standard level 170 kWh/m2a, which came into force in the Finnish
code [44].

13/33
`
In order to find a cost-optimal energy performance level in line with the recast EBPD
of 2010, 16 design and operational variables were included in the optimization, as reported in
Table 1. They were categorized into 3 groups:
- Energy saving measures: measures that reduce the energy demand (from X1 to X6),
- Renewable energy sources: technologies that use renewable sources of energy (from X7
to X11),
- Mechanical systems: can be used if the demand cannot be satisfied through the renewable
systems (from X12 to X15).
The installation and maintenance costs of the energy supply systems (solar thermal
collector, photovoltaic, and primary heating system) as well as their lifespan and their related
annual subscription/energy delivering fees are based on the Finnish market 2013/2014.
Further detailed about the design variables can be retrieved from [45]. As a consequence the
total number of all possible combinations of the design options is ~1.6 1010.
Decision variables
X
No.
possible
options
Costs
Energy saving measures
Package of building envelope (PBenv.)
X1
8
From EUR 8000 to EUR 17000*1
Efficiency of lighting and appliances
X2
2
According to [46] [47] *2
Type of heat recovery unit
(efficiency %)
X3
3
EUR 1500, EUR 2000, EUR 2500 *3
Efficiency of auxiliary systems (fans and
pumps)
X4
2
EUR 800, EUR 1500,
Figure 2: The model of the house using in this study. Small red windows are operable for
natural ventilation in summer
Table 1: Design variables and design options

14/33
Size of buffer tank (Vtank)
X5
4
370 * Vtank + 1720 [48]
Insulation level of the buffer tank (Thins)
X6
4
150 * Atank *Thins [49]
Renewable energy sources
Area of solar thermal collectors
X7
8
492 ASth + 500;
Area of photovoltaic module (Apv)
X8
11
3.1Apv2 + 202Apv + 1983;
Overall efficiency of the photovoltaic
X9
2
70% of the given price
100% of the given price
Slope angle of photovoltaic module
X10
3
-
Azimuth angle of photovoltaic module
X11
4
-
Mechanical systems
Type of primary heating unit
X12
5
According: www.gebwell.fi and [50]
Size of the primary heating unit
X13
13
-
Supply water temperature from the primary
heating unit (Ts)
X14
3
-
Operating hour start at
X15
5
-
Operating hour stop at
X16
5
-
* 1 The price is calculated based on price assumptions from our previous paper [51] . Prices are updated
assuming a 1.7 % inflation rate, which is the average inflation rate of the last ten years [52]. The lifespan of
the building envelope with the exception of the window is assumed to be 60 years. For windows, a lifespan of
30 years is assumed.
* 2 Lifetimes for lighting options are modeled based on cumulative hours of use. 1200 and 10 000 operating
hours are assumed for the incandescent and fluorescent lighting, respectively.
* 3 The heat recovery is replaced every 15 years.
3.3. Objectives to be minimized
Two objective functions were used in this study and they are described as follows:
Min
1 2 1 2
( ), ( ), [ , ,..., ]
m
f x f x x x x x=
(1)
where f1: primary energy consumption (PEC).
f2: life-cycle cost (LCC) of the design solution (€/m2).
x
: combination of the design-variables (x1, x2,. . .,xm).
m: number of the design variables.
The tradeoff between these two objective functions are described in Figure 3, which also
shows the financial and environmental gaps among the present building energy requirements
and the nZEB and cost optimal buildings.
The PEC is minimized as the first objective function. The PEC considers the energy use of
the house, including energy for heating, cooling, ventilation, lighting, pumps and fans, other
technical service systems, domestic hot water systems, cooking, appliances, lighting) and the

15/33
energy-saving by the renewable energy system. The hourly photovoltaic electricity
production and the DHW energy saving were simulated by IDA ESBO program and checked
by TRNSYS program.
The LCC is the sum of the present value of investment and operating costs for building and
service systems, including those related to maintenance and replacement, including taxes,
over a specified calculation period. To calculate the LCC of the house, this study assumed a
1.7 % inflation rate; a 3.2% nominal interest rate. The average energy price was taken from
the energy company Helsingin Energia [53]. The electricity, district heating and gas energy
prices are assumed to be 12.5, 5, and 3.2 €/kWh, respectively. In order to evaluate the
influence of financial assumptions on the optimization results, we provide two different
scenarios as follows:
- Escalation rate of energy price “e” is 2% and Feed-in-tariff (FIT) is zero (no
tariff);
- Escalation rate of energy price “e” is 6% and Feed-in-tariff (FIT) is 1 (full
tariff).
3.4. Criteria for performance comparison
The performance of an optimization algorithm can be evaluated from some aspects
which may give a comprehensive view. In this study, the performance of these seven
algorithms is assessed using the following criteria and their corresponding indicators:
- Execution time of the algorithms,
- Convergence to the benchmarking optimal set, indicated by the normalized
generational distance, denoted by GDn,
Figure 3: The two objective functions of the present optimization and financial, energy and
environmental gaps between current and cost-optimal requirements and nZEB levels (Adapted from
[54])

16/33
- Normalized inversed generational distance (IGDn) is used to quantify both the
convergence and the diversity,
- Diversity of solutions in the Pareto-optimal set, indicated by the normalized diversity
metric, denoted by DMn,
- Number of solutions on the Pareto-optimal set, denoted by NoPsolution,
- Contribution of the algorithm’s solutions to the best Pareto front.
As being introduced for the first time, the original generational distance [55, 25],
inversed generational distance [56] and diversity metric [25] were calculated on the obtained
optimal solutions for the two objective functions X and Y. In this study, to avoid the biases to
one objective function, we used the normalized generational distance, normalized inversed
generational distance and normalized diversity metric by calculating them on the obtained
optimal solutions using the normalized values Xn and Yn of the two objective functions as
described by the following equations:
( ) ( )
nX
XMax X Min X
=−
(2)
( ) ( )
nY
YMax Y Min Y
=−
(3)
The normalized generational distance GDn (a variant of the original GD) is a reliable
convergence metric which does not have a bias to one of the objective functions. It indicates
the average Euclidean distance between the best Pareto front P* and the optimal solution set
S obtained by each algorithm as follows:
2
1/2
1
1i
n
i
n
GDn d
=



=
(4)
where di is the Euclidean distance (in the objective space) between the obtained
solution
iS
and the nearest member of P*; n is the number of solutions in a Pareto front
(see Figure 4). By definition, lower value of GDn is preferred.
The normalized inversed generational distance IGDn (a variant of the original IGD)
is able to measure both the closeness and the diversity of the obtained solutions. The IGDn
can measure the average distance between each member of P* and the obtained non-
dominated solutions, it is therefore strongly influenced by the number of obtained non-
dominated solutions and their distribution:
1/2
2
1
1n
i
i
m
IGDn d
=




=
(5)
where m is the number of solutions consisting in the best Pareto front; di is the
Euclidean distance between each member of P* and the nearest obtained non-dominated
solution (see Figure 4). By definition, if the number obtained non-dominated solutions is
small, the value of IGDn will grow. Once again, a lower value of IGDn is preferred.

17/33
To accurately calculate the GDn and IGDn, a Pareto front with dense solutions is required.
Thus in this study, we generated the best Pareto front by running the optimization 20 times
on each algorithm, using two different algorithm’s settings (32 generations and 64
individuals/ 64 generations and 32 individuals). Best solutions from all these optimization
runs were derived and they built up the benchmarking Pareto front. For each benchmarking
Pareto front, 286720 model evaluations of 140 optimization runs were finished. As an
example, Figure 5 shows the benchmarking Pareto front and all candidates of all obtained
Pareto fronts found by the seven algorithms after 140 optimization runs.
Figure 4: The concept of the GDn, IGDn, and DMn

18/33
Although the GDn and IGDn can give some information about the spread of the
obtained solutions, the normalized diversity metric DMn can be used to accurately measure
the diversity of the obtained solutions returned by an algorithm. The DMn was calculated as
follows:
1
( 1)
N
i
fl
i
fl
d d d d
DMn d d N d
=
+ + −
=+ + −

(6)
where di is the Euclidean distance between the consecutive solutions in the obtained
non-dominated set of solutions and
d
is the average of all distances di (i=1,.., N), assuming
there are N solutions in the obtained non-dominated set. The parameters df and dl are the
Euclidean distances between the extreme and the boundary solutions (see Figure 4). This
metric gives smaller values to better distributions and the most widely and uniformly
spreadout set of non-dominated solutions returns a DMn of zero [25].
The number of the obtained non-dominated solutions of an algorithm is denoted by
NoPsolution. Obviously, a high value of NoPsolution is preferred in multi-objective
optimization.
Figure 5: The benchmarking Pareto (black squares) as well as all candidates of all obtained
Paratos (140 Pareto fronts = 20 runs * 7 algorithms) using 200 (red) and 1800 (green)
evaluations, respectively.

19/33
4. Performance of the optimization algorithms
As stated above, this study tests the performance of the 7 algorithms using two
different population sizes (32 and 64 individuals) and two financial scenarios (FIT = 0, e =2%
and FIT = 1, e =6%) to simulate the life-cycle cost. Consequently, there were four test cases
as summarized in Table 2.
Population size (number of
individuals in a generation)
Feed-in-tariff
(FIT)
Escalation rate of
energy price “e”
Test case A1
32
0
2%
Test case A2
64
0
2%
Test case B1
32
1
6%
Test case B2
64
1
6%
In each test case and for each algorithm, we performed optimization runs several times
and consequently increasing number of generations. By this way, this study could examine
the stability of the algorithms when the number of generations is increased and the
convergence of the performance indicators. Figure 6 and Figure 7 show the optimization
results of the two optimization algorithms – the PR_GA and the ENSES obtained after 3 and
27 generations, respectively, along with their performance indicators (the IGDn and DMn). It
can be seen that the quality of the obtained solutions was improved when we increased the
number of generations. Since it is hard to evaluate and compare the quality of the optimization
solutions by using the graph only, these figures was accompanied by the IGDn and DMn,
which can be directly compared among the obtained solutions.
Table 2: The four test cases and their financial scenarios
Figure 6: The best and the worst obtained tradeoff solutions of the PR_GA and the ENSES,
in terms of IGDn and DMn, compared with the benchmarking Pareto (Case A2 - after 3
generations)

20/33
4.1. Convergence to the best Pareto front and diversity of obtained solutions
Figure 8, Figure 9, Figure 10, and Figure 11 present the performance indicators of the
7 tested algorithms, resulting from 4 test cases A1, A2, B1, and B2 respectively. In these
figures, for each indicator, each algorithm has 5 consecutive boxplots, corresponding to 5
different settings of the number of generations. Each boxplot shows the variation of the
indicator’s sample generated by running the optimization 20 times.
Figure 7: The best and the worst obtained tradeoff solutions of the PR_GA and the ENSES,
in terms of IGDn and DMn, compared with the benchmarking Pareto (Case A2 - after 27
generations)
Figure 8: The IGDn, GDn and DMn of the tested algorithms - Test case A1: population size
= 32 individuals; financial assumption: FIT = 0, e =2%. For each algorithm, optimization
was executed 20 times, then the number of generations jumped from one to another value, i.e.
from 6 to 18, 20, 42, 54 generations (≈ 200, 600, 1000, 1400, 1800 evaluations), respectively.

21/33
In the test case A1, Figure 8 indicates that the PR_GA has the best performance
indicators, followed by the spMODE-II which has consistent small IGDn, GDn and DMn.
The PR_GA was therefore the strongest optimization method among these algorithms. The
ENSES shows the worst convergence indicators, but it could give a good spreadout set of
non-dominated solutions. In the test case A2, Figure 9 shows a similar result where the
PR_GA outperformed the remaining algorithms while the ENSES returned poor converged
solutions.
Figure 10 and Figure 11 demonstrate the results of the test case B1 and B2. Similarly,
the PR_GA and the spMODE-II continue to show their superiority in terms of convergence
and diversity. The ENSES always show the poorest convergence, but acceptable diversity.
The remaining algorithms – the NSGA-II, MOPSO, evMOGA and MODA were not
competitive.
Figure 9: The IGDn, GDn and DMn of the tested algorithms - Test case A2: population size
= 64 individuals; financial assumption: FIT = 0, e =2%. For each algorithm, optimization
was executed 20 times, then the number of generations jumped from one to another value, i.e.
from 3 to 9, 15, 21, 27 generations (≈ 200, 600, 1000, 1400, 1800 evaluations), respectively.

22/33
Based on the slopes of the boxplots in Figure 8-Figure 11, we could estimate the
convergence rates of the algorithms to their Pareto optimal solutions. It can be observed that
Figure 10: The IGDn, GDn and DMn of the tested algorithms - Test case B1: population size
= 32 individuals; financial assumption: FIT = 1, e =6%. For each algorithm, optimization
was executed 20 times, then the number of generations jumped from one to another value, i.e.
from 6 to 18, 20, 42, 54 generations (≈ 200, 600, 1000, 1400, 1800 evaluations), respectively.
Figure 11: The IGDn, GDn and DMn of the tested algorithms - Test case B2: population size
= 64 individuals; financial assumption: FIT = 1, e =6%. For each algorithm, optimization
was executed 20 times, then the number of generations jumped from one to another value, i.e.
from 3 to 9, 15, 21, 27 generations (≈ 200, 600, 1000, 1400, 1800 evaluations), respectively.

23/33
the pNSGA-II, MOPSO, evMOGA, ENSES and MODA reached their optimal solutions at a
low number of evaluations, e.g. after about 1000 evaluations. Their solutions after 1000
evaluations showed minor improvements. Conversely, the PR_GA and the spMODE-II
continued to improve their solutions considerably, even after 1800 evaluations. Their obtained
solutions at 1800 evaluations differed significantly from those at 200 and 600 evaluations. It
is therefore important to compare performance of the optimization algorithms when
convergence has been reached.
4.2. Execution time
In this study, execution time of an algorithm was defined as average time needed to
prepare all individuals for one generation. Figure 12 compares the execution time of 7
investigated algorithms. It can be seen the execution time of all algorithms varied
considerably from the case A1 to case B2. In the test A1 and A2, the MOPSO, ENSES and
MODA were the slowest algorithms while the remaining algorithms required short execution
time. The situation changed significantly in the test B1 and B2 where only the slowest
algorithm was recognized. In brief, it is hard to draw any solid results from the comparison
of execution time.
Figure 12: Box plots of the execution time of the 7 investigated algorithms (excluding the
simulation time)

24/33
4.3. Number of solutions on the Pareto-optimal set
Figure 13 shows the number of solutions on the Pareto-optimal set of the 7 algorithms.
In all test cases, the pNSGA-II yielded the best results, followed by the PR_GA. The ENSES
provided a limited number of non-dominated solutions, which can be used to explain the
uncompetitive IGDn and GDn found in Figure 8 - Figure 11. In our study we observed that
for a given population size, an increased number of generations (i.e. increased number of
evaluations) resulted in more non-dominated solutions. As observed from Figure 13, the
increased trends were almost leveled out at the 4th and 5th boxplots. It means that about 1400
to 1800 evaluations were sufficient to stabilize the number of non-dominated solutions. More
evaluations may not improve this value.
Although the NoPsolution can reveal the efficiency of the algorithms in finding non-
dominated solutions, it cannot tell us the quality of these obtained solutions. This study
therefore examined the number of solutions contributed by an algorithm to the best Pareto
solutions of each optimization series. Each optimization series includes 20 optimization runs
on each algorithm, and done on all 7 tested algorithms in turn. Table 3 shows the percentage
of the best Pareto solutions contributed by each algorithm.
Figure 13: The NoPsolution of the 7 investigated algorithms in the four test cases (top to
bottom: cases A1, A2, B1, and B2 respectively)

25/33
It is worthy of note that these contributions changed significantly when the total
number of evaluations was increased. At convergence; i.e. optimization results after more
than 1400 evaluations; the PR_GA contributes the most to the best Pareto solutions, from
50% in the test case A1 up to 73.8% in the test case B2. The second largest contributors are
likely the evMOGA and spMODE-II. The remaining algorithms almost have no or negligible
contribution, although at the earlier stage of the optimization (low number of evaluations)
they tend to contribute the larger portions.
Test case A1 (population size = 32, FIT = 0, e = 2%)
Number of
evaluations
pNSGA-
II
MOSP
O
PR_G
A
evMOG
A
ENSE
S
spMODE-
II
MOD
A
Total
(%)
200
boxplot 1
18.6
20.9
7.0
20.9
2.3
25.6
4.7
100.0
600
boxplot 2
16.9
6.8
27.1
22.0
0.0
25.4
1.7
100.0
1000
boxplot 3
0.0
6.7
66.7
5.0
0.0
21.7
0.0
100.0
1400
boxplot 4
0.0
3.6
73.2
5.4
0.0
16.1
1.8
100.0
1800
boxplot 5
1.7
10.3
50.0
6.9
0.0
31.0
0.0
100.0
Test case A2 (population size = 64, FIT = 0, e = 2%)
Number of
evaluations
pNSGA-
II
MOSP
O
PR_G
A
evMOG
A
ENSE
S
spMODE-
II
MOD
A
Total
(%)
200
boxplot 1
7.1
23.8
9.5
26.2
0.0
31.0
2.4
100.0
600
boxplot 2
31.1
13.1
9.8
18.0
1.6
23.0
3.3
100.0
1000
boxplot 3
3.5
0.0
63.2
12.3
3.5
12.3
5.3
100.0
1400
boxplot 4
3.4
0.0
77.6
10.3
0.0
6.9
1.7
100.0
1800
boxplot 5
4.3
0.0
66.7
17.4
1.4
8.7
1.4
100.0
Test case B1 (population size = 32, FIT = 1, e = 6%)
Number of
evaluations
pNSGA-
II
MOSP
O
PR_G
A
evMOG
A
ENSE
S
spMODE-
II
MOD
A
Total
(%)
200
boxplot 1
40,6
3,1
0,0
34,4
3,1
18,8
0,0
100,0
600
boxplot 2
26,5
2,0
12,2
32,7
0,0
24,5
2,0
100,0
1000
boxplot 3
8,9
0,0
53,3
17,8
0,0
20,0
0,0
100,0
1400
boxplot 4
4,8
0,0
54,8
12,9
0,0
27,4
0,0
100,0
1800
boxplot 5
8,8
0,0
40,4
21,1
0,0
29,8
0,0
100,0
Test case B2 (population size = 64, FIT = 1, e = 6%)
Number of
evaluations
pNSGA-
II
MOSP
O
PR_G
A
evMOG
A
ENSE
S
spMODE-
II
MOD
A
Total
(%)
200
boxplot 1
3.1
25.0
0.0
43.8
0.0
18.8
9.4
100.0
600
boxplot 2
22.9
10.4
10.4
37.5
0.0
16.7
2.1
100.0
1000
boxplot 3
13.7
3.9
49.0
19.6
0.0
9.8
3.9
100.0
1400
boxplot 4
0.0
0.0
98.1
0.0
0.0
1.9
0.0
100.0
1800
boxplot 5
1.6
0.0
73.8
16.4
0.0
8.2
0.0
100.0
4.4. Performance ranking and some observations
As this study includes many test cases, performance indicators on 7 tested algorithms, there
were too many things to compare or/and examine. We therefore ranked the performance of
Table 3: Contribution of each algorithm to the best Pareto solutions (%)

26/33
the algorithms based on the indicators and then summarized these rankings in Table 4. By this
way, we could obtain some clear observations as follows:
- In terms of convergence of the obtained solutions (indicated by the IGDn and GDn),
the PR_GA was superior to the others while the ENSES was the worst;
- In terms of diversity of obtained solutions (indicated by the DMn), the PR_GA,
ENSES and spMODE-II were slightly better than the remaining algorithms;
- In terms of execution time, none of the tested algorithms showed their outstanding
performance, but it is easy to observe that the MOPSO and the ENSES were the slowest;
- In terms of the number of Pareto optimal solutions (indicated by the NoPsolution), the
pNSGA-II and the PR_GA offered much more solutions than any other algorithms. The
ENSES was the worst.
- Finally, in terms of contributions of the best solutions to the best Pareto fronts, once
again the PR_GA outperformed the others while the MOPSO, the ENSES and the MODA
were not competitive.
Algorithm
Test
case
Ranking based on the performance indicators
IGDn
GDn
DMn
Time
NoPsolution
Contribution
pNSGA-II
A1
5
2
7
1
1
5
A2
4
2
6
2
1
4
B1
4
2
5
5
2
4
B2
3
2
6
4
1
4
MOPSO
A1
7
5
6
7
4-5*
3
A2
6
5
7
7
4
7
B1
6
5
7
6
5
5-6-7*
B2
6
4
7
3
4
5-6-7*
PR_GA
A1
1
1
1
4
2
1
A2
1
1
1
4
2
1
B1
1
1
2
1
1
1
B2
1
1
3
6
2
1
evMOGA
A1
3
4
4
3
3
4
A2
3
4
5
3
3
2
B1
3
4
4
3
3
3
B2
4
3
5
1
3
2
ENSES
A1
6
7
2
6
7
6-7*
A2
7
7
2
5
7
5-6*
B1
7
7
3
7
7
5-6-7*
B2
7
7
2
7
7
5-6-7*
spMODE-
II
A1
2
3
3
2
4-5*
2
A2
2
3
3
1
6
3
B1
2
3
1
4
4
2
B2
2
5
1
5
6
3
MODA
A1
4
6
5
5
6
6-7*
A2
5
6
4
6
5
5-6*
B1
5
6
6
2
6
5-6-7*
Table 4: Performance ranking of the 7 investigated algorithms, all performance criteria

27/33
B2
5
6
4
2
5
5-6-7*
*Algorithms that have the same rankings
In general, the PR_GA outperformed the others in most categories, including the most
important ones. Conversely, the ENSES was the least convincing algorithm as it achieved low
scores in most categories. In our test, the MODA and MOPSO did not exhibit any outstanding
feature. The pNSGA-II, evMOGA and the spMODE-II achieved average rankings in most
tests thus they can be seen as alternative choices if the PR_GA is not readily available.
5. Discussion and conclusion
A comparison of the performance of seven multi-objective optimization algorithms,
including the pNSGA-II, MOPSO, PR_GA, evMOGA, ENSES, spMODE-II and MODA has
been carried out by means of computational experiments using a complex building energy
model. We applied a multi-criteria system of performance indicators along with a
comprehensive test procedure which allowed us to quantify many features of these MOOAs.
Two important assumptions of this study were that (i) we used most of the default settings
of the algorithms (as recommended by the authors of the algorithms) and (ii) all algorithms
were executed using the same number of evaluations/population sizes/. These assumptions
could provide a fair competition arena among the algorithms, but they may fail to enable the
maximum power of these algorithms. As an example, if we test convergence of two
algorithms with different convergence speeds using the same number of evaluations, the
algorithm with the fast convergence speed will perform better. However, if we set another
termination criterion (e.g. objective function convergence: changes of the objective functions
are smaller than a user-specified threshold [1]), the obtained result might change. Hence the
results of this study need to be interpreted carefully, accompanied by a full description of the
assumptions.
In our test, the PR_GA consistently shows competitive performance indicators. The
pNSGA-II, evMOGA and spMODE-II were fairly stable and returned acceptable
performance indicators. The ENSES, MOPSO and MODA were not competitive in most
performance criteria. It is worthy of note that an algorithm may achieve a high score on one
performance criterion, but obtain low scores on one or some other criteria. This is
understandable as there is no single best algorithm for all optimization problems (no free
lunch theorem).
The PR_GA showed outstanding scores in the test; however it is not an optimization
algorithm. The PR_GA is intrinsically an optimization method in which a preparation phase
is conducted for seeding the initial population with viable solutions before running the
optimization using the GA. It is a well-established technique for improving optimization
efficiency of evolutionary algorithms [57] and showed competitive performance in BOPs [58,
19, 59, 60]. Thus the PR_GA can be considered the first choice for BOPs. If we only consider

28/33
the ‘single’ algorithms, the pNSGA-II, evMOGA and spMODE-II were the potential choices
among the tested algorithms.
Performance of an optimization algorithm is undoubtedly important, especially in some
areas where stringent criteria on system reliability and safety are usually required, e.g.
structural engineering or aerospace engineering. However, selected optimization algorithms
should be made based on the purpose of the analysis, i.e. conceivably even the worst
investigated MOOA could be good enough for the problem under investigated, or –
alternatively – even the best one couldn’t be good enough. For example, in building design
during the conceptual phase, a preliminary optimization run with a simple algorithm is
sometimes sufficient for the task. Thus users remain the most important decision makers in
selecting a suitable MOOA.
One important finding of this study is that none of the tested algorithms could converge
completely to the best Pareto front within 1800 evaluations. The population size and the
number of generations need to be defined carefully as the optimization result is sensitive to
these values. This is because the stochastic nature of the population-based optimization
algorithms which cannot assure a global minimum within a finite number of evaluations. We
found that quality of the obtained solutions was gradually improved when the number of
evaluations/generations increased. In our test, the performance indicators were likely to
stabilize at 1400 to 1800 evaluations. We therefore recommend these values as minimum
required number of evaluations for multi-objective optimization of building energy models.
To maximize the power of a MOOA, we strongly recommend that researchers perform
convergence tests to define the required number of evaluations.
As the main aim of this study is to enrich understanding on the natures of multi-objective
optimization rather than to recommend the best algorithm for other applications, we try to
provide an overall view of the performance and behavior of these algorithms based on which
researchers can make a choice for their specific problem. In addition, performance of the
MOOAs may be sensitive to test problems, which was not investigated in this study. Due to
computational expensive simulations, the present tests were performed on one selected
building energy model, thus we still have a question whether the building models have any
influence on the test results. Knowledge and understanding about the MOOAs is therefore
important in order to make a right choice.
References
[1] A. T. Nguyen, S. Reiter, P. Rigo, A review on simulation-based optimization methods
applied to building performance analysis, Applied Energy 113 (2014) 1043-1058.
[2] M. Hamdy, M. Palonen, A. Hasan, Implementation of pareto-archive NSGA-II
algorithms to a nearly-zero-energy building optimisation problem, in Proceedings of the
2012 Building Simulation and Optimization Conference, Loughborough, Leicestershire,
UK, 2012.

29/33
[3] M. Hamdy, A. Hasan, K. Siren, Impact of adaptive thermal comfort criteria on building
energy use and cooling equipment size using a multi-objective optimization scheme,
Energy and Buildings 43 (2011) 2055-2067.
[4] W. Wang, R. Zmeureanu, H. Rivard, Applying multi-objective genetic algorithms in
green building design optimization, Building and Environment, 40 (11) (2005) 1512-
1525.
[5] L. Magnier, F. Haghighat, Multiobjective optimization of building design using
TRNSYS simulations, genetic algorithm, and Artificial Neural Network, Building and
Environment 45 (3) (2010) 739-746.
[6] M. Hamdy, A. Hasan, K. Sirén, Trade-off Relation between Energy Consumption and
Comfort Level according to the Finnish-2008 Adaptive Thermal Comfort Criteria, in
Proceedings of the 12th International conference on Air distribution in Rooms,
Trondheim, Norway, 2011.
[7] M. Wetter, J. A. Wright, A comparison of deterministic and probabilistic optimization
algorithms for nonsmooth simulation-based optimization, Building and Environment 39
(2004) 989 – 999.
[8] F. Pernodet, H. Lahmidi, P. Michel, Use of genetic algorithms for multicriteria
optimization of building refurbishment, in In Proceedings of the 11th International
IBPSA Conference, Glasgow, Scotland, 2009.
[9] M. Wetter, J. A. Wright, Comparison of a generalized pattern search and a genetic
algorithm optimization method, in Proceedings of the 8th IBPSA Conference,
Eindhoven, Netherlands, 2003.
[10] M. Wetter, GenOpt, Generic optimization program - User manual, version 3.0.0.
Technical report LBNL-5419, Lawrence Berkeley National Laboratory, 2009.
[11] J. Kampf, M. Wetter, D. Robinson, A comparison of global optimisation algorithms
with standard benchmark functions and real-world applications using EnergyPlus,
Journal of Building Performance Simulation 3 (2010) 103-120.
[12] W. S. Lee, Y. T. Chen, Y. Kao, Optimal chiller loading by differential evolution
algorithm for reducing energy consumption, Energy and Buildings 43 (2) (2011) 599-
604.
[13] D. Tuhus-Dubrow, M. Krarti, Genetic-algorithm based approach to optimize building
envelope design for residential buildings, Building and Environment 45 (2010) 1574–
1581.
[14] Y. Bichiou, M. Krarti, Optimization of envelope and HVAC systems selection for
residential buildings, Energy and Buildings 43(12) (2011) 3373-3382.
[15] D. H. Wolpert, W. G. Macready, No Free Lunch Theorems for Optimization, IEEE
Transactions on Evolutionary Computation 1(1) (1997) 67–82.
[16] N. Nassif, S. Kajl, R. Sabourin, Evolutionary algorithms for multi-objective
optimization in HVAC system control strategy, in Processing NAFIPS'04. IEEE Annual
Meeting of the Fuzzy Information, Banff, Alta, Canada, 2004.

30/33
[17] C. J. Hopfe, Uncertainty and sensitivity analysis in building performance simulation for
decision support and design optimization, (PhD diss.) Eindhoven University,
Eindhoven, 2009.
[18] A. E. Brownlee, J. A. Wright, M. M. Mourshed, A multi-objective window optimisation
problem, in Proceedings of the 13th annual conference companion on Genetic and
evolutionary computation, Dublin, Ireland, 2011.
[19] M. Hamdy, A. Hasan, K. Sirén, Combination of optimization algorithms for a multi-
objective building design problem, in Proceedings of the 11th International Building
Performance Simulation Association Conference, Glasgow-UK, 2009.
[20] M. Hamdy, A. Hasan, K. Sirén, Optimum design of a house and its HVAC systems
using simulation-based optimization, International Journal of Low-Carbon
Technologies 5 (3) (2010) 120-124.
[21] R. Charron, A. Athienitis, The use of genetic algorithms for a net-zero energy solar
home design optimisation tool, in In Proceedings of PLEA 2006 (Conference on Passive
and Low Energy Architecture), Geneva, Switzerland, 2006.
[22] M. Palonen, A. Hasan, K. Siren, A Genetic algorithm for optimization of building
envelope and HVAC system parameters, in Proceedings of the eleventh International
IBPSA Conference, Glasgow, Scotland, 2009.
[23] M. Hamdy, Combining Simulation and Optimisation for Dimensioning Optimal
Building Envelopes and HVAC Systems, Aalto University publication series - Doctoral
Dissertation177/2012, 2012.
[24] R. Evins, A review of computational optimisation methods applied to sustainable
building design, Renewable and Sustainable Energy Reviews (22) (2013) 230-245.
[25] K. Deb, A. Pratap, S. Agarwal, T. A. M. T. Meyarivan, A fast and elitist multiobjective
genetic algorithm: NSGA-II, IEEE Transactions on Evolutionary Computation 6 (2)
(2002) 182-197.
[26] M. Hamdy, M. Palonen, A. Hasan, Implementation of pareto-archive NSGA-II
algorithms to a nearly-zero-energy building optimisation problem, in Proceedings of the
building simulation and optimization conference, Loughborough, UK, 2012.
[27] R. C. Eberhart, J. Kennedy, A new optimizer using particle swarm theory, in
Proceedings of the Sixth International Symposium on Micro Machine and Human
Science, Nagoya, Japan, 1995.
[28] A. P. Engelbrecht, Fundamentals of Computational Swarm Intelligence, John Wiley &
Sons, New York, 2005.
[29] M. Reyes-Sierra, C. C. Coello, Multi-objective particle swarm optimizers: A survey of
the state-of-the-art, International journal of computational intelligence research 2 (3)
(2006) 287-308.
[30] M. K. Heris, Multi-Objective Particle Swarm Optimization - Version 1.0 (Adapted
from: C. A. C. Coello, G. T. Pulido, M. S. Lechuga, Handling multiple objectives with
particle swarm optimization, IEEE Transactions on Evolutionary Computation 8 (3)

31/33
(2004) 256-279), Feb.2011. [Online]. Available: http://www.kalami.ir. [Accessed 1 8
2015].
[31] C. C. Coello, S. M. Lechuga, MOPSO: A Proposal for Multiple Objective Particle
Swarm Optimization, in Congress on Evolutionary Computation (CEC'2002),
Honolulu, Hawaii, U.S.A., 2002.
[32] M. Martínez-Iranzo, J. M. Herrero, J. Sanchis, X. Blasco, S. García-Nieto, Applied
Pareto multi-objective optimization by stochastic solvers, Engineering applications of
artificial intelligence 22 (3) (2009) 455-465.
[33] M. Laumanns, L. Thiele, K. Deb, E. Zitzler, Combining convergence and diversity in
evolutionary multiobjective optimization, Evolutionary computation 10 (3) (2002) 263-
282.
[34] A. B. Ilyani Akmar, O. Kramer, T. Rabczuk, Multi-Objective Evolutionary
Optimization of Sandwich Structures: An Evaluation by Elitist Non-dominated Sorting
Evolution Strategy, American Journal of Engineering and Applied Sciences 8 (1) (2015)
185-201.
[35] R. Storn, K. Price, Differential evolution–a simple and efficient heuristic for global
optimization over continuous spaces, Journal of global optimization 11 (4) (1997) 341-
359.
[36] F. Xue, A. Sanderson, R. J. Graves, Pareto-based multi-objective differential evolution,
in Proceedings of the 2003 Congress on Evolutionary Computation (CEC’2003),
Canberra, Australia, 2003.
[37] G. Reynoso-Meza, Controller Tuning by Means of Evolutionary Multi-objective
Optimization: a Holistic Multiobjective Optimization Design Procedure, (PhD. Thesis)
Universitat Politècnica de València, Valencia, 2014.
[38] S. Mirjalili, Dragonfly algorithm: a new meta-heuristic optimization technique for
solving single-objective, discrete, and multi-objective problems, Neural Computing and
Applications (Article in press), DOI: http://dx.doi.org/10.1007/s00521-015-1920-1,
2015.
[39] ResearchGate, "What is the optimal/recommended population size for differential
evolution?" 2015. [Online]. Available: Available from:
https://www.researchgate.net/post/What_is_the_optimal_recommended_population_si
ze_for_differential_evolution2/2. [Accessed 23 8 2015].
[40] S. Chen, J. Montgomery, A. Bolufé-Röhler, Measuring the curse of dimensionality and
its effects on particle swarm optimization and differential evolution, Applied
Intelligence 42 (3) (2015) 514-526.
[41] G. M. Mauro, M. Hamdy, G. P. Vanoli, N. Bianco, J. L. Hensen, A new methodology
for investigating the cost-optimality of energy retrofitting a building category, Energy
and Buildings 107 (2015) 456-478.
[42] K. Deb, Multi-objective optimization using evolutionary algorithms, John Wiley &
Sons, Kanpur, 2004.

32/33
[43] T. Kalamees, K. Jylhä, H. Tietäväinen, J. Jokisalo, S. Ilomets, R. Hyvönen, S. Saku,
Development of weighting factors for climate variables for selecting the energy
reference year according to the EN ISO 15927-4 standard, Energy and Buildings 47
(2012) 53-60.
[44] D3, D3 Finland Code of Building Regulation. Energy Management in Buildings,
Regulations and Guidelines, Ministry of Environment, Helsinki, 2012.
[45] M. Hamdy, K. Sirén, A multi-aid optimization scheme for large-scale investigation of
cost-optimality and energy performance of buildings, Journal of Building Performance
Simulation, (ahead-of-print) (2015) 1-20.
[46] Eartheasy, LED Light Bulbs: Comparison Charts, 2015. [Online]. Available:
http://eartheasy.com/live_led_bulbs_comparison.html. [Accessed 2 8 2015].
[47] B. Lapillonne, K. Pollier, N. Samci, Energy Efficiency Trends for Households in the
EU, Enerdata, 2013.
[48] Buffer Tanks, Hot Water Buffer Tanks, 2015.
[49] Homewyse, Cost of Mineral Wool Insulation, 2015. [Online]. Available:
http://www.homewyse.com. [Accessed 2 8 2015].
[50] I. Staffell, R. Green, The cost of domestic fuel cell micro-CHP systems Discussion
paper, 2012. [Online]. Available:
https://spiral.imperial.ac.uk/bitstream/10044/1/9844/6/Green%202012-08.pdf.
[Accessed 2 8 2015].
[51] M. Hamdy, A. Hasan, K. Sirén, A Multi-stage Optimization Method for Cost-Optimal
nearly-Zero-Energy Building Solutions in Line with the EPBD-Recast 2010, Energy
and Buildings 56 (1) (2013) 189–203.
[52] Triami Media BV, Worldwide Inflation Data, 2015. [Online]. Available:
http://www.inflation.eu/. [Accessed 2 8 2015].
[53] H. Energia, Helsingin Energia, 2015. [Online]. Available:
https://www.helen.fi/kotitalouksille/. [Accessed 01 August 2015].
[54] BPIE, Implementing the Cost-Optimal Methodology in EU Countries. Lessons Learned
from Three Case Studies, in I. Nolte, O. Rapf, D. Staniaszek, M. Faber (Eds.), Buildings
Performance Institute Europe (BPIE), Brussels, 2013.
[55] D. A. Van Veldhuizen, G. B. Lamont, Multiobjective evolutionary algorithm research:
A history and analysis. Technical Report TR-98-03, Wright-Patterson AFB, Ohio:
Department of Electrical and Computer Engineering, Graduate School of Engineering,
Air Force Institute of Technology, 1998.
[56] M. A. Villalobos-Arias, G. T. Pulido, C. A. C. Coello, A proposal to use stripes to
maintain diversity in a multi-objective particle swarm optimizer, in In Swarm
Intelligence Symposium - SIS 2005, Proceedings 2005 IEEE, Pasadena, CA, 2005.
[57] B. A. Julstrom, Seeding the population: improved performance in a genetic algorithm
for the rectilinear Steiner problem, in Proceedings of the 1994 ACM symposium on
Applied computing, Phoenix, AZ, 1994.

33/33
[58] R. Evins, Configuration of a genetic algorithm for multi-objective optimisation of solar
gain to buildings, in Proceedings of the Genetic and Evolutionary Computation
Conference 2010, Portland, OR, 2010.
[59] G. Kayo, R. Ooka, Building energy system optimizations with utilization of waste heat
from cogenerations by means of genetic algorithm, Energy and Buildings 42 (7) (2010)
985-991.
[60] J. A. Wright, A. Brownlee, M. M. Mourshed, M. Wang, Multi-objective optimization
of cellular fenestration by an evolutionary algorithm, Journal of Building Performance
Simulation 7 (1) (2014) 33-51.