Content uploaded by Fabio Lopez-Pires

Author content

All content in this area was uploaded by Fabio Lopez-Pires on Dec 28, 2017

Content may be subject to copyright.

An Experimental Comparison of Algorithms for Virtual

Machine Placement Considering Many Objectives

Fabio López-Pires12, Benjamín Barán2,

Augusto Amarilla2, Leonardo Benítez2, Rodrigo Ferreira2, Saúl Zalimben2

fabio.lopez@pti.org.py, bbaran@pol.una.py,

{agu.amarilla, benitez.leonardo.py, rodrigofepy, szalimben93}@gmail.com

1Itaipu Technological Park

2National University of Asunción

Paraguay

ABSTRACT

Cloud computing datacenters provide thousands to millions

of virtual machines (VMs) on-demand in highly dynamic

environments, requiring quick placement of requested VMs

into available physical machines (PMs). Due to the random-

ness of customer requests, the Virtual Machine Placement

(VMP) should be formulated as an online optimization pro-

blem. This work presents a formulation of a VMP problem

considering the optimization of the following objective func-

tions: (1) power consumption, (2) economical revenue, (3)

quality of service and (4) resource utilization. To analyze

alternatives to solve the formulated problem, an experimen-

tal comparison of ﬁve diﬀerent online deterministic heuris-

tics against an oﬄine memetic algorithm with migration of

VMs was performed, considering several experimental work-

loads. Simulations indicate that First-Fit Decreasing algo-

rithm (A4) outperforms other evaluated heuristics on ave-

rage. Experimental results prove that an oﬄine memetic

algorithm improves the quality of the solutions with migra-

tions of VMs at the expense of placement reconﬁgurations.

Keywords

Virtual Machine Placement; Cloud Computing; Online Al-

gorithms; Experimental Comparison; Optimization.

1. INTRODUCTION

A main concern of cloud datacenters design is to eﬃciently

manage available resources in order to improve performance

and reduce energy consumption of a given computational

infrastructure. Most of the time, servers operate in a very

low energy-eﬃciency region (i.e. between 10% and 50% of

resource utilization), even considering that workload peaks

rarely occur in practice [2]. Several methods were considered

for addressing this issue, being the virtualization of resources

the most studied for cloud computing datacenters.

Permission to make digital or hard copies of all or part of this work for personal or

classroom use is granted without fee provided that copies are not made or distributed

for proﬁt or commercial advantage and that copies bear this notice and the full cita-

tion on the ﬁrst page. Copyrights for components of this work owned by others than

ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re-

publish, to post on servers or to redistribute to lists, requires prior speciﬁc permission

and/or a fee. Request permissions from permissions@acm.org.

LANC ’16, October 13-14, 2016, Valparaiso, Chile

c

2016 ACM. ISBN 978-1-4503-4591-0/16/10. . . $15.00

DOI: http://dx.doi.org/10.1145/2998373.2998374

In cloud computing datacenters, resources are allocated

and released dynamically trying to serve requested demands.

In this context, deciding the right allocation of virtual ma-

chines (VMs) into physical machines (PMs) is known in the

specialized literature as Virtual Machine Placement (VMP).

It is relevant to remember that the VMP problem is a

NP-Hard combinatorial optimization problem [17]. From

an Infrastructure as a Service (IaaS) provider’s perspective,

the VMP problem must be formulated as an online problem

(considering that requests from customers are unknown a

priori) and it should be solved with short time constraints.

Consequently, solution techniques with low computational

complexity are very studied for this VMP environment [13].

The most studied heuristics in the VMP literature are:

Best-Fit (BF), Best-Fit Decreasing (BFD), First-Fit (FF),

First-Fit Decreasing (FFD) and Worst-Fit (WF) [13].

As presented by L´opez-Pires and Bar´an in [13], over 60

diﬀerent objectives have been proposed for VMP problems.

The number of considered objective functions may rapidly

increase once a complete understanding of the VMP problem

is accomplished for practical problems, where many diﬀerent

parameters should be ideally taken into account.

This work focuses in online formulations of a provider-

oriented VMP problem in federated-clouds [13] to provide

relevant information for design and implementation of re-

source management systems, more speciﬁcally, resource al-

location algorithms for VMP problems. Many relevant ob-

jectives for IaaS providers are considered in this work. To

solve the formulated online problem, ﬁve of the most stu-

died heuristics were compared against a Memetic Algorithm

(MA) solving an oﬄine formulation of the problem.

Background and Motivation

L´opez-Pires and Bar´an recently proposed in [9] and [12]

oﬄine formulations of VMP problems considering many ob-

jectives, proposing novel memetic algorithms for solving the

formulated problems. Considering the on-demand model of

cloud computing with dynamic resource provisioning and

dynamic workloads of cloud applications [14], the resolution

of the VMP problem should be performed as fast as possible.

Consequently, applying only oﬄine formulations of the VMP

problem with meta-heuristics as solution technique, may not

be appropriate for these dynamic environments. Therefore,

as previously mentioned, solution techniques with low com-

putational complexity (e.g. heuristics) are very studied for

solving online formulations of VMP problems [13].

To help IaaS providers in the design and implementation

of resource allocation algorithms considering many objective

functions, the following research question must be answered:

Which heuristics are most suitable for solving online VMP

problems in federated-clouds considering many objectives?

In this context, Fang et al. presented in [6] a validation of

a proposed power-aware algorithm against well-known heu-

ristics such as: BF, FF and WF. Additionally, Jin et al. stu-

died FFD and Dominant Resource First (DRF) algorithms

[10], while Anand et al. evaluated in [1] a proposed In-

teger Linear Programming (ILP) formulation and an FDD

algorithm to attend common limitations of ILP algorithms

for large instances of NP-Hard optimization problems. On

the contrary, this work does not compare novel algorithms

against well-known heuristics as presented in the above men-

tioned works. The main goal of the presented experimental

comparison is to analyze the online nature of the VMP pro-

blem optimizing many diﬀerent objectives, identifying ad-

vantages and disadvantages of well-known online heuristics

against oﬄine alternatives such as the MA proposed in [9].

Experimental results presented by Ihara et al. in [9] rec-

ommend the combination of many objective functions into

a single objective for IaaS environments, given the require-

ment of obtaining solutions in a short period of time.

To compare the considered algorithms (see Section 3),

a provider-oriented VMP problem formulation is proposed

for the optimization of four objective functions: (1) power

consumption, (2) economical revenue, (3) quality of service

and (4) resource utilization (see Section 2). To combine the

above mentioned objectives, a weighted sum method is con-

sidered. Experiments were performed to deﬁne appropriate

weights for each objective function (see Section 4).

The remainder of this paper is structured in the following

way: Section 2 summarizes the proposed provider-oriented

VMP problem formulation with many objectives. Section 3

describes the considered algorithms to solve the formulated

problem, while Section 4 presents experimental workloads,

obtained results and main ﬁndings of the comparison. Fi-

nally, conclusions and future work are left to Section 5.

2. PROBLEM FORMULATION

This section presents a formulation of a VMP problem

considering the optimization of the following objective func-

tions: (1) power consumption, (2) economical revenue, (3)

quality of service and (4) resource utilization. According

to the taxonomy presented in [13], this work focuses on

a provider-oriented VMP for federated-cloud deployments,

considering two formulation types: (1) online and (2) oﬄine.

An online problem formulation is considered when an algo-

rithm makes decisions on-the-ﬂy without knowing upcoming

events (e.g. online heuristics for VMP problems) [4]. On the

other hand, if an algorithm has a complete knowledge of the

future events of a problem instance, the formulation is called

oﬄine (e.g. MAs for VMP problems used in [9] and [12]).

The formulation of the proposed provider-oriented VMP

problem is based on [9] and could be enunciated as:

Given a cloud infrastructure composed by a set of PMs (H),

a dynamic scenario composed by a set of VMs requested at

each discrete time t(V(t)) and the current placement of VMs

into PMs (P(t)), it is incrementally sought a placement of

V(t)into Hfor the next time instant (P(t+ 1)), satisfying

the constraints and optimizing the considered objectives.

In online algorithms for solving the proposed VMP pro-

blem, placement decisions are performed at each discrete

time t, without knowledge of upcoming VM requests. On

the other hand, an oﬄine algorithm have knowledge of the

complete set of VM requests in order to decide the place-

ment of these VMs into available PMs. Consequently, the

current placement is not necessary for oﬄine algorithms.

2.1 Input Data

The proposed formulation of the VMP problem models a

cloud computing datacenter, composed by PMs and VMs,

receiving the following information as input data:

•a set of navailable PMs and their speciﬁcations;

•a set of m(t) VMs at each discrete time tand their

speciﬁcations;

•the current placement at each discrete time t.

The set of PMs is represented as a matrix H∈Rn×4, as

presented in (1). Each PM Hiis represented by its physi-

cal resources. This work considers 3 diﬀerent physical re-

sources (P r1-P r3): CPU [ECU], RAM memory [GB] and

network capacity [Mbps]. It is important to mention that

the proposed notation is general enough to include more

characteristics associated to physical resources. Finally, the

maximum power consumption [W] is also considered, i.e.

H=

P r1,1P r2,1P r3,1pmax1

. . . . . . . . . . . .

P r1,n P r2,n P r3,n pmaxn

(1)

where:

P r1,i: Processing resources of Hiin [ECU];

P r2,i: Memory resources of Hiin [GB];

P r3,i: Network capacity resources of Hiin [Mbps];

pmaxi: Maximum power consumption of Hiin [W];

n: Number of PMs.

The set of VMs requested by customers at each discrete

time tis represented as a matrix V(t)∈Rm×5, as presented

in (2). In this work, each VM Vjrequires 3 diﬀerent virtual

resources (V r1-V r3): CPU [ECU], RAM memory [GB] and

network capacity [Mbps]. It is important to mention that the

notation could represent any other set of virtual resources

such as: Block Storage or even Graphics Processing Unit

(GPU). An economical revenue Rj[$] is associated to each

VM Vjas well as a priority level represented as a Service

Level Agreement (SLA). The VMs try to lease the requested

virtual resources for a ﬁxed period of discrete time.

V(t) =

V r1,1V r2,1V r3,1SLA1R1

. . . . . . . . . . . . . . .

V r1,m(t)V r2,m(t)V r3,m(t)SLAm(t)Rm(t)

(2)

where:

V r1,j : Processing requirements of Vjin [ECU];

V r2,j : Memory requirements of Vjin [GB];

V r3,j : Network requirements of Vjin [Mbps];

Rj: Economical revenue for attending Vjin [$];

SLAj: SLA of Vj, where SLAj∈ {1,2,...,s}, being

sthe highest priority level;

m(t): Number of VMs at each discrete time t, then

m(t)∈ {1,...,mmax};

mmax: Maximum number of acceptable VMs.

Once a VM Vjis powered-oﬀ by the customer, its virtual

resources are released, so the provider can reuse them. For

simplicity, in what follows the index jis not reused; there-

fore, for this work Vjis not a function of time.

The current placement of VMs into PMs (P(t)) is also

considered as an input data, taking into account that the

placement of requested VMs is performed incrementally at

each discrete time t. The placement at each discrete time t

is represented as a matrix P(t)∈Rm(t)×n, deﬁned as:

P(t) =

P1,1(t)P1,2(t). . . P1,n(t)

. . . . . . . . . . . .

Pm(t),1(t)Pm(t),2(t). . . Pm(t),n (t)

(3)

where:

Pj,i(t)∈ {0,1}: Indicates if Vjis allocated (Pj,i(t) = 1) or

not (Pj,i(t) = 0) for execution on a PM Hi

(i.e., Pj,i(t) : Vj→Hi) at a discrete time t.

2.2 Output

The result of the proposed VMP problem at each discrete

time tis a new placement for the next time instant (P(t+1)).

This is represented by a matrix P(t+ 1) ∈Rm(t+1)×n.

P(t+ 1) =

P1,1. . . P1,n

. . . . . . . . .

Pm(t+1),1. . . Pm(t+1),n

(4)

where:

Pj,i(t+ 1) ∈ {0,1}: Indicates if Vjis allocated (Pj,i (t) = 1)

or not (Pj,i(t) = 0) for execution on a PM

Hi(i.e., Pj,i(t) : Vj→Hi) at instant t+ 1.

2.3 Constraints

2.3.1 Constraint 1: Unique Placement of VMs

A VM Vjshould be located to run on a single PM Hi

or alternatively for each Vjsuch that SLAj< s, it could

not be located into any PM. Consequently, this placement

constraint is expressed as:

n

X

i=1

Pj,i(t)≤1∀j∈ {1,...,m(t)}(5)

2.3.2 Constraint 2: Assure SLA Provisioning

A VM Vjwith the highest level of SLA (i.e. SLAj=s)

must be mandatorily allocated to run on a PM Hi. Conse-

quently, this constraint is mathematically expressed as:

n

X

i=1

Pj,i(t) = 1 ∀jsuch that SLAj=s(6)

2.3.3 Constraints 3-5: Physical Resources of PMs

A PM Himust have suﬃcient available resources to meet

the requirements of all VMs Vjthat are allocated to run on

Hi. Consequently, these constraints can be formulated as:

m(t)

X

j=1

V r1,j ×Pj,i(t)≤P r1,i (7)

m(t)

X

j=1

V r2,j ×Pj,i(t)≤P r2,i (8)

m(t)

X

j=1

V r3,j ×Pj,i(t)≤P r3,i (9)

∀i∈ {1, ..., n},i.e. for all PM Hi.

Physical resources are considered as resources available for

VMs, without considering resources for the PM’s hypervisor.

2.4 Objective Functions

Each of the considered objective functions must be for-

mulated in a single optimization context (i.e. minimization

or maximization) and each objective function cost must be

normalized to be comparable and combinable as a single ob-

jective. This work normalizes each objective function cost

by calculating ˆ

fi(x)∈R, where 0 ≤ˆ

fi(x)≤1.

2.4.1 Power Consumption Minimization

Based on Beloglazov et al. [3], this work models the power

consumption of PMs considering a linear relationship with

the CPU utilization of PMs. This can be represented by the

sum of the power consumption of each PM Hi.

min f1(x) =

n

X

i=1

((pmaxi−pmini)×Ur1,i(t) +pmini)×Yi(t)

(10)

where:

f1(x): Total power consumption of PMs;

pmini: Minimum power consumption of a PM Hi. As

suggested in [3], pmini≈pmaxi∗0.6;

Ur1,i(t): Utilization ratio of resource 1 (in this case CPU)

by Hiat instant t;

Yi(t)∈ {0,1}: Indicates if Hiis turned on (Yi(t) = 1) or

not (Yi(t) = 0) at instant t.

2.4.2 Economical Revenue Maximization

For IaaS customers, cloud computing resources often ap-

pear to be unlimited and can be provisioned in any quantity

at any required time t[14]. Consequently, this work con-

sidered a federated-cloud deployment architecture, where a

main provider may attend requested resources that are not

able to be provided (e.g. a workload peak) by transparently

leasing low-price resources at a federated provider [8].

In this work, the maximization of the total economical

revenue that a provider receives for attending the require-

ments of its customers is achieved by minimizing the total

costs of leasing resources from alternative datacenters of the

cloud federation. Equation (11) represents the mentioned

leasing costs, deﬁned by the sum of the total costs of leasing

each VM Vjthat is eﬀectively allocated for execution on any

PM of an alternative datacenter of the cloud federation.

A provider must oﬀer its idle resources to the cloud fede-

ration at lower prices than oﬀered to customers in the ac-

tual cloud market. The pricing scheme may depend on the

particular agreement between providers of the cloud federa-

tion [8]. Consequently, this work considers that the main

provider may lease requested resources (that are not able to

provide) from the cloud federation at 70% ( ˆ

Xj= 0.7) of its

price in markets (Rj). This objective may be formulated as:

min f2(x) =

m(t)

X

j=1

(Rj×Xj(t)×ˆ

Xj) (11)

where:

f2(x): Total costs for not allocating VMs in the main

provider;

Xj(t)∈ {0,1}: Indicates if Vjis allocated for execution on

a PM (Xj(t) = 1) or not (Xj(t) = 0) at instant t;

ˆ

Xj: Indicates if Vjis allocated on the main provider

(ˆ

Xj= 0) or on an alternative datacenter of the

cloud federation ( ˆ

Xj= 0.7).

It is important to note that ˆ

Xjis not a function of time.

The decision of locating a VM Vjon a federated provider is

considered only in the placement process, with no possible

migrations between providers. The value of ˆ

Xjdepends on

the agreement celebrated between federated providers.

2.4.3 Quality of Service Maximization

In this work, the quality of service (QoS) maximization

proposes the location of the maximum number of VMs with

the highest level of priority associated to the SLA prior to

VMs with smaller SLA. In case the main provider allocates

a VM in a federated provider the QoS is considered as 0. In

order to evaluate this objective in a minimization context,

the total of SLA violations is minimized and formulated as:

min f3(x) =

m(t)

X

j=1

(ˆ

CSLAj×SLAj×Xj(t)×ˆ

Yj) (12)

where:

f3(x): Total SLA violations ﬁgure for a given placement;

ˆ

C: Constant, large enough to prioritize services with

a larger SLA over the ones with a lower SLA;

ˆ

Yj∈ {0,1}: Indicates if Vjis allocated for its execution on

the main provider ( ˆ

Yj= 0) or on an alternative

datacenter of the cloud federation ( ˆ

Yj= 1).

2.4.4 Resources Utilization Maximization

An eﬃcient utilization of resources is a relevant manage-

ment challenge to be addressed by IaaS providers. This work

proposes the resource utilization maximization strategy by

minimizing the average ratio of wasted resources on each

PM Hi(i.e. resources that are not allocated to any VM Vj).

This objective function is formulated in Equation (13).

min f4(x) = Pn

i=1 1−Ur1,i(t) + · · · +U rr,i(t)

r×Yi(t)

Pn

i=1 Yi(t)

(13)

where:

f4(x): Average ratio of wasted resources;

Ur1,i(t): Utilization ratio of resource 1 (in this case CPU)

by Hiat instant t;

Urr,i(t): Utilization ratio of resource r(any resource) by

Hiat instant t;

r: Number of considered resources. In this paper 3:

CPU, RAM memory and network capacity;

The following sub-section summarizes the main conside-

rations taken into account to combine the four presented

objective functions into a single objective.

2.5 Scalarization Method

Experimental results presented in [9] recommend the com-

bination of many normalized objective functions into a single

objective (e.g. minimum distance to origin) for IaaS envi-

ronments. This work considers a weighted sum method to

combine many objectives into a single objective.

As previously mentioned, each objective function cost must

is normalized to be comparable and combinable as a single

objective. This work normalizes each objective function cost

by calculating ˆ

fi(x)∈R, where 0 ≤ˆ

fi(x)≤1, as deﬁned in

[16]:

ˆ

fi(x) = fi(x)−fi(x)min

fi(x)max −fi(x)min

(14)

where:

ˆ

fi(x): Normalized cost of objective function fi(x);

fi(x): Cost of objective function fi(x);

fi(x)min: Minimum possible cost for fi(x);

fi(x)max: Maximum possible cost for fi(x).

Finally, the four previously presented normalized objec-

tive functions are combined into a single objective conside-

ring a weighted sum method, expressed as follows:

min f(x) =

q

X

i=1

ˆ

fi(x)×wi(15)

where:

f(x): Single objective function combining each fi(x);

ˆ

fi(x): Normalized cost of objective function fi(x);

wi: Weight of importance associated to fi(x);

q: Number of objective functions. In this case 4.

In this work, values of each weight wiassociated to an ob-

jective function fi(x) were obtained empirically by analyzing

a large number of experiments to be presented in Section 4.

3. EVALUATED ALGORITHMS

To analyze alternatives to solve the formulated VMP pro-

blem (see Section 2), an experimental comparison of ﬁve

diﬀerent online deterministic heuristics against an oﬄine

memetic algorithm with migration of VMs was performed.

This section summarizes the six diﬀerent algorithms conside-

red in the experimental comparison presented in this work.

3.1 A1: First-Fit (FF)

In the First-Fit (FF) algorithm, requested VMs Vjare

allocated on the ﬁrst PM Hiwith available resources (see

Section 2.3). Interested readers can refer to [6] for details

on FF, BF and WF algorithms applied to VMP problems.

3.2 A2: Best-Fit (BF)

The Best-Fit (BF) algorithm, allocates requested VMs Vj

on the ﬁrst PM Hiwith available capacity from a sorted list

of PMs in increasing order by a score associated to each PM.

The score of a PM Hiis determined as a sum of its ratios

of unutilization of resources, as detailed in [6].

3.3 A3: Worst-Fit (WF)

In the Worst-Fit (WF) algorithm, VMs Vjare allocated

on the ﬁrst available PM Hiof a decreasingly ordered list of

PMs based on the score of the PM, inversely to the operation

of the BF algorithm. For more details, refer to [6].

3.4 A4: First-Fit Decreasing (FFD)

The First-Fit Decreasing (FFD) algorithm operates simi-

larly to the previously presented FF algorithm. The main

diﬀerence with the FF algorithm is that FFD algorithm sorts

the list of requested VMs Vjin decreasing order by requested

CPU resources, as described in [7].

3.5 A5: Best-Fit Decreasing (BFD)

The Best-Fit Decreasing (BFD) algorithm has a similar

behavior to the BF algorithm. The main diﬀerence between

the BF algorithm and the BFD one is that the BFD sorts

the list of requested VMs Vjin decreasing order by requested

CPU resources. Interested readers can refer to [5] for details

on the BFD algorithm applied to VMP problems.

3.6 A6: Memetic Algorithm (MA)

A Memetic Algorithm (MA) is considered for an oﬄine

alternative to solve the formulated VMP problem taking into

account many objectives (see Section 2). This algorithm is

based on the one proposed in [9] by Ihara et al.

The considered algorithm may be classiﬁed as a meta-

heuristic, following an evolutionary process to ﬁnd appro-

priate solutions. Basically, the considered MA follows this

evolutionary behavior: solutions are selected from an evo-

lutionary set of solutions (or population). Crossover and

mutation operators are applied as usual, and eventually so-

lutions are repaired, as there may be unfeasible solutions.

Improvements of solutions of the evolutionary population

may be generated using local optimization operators. Next,

a new evolutionary population is selected from the union of

the (until that generation) best population and the set of

improved solutions. The evolutionary process is repeated

until the algorithm meets a stopping criterion (such as a

maximum number of generations), returning from the evolu-

tionary population the solution with minimum cost of f(x).

4. EXPERIMENTAL COMPARISON

In this work, an experimental comparison of ﬁve of the

most studied online deterministic heuristics was performed

considering several experimental workloads. The quality of

solutions obtained by the evaluated online heuristics was

compared to the average performance (in ten runs) of an

oﬄine MA that has complete knowledge of future VM re-

quests and the ability of migrating VMs between PMs.

4.1 Experimental Environment

The six evaluated algorithms were implemented using ANSI

C programming language. The source code is available on-

line1, as well as all the considered experimental data. Expe-

riments were performed on a Linux Operating System with

an Intel Core i7 2.3 GHz CPU and 12 GB of RAM memory.

Physical resources (matrix H) represent an homogeneous

IaaS cloud composed by 10 PMs with the following speci-

ﬁcations: 8 [ECU] of CPU, 10 [GB] of RAM memory, 780

[Mbps] of network capacity and 960 [W] of maximum power

consumption (see Section 2.1 for notation details).

In this work, requested virtual resources (matrix V(t))

were considered using four diﬀerent workload traces genera-

ted using a Workload Trace Generator for provider-oriented

VMP problems [15] available for research purposes2.

1http://github.com/DynamicVMP/vmpCompetitiveAnalysis

2http://wtg.cba.com.py

Table 1: Considered Data for Generated Experi-

mental Workload Traces (W1to W4). Details in [15].

Parameter Description Input Data

1. Environment No overbooking,

No elasticity

2. Workload Trace Duration [t] 100

3. Number of Cloud Datacenters 1

4. Number of Cloud Services 100

5. Number of VMs per Service 1

6. VMs creation time [t]

W1:Poisson(λ= 10)

W2:Poisson(λ= 50)

W3:Poisson(λ= 70)

W4:Uniform(0,100)

7. CPU Resources [ECU] Uniform(1,8)

8. RAM Resources [GB] Uniform(1,8)

9. Network Capacity [Mbps] Uniform(10,1000)

10. Revenue of VMs (Rj) [$] Uniform(0.1,1.5)

11. SLA Level of VMs (SLAj)Uniform(1,5)

Each considered workload trace simulates customers re-

quests to allocate a set of 100 VMs following diﬀerent Pro-

bability Distribution Functions (PDFs) for the VM request

arrivals. Three workload traces follow a Poisson PDF with

diﬀerent expected values (λ): (W1): λ= 10, (W2): λ= 50

and (W3): λ= 70. In this case, diﬀerent values of λmay

represent workload peaks at diﬀerent time instants t(see

Figure 1). The fourth workload trace (W4) follows an Uni-

form PDF, representing a stable workload of VM requests.

All parameters considered for generating the experimental

workload traces are presented in Table 1.

In order to eﬀectively analyze the described federated-

cloud deployment architecture and provider-oriented VMP

formulation considering many objectives, experimental work-

load traces requested more virtual resources than the availa-

ble ones in the considered main cloud computing datacenter.

Experiments could be summarized as: For each conside-

red workload trace (W1to W4), one run of the following

deterministic heuristics was performed in an online context:

(1) FF, (2) BF, (3) WF, (4) FFD and (5) BFD. Considering

the randomness of the MA compared against the previously

mentioned heuristics, ten runs of the mentioned algorithm

were performed. The average values of the ten runs were

considered for the experimental comparison, as summarized

in Table 2, where the oﬄine MA clearly outperforms all 5

online heuristics as expected, given that it uses complete

knownledge of VM requests and migration of VMs to opti-

mize the objective function presented in Equation 15.

4.2 Objective Functions Weights

To determine appropriate values for the weights wiasso-

ciated to each objective function fi(x), an exploration of the

VMP problem domain was performed. In this context, 1000

feasible solutions (x1to x1000) were randomly generated by

the MA (A6) considering: Has described in Section 4.1,

V(t) as presented in entorno00-1 (a benchmark available

online2), highest priority of VMs s= 4 and ˆ

C= 1000.

Obtained values of each objective function fi(x) were nor-

malized in ˆ

fi(x), as described in Section 2.5. Consequently,

each weight wiwas deﬁned as:

0 10 20 30 40 50 60 70 80 90 100

0

3

6

9

12

15

Time (t)

Number of created VMs

W1: Poisson λ= 10

W2: Poisson λ= 50

W3: Poisson λ= 70

W4: Uniform

Figure 1: Experimental Workload Traces: Number of created VMs as a function of time t.

wi=1000

P1000

k=1 ˆ

fi(xk)(16)

resulting:

w1: 1.3903; w2: 2.1379; w3: 2.7393; w4: 1.4586.

4.3 Comparison of Online Heuristics

This section summarizes the main ﬁndings obtained in the

experimental comparison of algorithms for the VMP formu-

lation with many objectives presented in Section 2.

The main goal of the experimental comparison presented

in this section is to deﬁne which heuristics are most suitable

for solving online formulations of provider-oriented VMP

problems in federated-clouds considering many objectives.

Table 2 summarizes the costs of objective function f(x),

that combines the four considered objectives (see Section

2.5), summarizing performed experiments. It worth noting

that according to the experimental results presented in Ta-

ble 2, there is no evaluated heuristic that outperforms all

other alternatives in all experimental workload traces. Con-

sequently, Table 2 also presents the average costs of ob jec-

tive function f(x) in the 4 experimental workloads as well

as the corresponding ranking. It seems that FFD algorithm

is the best heuristic amount the ﬁve considered ones closely

followed by BFD (logically, MA is ﬁrst in this ranking).

To better understand the presented comparison, Table 3

details the average normalized costs of each ob jective func-

tion f1(x) to f4(x), to analyze particular preferences of IaaS

providers for the optimization of the considered objectives

(e.g. power consumption could be more important in hours

where the electricity price is higher).

Based on the information presented in Tables 2 and 3,

the Main Findings (MF) of the experimental comparison

performed in this paper are summarized as follows:

MF1: There is no evaluated heuristic that can clearly be

considered as the best alternative for all objective functions,

considered simultaneously.

Additionally, none of the evaluated heuristics performed equ-

ally well in all 4 experimental workloads (see Table 2). Con-

sequently, an heuristic performing good enough in average

when considering diﬀerent types of workloads could be suf-

ﬁciently convenient.

A more detailed evaluation could be performed to obtain in-

formation for conjuntural preferences of IaaS providers (see

Table 3, where the best heuristic for each objective function

and each workload trace is highlighted in the last column).

MF2: FFD heuristic (A4) was the algorithm that outper-

forms all other evaluated heuristics taking into account ave-

rage results in performed experiments.

According to the average performance of each evaluated

heuristic (see Table 2), the following ranking was built: (1st)

FDD (A4), (2nd) BFD (A5), (3th) BF (A2), (4th) FF (A1),

and (5th) WF (A3), where BFD follows very close the ave-

rage performance of FDD (with a diﬀerence of 1.75%).

MF3: WF algorithm (A3) is suggested for workloads that

can be considered stable (i.e. no workload peaks).

For experimental workload trace W4that considers an Uni-

form PDF for VM request arrivals, A3 clearly presented the

best performance obtaining minimum average results in Ta-

ble 2. It also performed as the best algorithm for 3 out of

the 4 objectives, considering only W4(see Table 3).

MF4: The WF (A3), BFD (A5), FFD (A4) and BFD

(A5) algorithms are recommended for f1(x),f2(x),f3(x)

and f4(x)objective functions respectively, when there is a

preferred objective function (lexicographic order).

The BFD algorithm (A5) performed as the best algorithm on

average for f2(x) and f4(x). For f1(x) the best alternative

seems to be A3 while A4 could be considered as the best for

f3(x) (see Table 3).

MF5: As expected, the oﬄine MA (A6) outperformed all

evaluated online heuristics in all experimental workloads.

An oﬄine MA was compared to the ﬁve evaluated heuristics

to experimentally demonstrate that online decisions made

along a simulation aﬀects the quality of solutions. Clearly,

oﬄine algorithms such as the considered MA present a sub-

stantial advantage over online heuristics when considering

the quality of solutions for the objective function f(x) (see

Table 2). This advantage is presented for the following two

main reasons: (1) oﬄine algorithms have a complete knowl-

edge of future events of a problem instance and (2) only A6

considered migrations of VMs between PMs in the compa-

rison, reconﬁguring the placement when convenient.

Table 2: Objective Function Costs of Evaluated Algorithms.

Workload

Algorithm A1: FF A2: BF A3: WF A4: FFD A5: BFD A6: MA

W1: Poisson λ= 10 3.2927 3.3098 3.5250 3.0205 3.1392 2.6096

W2: Poisson λ= 50 2.4602 2.5112 2.4811 2.4602 2.4555 2.0001

W3: Poisson λ= 70 1.7054 1.6458 1.7054 1.6458 1.6458 1.3588

W4: Uniform 3.1875 3.1556 3.0489 3.0907 3.1556 2.3420

Average 2.6615 2.6556 2.6901 2.5543 2.5990 2.0776

Ranking 5th 4th 6th 2nd 3th 1st

Table 3: Objective Functions Costs of Evaluated Heuristics.

Workload fi(x)

Heuristic A1: FF A2: BF A3: WF A4: FFD A5: BFD Best Heuristic

W1

f1(x) 0.9240 0.9107 0.8929 0.9255 0.9208 A3

f2(x) 0.0372 0.0373 0.0391 0.0359 0.0359 A4,A5

f3(x) 0.6104 0.6139 0.6670 0.5101 0.5611 A4

f4(x) 0.1756 0.1933 0.2556 0.1778 0.1679 A5

W2

f1(x) 0.5900 0.5756 0.5612 0.5900 0.5903 A3

f2(x)0.0404 0.0406 0.0420 0.0404 0.0404 A4,A5

f3(x) 0.4790 0.4975 0.4715 0.4790 0.4790 A3

f4(x) 0.1652 0.1789 0.2189 0.1652 0.1618 A5

W3

f1(x)0.4146 0.4198 0.4203 0.4198 0.4198 A1

f2(x) 0.0243 0.0235 0.0245 0.0235 0.0235 A2,A4,A5

f3(x) 0.3155 0.3003 0.3103 0.3003 0.3003 A2,A4,A5

f4(x) 0.1457 0.1296 0.1497 0.1296 0.1296 A2,A4,A5

W4

f1(x) 0.8549 0.8544 0.8694 0.8598 0.8544 A2,A5

f2(x) 0.0375 0.0364 0.0356 0.0371 0.0364 A3

f3(x) 0.5311 0.5283 0.4884 0.4931 0.5283 A3

f4(x) 0.3177 0.3034 0.2919 0.3188 0.3034 A3

Average

f1(x) 0.6959 0.6901 0.6859 0.6988 0.6963 A3

f2(x) 0.0349 0.0345 0.0353 0.0343 0.0341 A5

f3(x) 0.4841 0.4851 0.4844 0.4457 0.4672 A4

f4(x) 0.2011 0.2014 0.2291 0.1979 0.1907 A5

Table 4: VMs Migration Overhead for MA (A6).

Workload

Migration # of VMs Memory in [GB]

Poisson λ= 10 1493.7 4865.9

Poisson λ= 50 740.0 2775.1

Poisson λ= 70 643.6 2415.5

Uniform 1416.7 5329.8

Having a complete knowledge of future VMs requests is con-

sidered unrealistic for IaaS environments [4]. Consequently,

online algorithms, such as the heuristics evaluated in this

work, are a good alternative for VMP problems in IaaS en-

vironments. The evaluated oﬄine MA (A6) improves the

quality of solutions with migrations of VMs between PMs,

at expense of costs associated to placement reconﬁgurations,

as seen in Table 4 where the average VMs migration over-

head of each workload is presented as: (1) total number of

VM migrations and (2) total RAM memory migrated.

MF6: As expected, the execution time of all online heuris-

tics are considerably shorter than the oﬄine alternative.

Considering the low complexity of the evaluated heuristics,

these algorithms are able to ﬁnd promising solutions in short

periods of time (a main concern for IaaS providers solving

VMP problems) at the expense of not exploring solutions

that can potentially result in better quality solutions.

Clearly, any oﬄine alternative that intelligently explore a

large space of feasible solutions will result in higher exe-

cution times, mainly considering the exponential computa-

tional complexity of the VMP problem.

5. CONCLUSIONS AND FUTURE WORKS

This work presented a ﬁrst experimental comparison of

ﬁve diﬀerent online heuristics for solving a proposed provider-

oriented VMP problem for the optimization of the following

objectives: (1) power consumption, (2) economical revenue,

(3) quality of service, and (4) resource utilization. The qual-

ity of solutions obtained by online heuristics was compared

to the best performance of an oﬄine MA that have complete

knowledge of future VM requests and the capacity of VM

migration. A previously studied MA [9] was implemented ta-

king into account that obtaining optimal solutions for large

instances of the problem could result impracticable [11].

Considering the evaluation of the mentioned algorithms

solving several experimental workload traces, six main ﬁn-

dings (MF1 to MF6) were identiﬁed (see Section 4.3).

The main advantage of the oﬄine MA (A6) over experi-

mentally evaluated online heuristics (A1 to A5) is a better

quality of solutions, as summarized in Table 2.

Unfortunately, oﬄine alternatives such as A6 are not ap-

propriate for dynamic environments of VMP problems for

IaaS providers, where future VM requests are unknown and

placement should be solved in a very short time.

In order to improve the quality of solutions obtained by

the evaluated heuristics, the VMP problem could be formu-

lated as a two-phase optimization problem. In this context,

VMP problems with many objectives for an IaaS could be

decomposed into two diﬀerent sub-problems: (1) incremen-

tal VMP (iVMP) and (2) VMP reconﬁguration (VMPr) [18].

A two-phase optimization strategy can be considered, com-

bining both online (iVMP) and oﬄine (VMPr) algorithms

for solving each considered VMP sub-problem. The iVMP

problem is considered for dynamic arriving requests when

VMs should be created and removed at runtime. Conse-

quently, this sub-problem should be formulated as an online

problem and solved in a short period of time, where the stu-

died heuristics could be appropriate. On the contrary, the

VMPr problem is considered for improving the quality of

solutions obtained by the iVMP, considering placement re-

conﬁgurations through migration of VMs. This sub-problem

could be formulated oﬄine, where alternative solution tech-

niques (e.g. meta-heuristics) could be appropriate.

Authors of this work are already working on the men-

tioned two-phase optimization (iVMP + VMPr), conside-

ring more realistic representation of cloud infrastructures

and more complex VMP environments with both overboo-

king and elasticity considerations [15], as well as extending

experiments to more workload traces including real-world

experimental workloads [4]. Finally, future directions in-

clude exploring research questions as: (1) when the VMPr

problem should be triggered? and (2) what should the VMPr

problem do with requests arriving during reconﬁguration?

6. REFERENCES

[1] A. Anand, J. Lakshmi, and S. Nandy. Virtual machine

placement optimization supporting performance SLAs.

In Cloud Computing Technology and Science

(CloudCom), 2013 IEEE 5th International Conference

on, volume 1, pages 298–305. IEEE, 2013.

[2] L. A. Barroso and U. H¨

olzle. The case for

energy-proportional computing. IEEE computer,

40(12):33–37, 2007.

[3] A. Beloglazov, J. Abawajy, and R. Buyya.

Energy-aware resource allocation heuristics for

eﬃcient management of data centers for cloud

computing. Future Generation Computer Systems,

28(5):755–768, 2012.

[4] A. Beloglazov and R. Buyya. Optimal online

deterministic algorithms and adaptive heuristics for

energy and performance eﬃcient dynamic

consolidation of virtual machines in cloud data

centers. Concurrency and Computation: Practice and

Experience, 24(13):1397–1420, 2012.

[5] D. Dong and J. Herbert. Energy eﬃcient VM

placement supported by data analytic service. In

Cluster, Cloud and Grid Computing (CCGrid), 2013

13th IEEE/ACM International Symposium on, pages

648–655. IEEE, 2013.

[6] S. Fang, R. Kanagavelu, B.-S. Lee, C. H. Foh, and

K. M. M. Aung. Power-eﬃcient virtual machine

placement and migration in data centers. In Green

Computing and Communications (GreenCom), 2013

IEEE and Internet of Things (iThings/CPSCom),

IEEE International Conference on and IEEE Cyber,

Physical and Social Computing, pages 1408–1413.

IEEE, 2013.

[7] T. Ferreto, C. A. De Rose, and H.-U. Heiss. Maximum

migration time guarantees in dynamic server

consolidation for virtualized data centers. In Euro-Par

Parallel Processing, pages 443–454. Springer, 2011.

[8] M. Gahlawat and P. Sharma. Survey of virtual

machine placement in federated clouds. In Advance

Computing Conference (IACC), 2014 IEEE

International, pages 735–738, Feb 2014.

[9] D. Ihara, F. L´opez-Pires, and B. Bar´an.

Many-objective virtual machine placement for

dynamic environments. In Proceedings of the 2015

IEEE/ACM 8th International Conference on Utility

and Cloud Computing. IEEE Computer Society, 2015.

[10] H. Jin, D. Pan, J. Xu, and N. Pissinou. Eﬃcient VM

placement with multiple deterministic and stochastic

resources in data centers. In Global Communications

Conference (GLOBECOM), 2012 IEEE, pages

2505–2510. IEEE, 2012.

[11] F. L´opez-Pires and B. Bar´an. Multi-objective virtual

machine placement with service level agreement: A

memetic algorithm approach. In Proceedings of the

2013 IEEE/ACM 6th International Conference on

Utility and Cloud Computing, pages 203–210. IEEE

Computer Society, 2013.

[12] F. L´opez-Pires and B. Bar´an. A many-objective

optimization framework for virtualized datacenters. In

Proceedings of the 2015 5th International Conference

on Cloud Computing and Service Science, 2015.

[13] F. L´opez-Pires and B. Bar´an. A virtual machine

placement taxonomy. In Proceedings of the 2015

IEEE/ACM 15th International Symposium on

Cluster, Cloud and Grid Computing. IEEE Computer

Society, 2015.

[14] P. Mell and T. Grance. The NIST deﬁnition of cloud

computing. National Institute of Standards and

Technology, 53(6):50, 2009.

[15] J. Ortigoza, F. L´opez Pires, and B. Bar´an. A

taxonomy on dynamic environments for

provider-oriented virtual machine placement. In

Proceedings of the 2016 IEEE 4th International

Conference on Cloud Engineering. IEEE Computer

Society, 2016.

[16] S. B. Salem, M. Fakhfakh, D. S. Masmoudi,

M. Loulou, P. Loumeau, and N. Masmoudi. A high

performances cmos ccii and high frequency

applications. Analog Integrated Circuits and Signal

Processing, 49(1):71–78, 2006.

[17] B. Speitkamp and M. Bichler. A mathematical

programming approach for server consolidation

problems in virtualized data centers. Services

Computing, IEEE Transactions on, 3(4):266–278,

2010.

[18] Q. Zheng, R. Li, X. Li, N. Shah, J. Zhang, F. Tian,

K.-M. Chao, and J. Li. Virtual machine consolidated

placement based on multi-objective

biogeography-based optimization. Future Generation

Computer Systems, 2015.