Content uploaded by Fabio Lopez-Pires

Author content

All content in this area was uploaded by Fabio Lopez-Pires on May 24, 2021

Content may be subject to copyright.

Cloud Computing Resource Allocation Taxonomies

Fabio López-Pires

Itaipu Technological Park, National University of Asunción - Paraguay

fabio.lopez@pti.org.py

Benjamín Barán

National University of Asunción - Paraguay

bbaran@pol.una.py

Abstract Cloud computing datacenters dynamically provide millions of vir-

tual machines in actual cloud computing markets. Several challenging problems

have to be addressed towards an efﬁcient resource management of these cloud

computing infrastructures. In the context of resource allocation, Virtual Machine

Placement (VMP) is one of the most studied problems with several possible

formulations and a large number of existing optimization criteria, considering so-

lutions with high economical and ecological impact. Based on systematic reviews

of the VMP literature, this work presents novel taxonomies in order to: (1) un-

derstand different possible environments where a VMP problem could be studied

from both provider and broker perspectives in different deployment architectures,

(2) identify existing approaches for the formulation and resolution of the VMP

as an optimization problem and (3) present a detailed view of the VMP problem,

identifying research opportunities to further advance in this area.

Keywords: Resource Allocation; Virtual Machine Placement; Cloud Computing;

Cloud Brokerage; Cloud Infrastructure; Optimization; Datacenters; Taxonomy.

1 Introduction

Cloud computing datacenters deliver infrastructure (IaaS), platform (PaaS) and software

(SaaS) as services, available to end users in a pay-as-you-go basis [1]. In this context,

a signiﬁcant number of research challenges for delivering computational resources as an

utility have already been identiﬁed [2]. Achieving an efﬁcient resource management in cloud

computing datacenters could be considered one of the most relevant challenges, including

important topics such as: resource allocation, resource provisioning, resource mapping and

resource adaptation [3]. Additionally, admission control and proactive elasticity could be

also considered relevant topics related to resource management in cloud datacenters [4].

The present work focuses on resource allocation, speciﬁcally in one of the most studied

problems for resource allocation in cloud computing datacenters: the process of selecting

which virtual machines (VMs) should be hosted at each physical machine (PM) of a cloud

Copyright © 2009 Inderscience Enterprises Ltd.

2Fabio López-Pires and Benjamín Barán

datacenter, commonly known as Virtual Machine Placement (VMP) problem. Several re-

search articles demonstrated that solving the VMP problem for efﬁcient allocation of cloud

resources could signiﬁcantly improve energy-efﬁciency, quality of service (QoS) and carbon

dioxide emissions; all of them with signiﬁcant economical and ecological impact [5, 6, 7].

A conceptual cloud computing architecture considering the most studied problems in

cloud computing resource allocation is presented in [4]. According to the proposed archi-

tecture, the VMP is one of the main problems in the mentioned context, where placement of

admitted services could be solved considering different requirements and several criteria.

Beloglazov and Buyya proposed in [8] four different sub-problems for resource allo-

cation in cloud datacenters: (1) determining when a PM is overloaded, requiring migration

of VMs from this PM; (2) determining when a PM is underloaded, requiring migration of

all VMs from this PM and switching the PM to sleep mode; (3) selecting VMs to migrate

from an overloaded PM; and (4) ﬁnding a new placement of the VMs selected for migration

considering the overloaded and underloaded PMs.

The VMP problem has been extensively studied in cloud computing literature and several

surveys have already been presented. Existing surveys focused on speciﬁc issues such as:

(1) energy-efﬁcient techniques applied to the problem [5, 9], (2) particular deployment

architectures where the VMP problem is applied, as federated clouds [10], and (3) methods

for comparing performance of placement algorithms in large on-demand clouds [11].

The above mentioned surveys and research articles focused into very speciﬁc issues re-

lated to the VMP problem. Consequently, López-Pires and Barán proposed in [12] a general

and extensive study of a large part of the VMP literature including more than 80 studied

articles [13], presenting a wide analysis of the existing approaches for the formulation and

resolution of the VMP as an optimization problem. Additionally, a novel taxonomy was

proposed in [12] for the classiﬁcation of the studied articles by the main following criteria:

(1) optimization approach, (2) objective function and (3) solution technique.

The present work extends [12], complementing the proposed taxonomy with novel

taxonomies and presenting a detailed view of the existing approaches as well as several

possible research opportunities to further advance in this research area. The presented

taxonomies could guide interested readers to: (1) understand different possible environments

where a VMP problem could be studied, considering both provider and broker perspectives

in different deployment architectures, (2) identify existing approaches for the formulation

and resolution of the VMP as an optimization problem and (3) present a detailed view of the

VMP problem, identifying research opportunities to further advance in this research area.

The remainder of this paper is organized as follows: Section 2 presents a VMP pro-

blem environment taxonomy for the classiﬁcation of related articles by: (1) orientation, (2)

deployment architecture and (3) types of formulation. Section 3 presents a VMP problem

formulation and resolution taxonomy considering the following criteria: (1) optimization

approach, (2) objective function and (3) solution technique. Section 4 details identiﬁed

research opportunities on this research area, while conclusions are left to Section 5.

2 VMP Problem Environment Taxonomy

Depending on the particular environment where a VMP problem will be studied, several

different considerations should be taken into account before considering a particular for-

mulation or technique for the resolution of the considered VMP problem. In this context,

Cloud Computing Resource Allocation Taxonomies 3

different possible environments could be identiﬁed by classifying research articles in the

VMP literature by: (1) orientation, (2) deployment architecture and (3) type of formulation.

First, a VMP problem could be studied from both provider or broker perspectives (see

Section 2.1). A provider-oriented VMP problem could be studied considering one of the fo-

llowing deployment architectures: single-cloud, distributed-cloud or federated-cloud, while

a broker-oriented VMP problem could be studied considering a multi-cloud deployment ar-

chitecture (see Section 2.2). Additionally, both provider-oriented and broker-oriented VMP

problems could be studied considering two different types of formulation: ofﬂine or online

formulations (see Section 2.3).

For a complete understanding of the possible environments where a VMP problem

could be studied, considering both provider and broker perspectives in different deployment

architectures, Figure 2.1 presents the taxonomy described in this section including relevant

references from the studied VMP literature [13].

The following subsections present a detailed description of the mentioned classiﬁcation

criteria, including relevant deﬁnitions for a complete understanding of the VMP problem.

2.1 Orientations: Provider-oriented or Broker-oriented

Deﬁnition 2.1: A provider-oriented VMP problem is the process of selecting which VMs

should be hosted at each PM of a cloud computing infrastructure.

Resource allocation in cloud computing datacenters is a main concern for Cloud Service

Providers (CSPs). According to [12], the VMP problem is mainly formulated from this

perspective. It should be mentioned that in a provider-oriented VMP, a Cloud Service Tenant

(CST) cannot decide which PMs will host the requested VMs. In the specialized literature,

this particular problem is also known as Virtual Machine Allocation (VMA) problem [14].

Deﬁnition 2.2: A broker-oriented VMP problem is the process of selecting which VMs

should be hosted at each Cloud Service Provider (CSP) of a cloud computing market.

Considering that the number of CSPs has been rapidly increased and nowadays there

are different pricing schemes, VM offers and features, it is difﬁcult for CSTs to search

for the most convenient option in cloud computing markets and decide which CSP is the

most convenient to host the requested resources. Consequently, a VMP problem can also

be formulated from a broker perspective who is responsible of helping CSTs to ﬁnd good

allocation deals. In the specialized literature, this particular problem is also known as Cloud

Resource Brokerage (CRB) problem [15].

2.2 Deployment Architectures

Different deployment architectures can be considered for a VMP problem, depending on

the number of cloud computing datacenters associated to the problem instance and the type

of interconnection of these datacenters.

A provider-oriented VMP problem could be studied considering one of the following de-

ployment architectures: single-cloud, distributed-cloud or federated-cloud, while a broker-

oriented VMP problem could be studied in a multi-cloud deployment architecture.

4Fabio López-Pires and Benjamín Barán

VMP Problem

Provider-oriented

Single-Cloud

Ofﬂine

[16]

Online

[17]

Distributed-Cloud

Ofﬂine

[18]

Online

[19]

Federated-Cloud

Ofﬂine

RO

Online

[20]

Broker-oriented

Multi-Cloud

Ofﬂine

[21]

Online

RO

Figure 2.1:VMP Problem Environment Taxonomy. Relevant references for each envi-

ronment are presented. Unexplored environments are considered Research Opportunities

(RO).

Deﬁnition 2.3: A provider-oriented VMP problem in a single-cloud deployment is the

process of selecting which VMs should be hosted at each PMs of a single cloud computing

datacenter.

In scenarios considering a single cloud computing datacenter, a CSP could formulate a

VMP problem subject to commonly studied constraints such as: unique placement of VMs,

maximum capacity of resources [16] or Service Level Agreement (SLA) compliance [6, 22].

According to [12], the single-cloud deployment architecture is the most studied scenario in

the considered literature.

Deﬁnition 2.4: A provider-oriented VMP problem in a distributed-cloud deployment is

the process of selecting which VMs should be hosted at each PMs of more than one cloud

computing datacenter owned by the same Cloud Service Provider (CSP).

For CSPs with a global infrastructure (e.g. Amazon), a single-cloud deployment ar-

chitecture could be extended to several distributed cloud computing datacenters, where

the formulation of a VMP problem may include particular constraints and considerations.

CSPs with geo-distributed cloud computing datacenters may be interested in studying a

VMP problem for the provisioning of differentiated services to world-wide tenants, such

as redundancies of services (placement of VMs in different datacenters) [23] or quality

and performance of services (placement of VMs in datacenters closer to end users) [24],

increasing the complexity of the VMP problem formulation.

Deﬁnition 2.5: A provider-oriented VMP problem in a federated-cloud deployment is the

process of selecting which VMs should be hosted at each PMs of more than one cloud com-

puting datacenter owned by different Cloud Service Providers (CSPs) in a cloud federation.

In federated clouds, CSPs with excess capacity lease resources to other CSPs in need

of temporary additional resources; consequently, particular considerations associated to a

VMP problem in this deployment architecture have to be studied [10]. For example, trading

policies are generally not the same for CSPs that form part of the same cloud federation, so

Cloud Computing Resource Allocation Taxonomies 5

a VMP problem could be studied for cost-effective selection of CSPs for workload peaks,

just to cite a simple example.

Deﬁnition 2.6: A broker-orientedVMP problem in a multi-cloud deployment is the process

of selecting which VMs should be hosted at each Cloud Service Provider (CSP) of a cloud

computing market composed by more than one CSP.

Cloud Service Brokers (CSBs) or CSTs can require VMs to be deployed in cloud com-

puting datacenters owned by different CSPs, according to particular requirements such as

disaster recovery reasons or due to legislation [25], just to cite a few.

2.3 Types of Formulation: Ofﬂine or Online

A VMP problem could be formulated as an ofﬂine problem considering the placement of

VMs into PMs (for provider-oriented VMP) or VMs into CSPs (for broker-oriented VMP)

for a given needed requirement, i.e. a static service deployment.

Deﬁnition 2.7: An ofﬂine formulation of a provider-oriented VMP problem in any possible

deployment architecture could be understood as the process of selecting which VMs should

be hosted at each PMs of the considered cloud computing datacenters for a static cloud

service deployment.

For a provider-oriented VMP, an ofﬂine formulation does not consider possible re-

locations of VMs; therefore, there is no need for migration techniques to be applied. It should

be noted that ofﬂine formulations are mostly appropriate for initial placement of VMs or

for virtualized datacenters with deployments of VMs that rarely change its conﬁguration

over time [26, 16, 27].

Deﬁnition 2.8: An ofﬂine formulation of a broker-oriented VMP problem could be under-

stood as the process of selecting which VMs should be hosted at each CSP of the considered

cloud computing market for a given (static) service deployment.

The broker-oriented VMP is mostly formulated as an ofﬂine problem, considering that

possible re-locations of VMs to different CSPs are still limited by interoperability factors.

A VMP problem could also be formulated as an online problem considering the place-

ment of VMs into PMs (provider-oriented VMP) or VMs into CSPs (broker-oriented VMP)

for a cloud service deployment with dynamic demand or parameters that change over time.

Deﬁnition 2.9: An online formulation of a provider-oriented VMP problem in any possible

deployment architecture could be understood as the process of selecting which VMs should

be hosted at each PMs of the considered cloud computing datacenters considering dynamic

demand or parameters that change over time.

It should be noted that an online formulation for a provider-oriented VMP problem could

be the most appropriate approach for resource allocation in cloud computing datacenters,

considering the on-demand model of cloud computing with dynamic resource provisioning

and dynamic workloads of modern cloud applications [8]. In this context, the authors of

this work are working on a detailed taxonomy for the classiﬁcation of possible online for-

mulations for the provider-oriented VMP, presenting an analysis of possible challenges for

6Fabio López-Pires and Benjamín Barán

providers in cloud computing environments based on the most relevant dynamic parameters

studied so far in the existing VMP literature such as: (1) resource capacities of VMs (related

to vertical elasticity), (2) number of VMs of a service (related to horizontal elasticity) and

(3) utilization of resources of VMs (related to overbooking) [28].

Deﬁnition 2.10: An online formulation of a broker-oriented VMP problem in any possi-

ble deployment architecture could be understood as the process of selecting which VMs

should be hosted at each CSPs of a given cloud computing market considering demands or

parameters that change over time.

Considering that the number of CSPs has increased rapidly, nowadays several dynamic

parameters could be considered for the broker-oriented VMP problem such as: dynamic

pricing schemes, dynamic VM offers or dynamic requirements. Research on online formula-

tions of the broker-oriented VMP problem should advance to address existing opportunities

in emerging cloud computing markets as presented in a few articles [29, 30].

3 VMP Problem Formulation and Resolution Taxonomy

Considering each possible environment where a VMP problem can be studied (see Figure

2.1), several different formulations of the problem could be proposed. In this context,

formulations of a VMP problem may be classiﬁed by the: (1) optimization approach, (2)

objective function and (3) solution technique, as considered in [12].

First, a VMP problem could be formulated considering one of the following optimiza-

tion approaches: (1) mono-objective (MOP), (2) multi-objective solved as mono-objective

(MAM) or (3) pure multi-objective (PMO). Once the optimization approach is deﬁned,

formulations may also be classiﬁed by the objective function(s) studied, both in minimiza-

tion and maximization contexts. These objective functions could be optimized separately

or simultaneously, depending on the selected optimization approach. Finally, solution tech-

niques for solving a VMP problem could be used as a third classiﬁcation criterion [12].

For a complete understanding of possible approaches for the formulation and resolution

of a VMP problem, Table 1 presents the taxonomy described in this section, including exam-

ple references from the studied VMP literature [13]. The following subsections present a

detailed description of the classiﬁcation criteria above deﬁned as well as special conside-

rations for particular formulations.

3.1 Optimization Approaches

This section presents the optimization approaches considered in the articles studied in [12].

The identiﬁed optimization approaches may be classiﬁed as: (1) mono-objective (MOP),

(2) multi-objective solved as mono-objective (MAM) and (3) pure multi-objective (PMO).

The mentioned optimization approaches are detailed in the following subsections.

3.1.1 Mono-Objective (MOP)

A mono-objective optimization approach (MOP) considers the optimization of one objective

function or the individual optimization of more than one objective function, one at a time.

Research on the VMP problem has been mainly guided by this MOP approach, identify-

ing almost 40 different objective functions already proposed considering this optimization

Cloud Computing Resource Allocation Taxonomies 7

approach [12]. It should be noted that an objective function could be studied considering

different modeling approaches (e.g. economical revenue maximization could be achieved

by minimizing the total economical penalties for SLA violations [31], by minimizing ope-

rational costs [22, 32] or even by maximizing the total proﬁt for leasing resources [33]).

3.1.2 Multi-Objective solved as Mono-Objective (MAM)

The optimization of multiple objective functions combined into one objective function is

considered in this work as a multi-objective problem solved as a mono-objective approach

(MAM). This hybrid approach enables the optimization of multiple objectives functions

with the disadvantage that it requires a deep knowledge of the problem domain to allow a

correct combination of the objective functions, which in most cases is not possible [34].

In the last years, a growing number of articles have proposed formulations of the VMP

problem with this hybrid optimization approach [13]. Different methods could be considered

for a formulation of a VMP problem with a MAM approach such as: weighted sum, solving

the problem as mono-objective while considering the other objective functions as constraints

of the problem or even proposing fuzzy logic to provide an efﬁcient way for combining

conﬂicting objectives and expert knowledge [13].

3.1.3 Pure Multi-Objective (PMO)

A general Pure Multi-Objective Optimization problem (PMO) includes a set of pdecision

variables, qobjective functions, and rconstraints. Objective functions and constraints are

functions of decision variables. In a PMO formulation, xrepresents the decision vector,

while yrepresents the objective vector. The decision space is denoted by Xand the

objective space as Y. These can be expressed as [35]:

Optimize:

y=f(x) = [f1(x), f2(x), ..., fq(x)] (1)

subject to:

e(x)=[e1(x), e2(x), ..., er(x)] ≥0(2)

where:

x= [x1, x2, ..., xp]∈X(3)

y= [y1, y2, ..., yq]∈Y(4)

The set of constrains e(x)≥0deﬁnes the set of feasible solutions Xf⊂Xand its

corresponding set of feasible objective vectors Yf⊂Y. The feasible decision space Xfis

the set of all decision vectors xin the decision space Xthat satisﬁes the constraints e(x).

The feasible objective space Yfis the set of the objective vectors ythat represents the

image of Xfonto Y. These feasible spaces are deﬁned as:

Xf={x|x∈X∧e(x)≥0}(5)

Yf={y|y=f(x)∀x∈Xf}(6)

8Fabio López-Pires and Benjamín Barán

To compare two solutions in a pure multi-objective context, the concept of Pareto dom-

inance is used. Given two feasible solutions u,v∈X,udominates v, denoted as uv,

if f(u)is better or equal to f(v)in every objective function and strictly better in at least

one objective function. If neither udominates v, nor vdominates u,uand vare said to be

non-comparable (denoted as u∼v).

A decision vector xis non-dominated with respect to a set U, if there is no member

of Uthat dominates x. The set of non-dominated solutions of the whole set of feasible

solutions Xf, is known as optimal Pareto set P∗. The corresponding set of objective vectors

constitutes the optimal Pareto front P F ∗[35].

Taking into account the large number of existing objective functions and possible approa-

ches for objective function modeling identiﬁed in [12, 13], PMO approaches could result

in more realistic formulations of a VMP problem, optimizing more than just one objective

function at a time (e.g. achieve economical revenue maximization by simultaneously mini-

mizing the total economical penalties for SLA violations, minimizing operational costs and

maximizing the proﬁt for leasing resources). In this context, PMOs optimizing more than

three objective functions are known as Many-Objective Optimization Problems (MaOPs),

as deﬁned in [57].

MaOPs differ signiﬁcantly from PMOs because several issues should be considered

when solving optimization problems with more than three objective functions [58]. In case

of Pareto-based algorithms, these issues are intrinsically related to the fact that as the number

of objective functions increases, the proportion of non-dominated solutions grows, being

increasingly difﬁcult to discriminate among solutions using only the Pareto dominance

relation [59]. Additionally, determining which solution to keep and which to discard in

order to converge toward the Pareto set is still a relevant issue to be addressed [58]. As the

number of objective functions grows, the proportion of non-dominated solutions to the total

number of solutions tends to one [60], making more difﬁcult to solve a MaOP.

Table 1 VMP Formulation and Resolution Taxonomy. Example references for each environment

are presented. Unexplored environments are considered Research Opportunities (RO).

Technique Approach Objective Functions

f1(x)f2(x)f3(x)f4(x)f5(x)

Deterministic

Algorithms

MOP [36] [37] [31] [19] [38]

MAM [6, 39] [6, 40] RO RO [39, 40]

PMO RO RO RO RO RO

Heuristics

MOP [20] [41] [33] [42] [38]

MAM [43, 17] [43, 44] [44, 45] [17, 46] [47, 48]

PMO RO RO RO RO RO

Meta-Heuristics

MOP [49] RO [21] [50] RO

MAM [18, 51] [52, 18] [53, 51] [52] [53, 51]

PMO [16, 54] [16, 26] [16, 26] RO [54]

Approximation

Algorithms

MOP [55] RO RO RO RO

MAM [56] RO RO RO RO

PMO RO RO RO RO RO

Cloud Computing Resource Allocation Taxonomies 9

3.2 Objective Function Groups

In cloud computing datacenters or different markets, there are several criteria that can be

considered when selecting a possible solution for a VMP problem, depending on manage-

ment policies and optimization objectives. These criteria can even change from one period

of time to another, which implies a variety of possible formulations of the problem and

different objectives to be optimized.

According to [12], the VMP literature mainly focuses on the optimization of objective

functions that speciﬁcally concerns CSPs (provider-oriented VMP). Objective functions

could also be formulated considering the requirements of CSTs for allocation of a particular

service or application, often composed by more than one VM (broker-oriented VMP).

It should be mentioned that in [12], nearly 60 different objective functions were iden-

tiﬁed for the three optimization approaches presented in Section 3.1. Considering the large

number of proposed objective functions, identiﬁed objective functions with similar cha-

racteristics and goals were classiﬁed into 5 objective function groups that are presented

in the following subsections. Additionally, considering that MAM and PMO optimization

approaches may take into account any combination of each objective function group or

even different objective functions from the same objective function group, a simpliﬁed

classiﬁcation is considered in this work (see Tables 1 and 2).

3.2.1 f1(x)- Energy Consumption

Energy consumption management is an important studied issue in the provider-oriented

VMP literature, with high impact in operational costs and carbon dioxide emissions for

cloud datacenter operations. According to [61], most of the time, servers operate in a very

low energy-efﬁciency possible region (i.e. between 10 and 50% of resource utilization), even

though energy efﬁciency is a very important issue to address, considering its economical and

ecological impact in modern datacenters. Energy consumption is the most studied objective

function for a VMP problem including different modeling approaches [12].

The most studied alternatives for modeling energy consumption include [12]: conso-

lidating VMs on the minimum number of PMs [62] and considering a linear relationship

between power consumption and Central Process Unit (CPU) utilization [5]. Joint optimiza-

tion of energy consumption and network trafﬁc is also very studied in the VMP literature,

considering that network communication equipment represents between 10 and 20% of the

total datacenter energy consumption [56].

3.2.2 f2(x)- Network Trafﬁc

As proposed in [5], network trafﬁc is an important objective function for optimization with

open challenges in cloud computing datacenters for the provider-oriented VMP.

The main approaches for modeling network trafﬁc include the optimization of: network

communication costs, live migration overhead, network metrics such as: delay, data access

and data transfer time, link congestion, network performance, service response time as well

as average latency, and Wide Area Network (WAN) communication in distributed-cloud

deployments [13].

A very studied approach for network trafﬁc minimization is the placement of VMs

with high communication rate in the same PM (or at least in the same rack) to avoid

the utilization of network resources (or at least core network equipment). This approach

includes workload characterization and clustering techniques applied to a VMP problem

10 Fabio López-Pires and Benjamín Barán

to minimize the inter-VM network trafﬁc [13]. Additionally, modeling and quantiﬁcation

of live migration network overhead is an important open challenge in a provider-oriented

VMP network trafﬁc optimization context [12].

Considering that VMs are dynamically created and destroyed in cloud computing en-

vironments, a consolidation process could require high level of ﬂexibility where traditional

routing protocols present limitations to adjust ﬂow paths. In [63], the authors proposed net-

work trafﬁc load balancing to improve QoS in a VMP context considering Software Deﬁned

Networks (SDN) [64], where ﬂow paths are determined based on network status metrics

such as low delay, low packet loss or high security, just to cite a few.

3.2.3 f3(x)- Economical Costs

Economical costs optimization is an important studied issue in both provider-oriented and

broker-oriented VMP literature.

For a provider-oriented VMP, economical costs optimization is a key issue to be addre-

ssed and could be achieved by reducing operational costs. These operational costs are mainly

related to energy consumption minimization but other formulations could also be studied,

such as thermal dissipation costs [51]. Reducing penalty costs of SLA violations is another

studied approach in order to maximize the economical revenue of a CSP [13].

Finally, CSPs can maximize its economical revenue by leasing all its available resources

or at least the maximum possible [13]. In this context, VMP could be studied jointly with

admission control problem [4] and two possible scenarios could be identiﬁed: (1) if demand

for resources exceeds the current available resources, overbooking techniques or cloud

federation [4] can help CSPs to attend the requirements of the CSTs; on the other hand, (2)

idle resources can be offered in an auction-based scheme such as Amazon’s Spot Instances

[65]. Both scenarios represent open challenges for the VMP problem as well as emerging

cloud computing markets.

For a broker-oriented VMP, economical costs optimization is a key issue in actual cloud

markets and CSTs look for CSPs that meet the speciﬁc requirements of a particular service,

preferably with the minimal economical costs for the required cloud infrastructure.

The most studied pricing scheme in the considered VMP literature is ﬁxed price. How-

ever, with the recent trend of dynamic pricing of cloud resources, where the price of resources

can vary depending on the free capacity and load of the CSP, few articles have recently

proposed formulations of a VMP problem considering dynamic prices schemes [66].

3.2.4 f4(x)- Performance

Performance optimization is an important studied issue in both provider-oriented and broker-

oriented VMP literature.

Performance modeling includes the optimization of: availability, CPU demand satis-

faction, deployment time, QoS, resource interference, security metrics, Shared Last Level

Cache (SLLC) contention and total job completion time [13]. Most of these performance

metrics may be considered for CSPs to improve the QoS in a provider-oriented VMP or for

CSTs in order to select an appropriate CSP to host their services in a broker-oriented VMP.

3.2.5 f5(x)- Resource Utilization

Cloud computing datacenters are commonly composed by different types of physical and

virtual resources such as CPU, RAM, storage, network bandwidth and even Graphical

Process Unit (GPU) in some cases.

Cloud Computing Resource Allocation Taxonomies 11

The main approaches include the maximization of resource utilization, but an important

issue to consider is the balanced utilization of each resource [13]. Li et al. studied the

concept of elasticity, referring to how well a datacenter can satisfy the growth of the input

VMs resource demands under limitations of PMs and network link capacities [67].

Considering the importance of the efﬁcient utilization of resources, an interesting anal-

ysis of the anomalies and drawbacks in some existing strategies for efﬁcient resource

utilization is presented in [68], proposing a novel vector-based technique that solves the

considered anomalies.

3.3 Solution Techniques

In the VMP literature considered in [12, 13], different techniques were considered for

solving a VMP problem. The main solution techniques include: (1) deterministic algorithms,

(2) heuristic algorithms, (3) meta-heuristic algorithms, and (4) approximation algorithms.

The four mentioned solution techniques are detailed in the following subsections.

3.3.1 Deterministic Algorithms

Classical deterministic techniques were considered for a VMP problem, including Con-

straint Programming (CP), Linear Programming (LP), Integer Linear Programming (ILP),

Mixed Integer Linear Programming (MILP), Pseudo-Boolean Optimization (PBO) and Dy-

namic Programming (DP) [12]. Most of these approaches are considered for solving novel

mathematical formulations of a VMP problem without any practical intention, considering

that obtaining the optimal solution implies a search in an universe of Npossible solutions

[16]:

N= (n+ 1)m(7)

where:

N: Size of the searching universe

n: Number of physical machines

m: Number of virtual machines

3.3.2 Heuristics

Considering that VMP is a combinatorial NP-complete problem [39], it is impracticable to

optimally solve instances of the problem for large number of PMs and VMs. Commonly,

CSPs are composed by thousands to millions of PMs and VMs, a scenario where optimal

solutions with exhaustive search algorithms can result extremely expensive. Therefore, a

trade-off between quality of solutions and computational cost has to be considered for real

world cloud management systems.

Heuristics have already been extensively studied in the literature for exponential com-

plexity problems. Several articles proposed heuristic-based solution techniques for a VMP

problem [12]. Most of the studied articles have proposed heuristics based on well-studied

algorithms such as: First-Fit, First-Fit Decreasing, Best-Fit, Best-Fit Decreasing, Worst-Fit

and Heaviest-Fit [13]. Other greedy algorithms were also proposed in addition to novel

heuristics proposed for the resolution of the VMP problem [13].

12 Fabio López-Pires and Benjamín Barán

3.3.3 Meta-Heuristics

As mentioned before, approximations to optimal solutions are sufﬁcient in most of cloud

infrastructure environments. Meta-heuristics are also very useful in order to obtain good

solutions in practical time. Meta-heuristics include [12]: Memetic Algorithms (MA), Particle

Swarm Optimization (PSO), Ant Colony Optimization (ACO), Genetic Algorithm (GA),

Neighborhood Search (NS), Cut-and-Search,Simulated Annealing (SA) and Tabu Search

(TS) [13]. According to the proposed taxonomy, meta-heuristics are mainly studied with

multi-objective approaches (MAM and PMO) for solving VMP problems (see Table 1).

3.3.4 Approximation Algorithms

Heuristics and meta-heuristics provide good quality solutions, but the quality of the expected

solutions is hardly measurable. In a p-approximation algorithm, the value of a solution will

not be more (or less) than a factor ptimes the optimum solution. A small number of articles

proposed approximation algorithms for solving a VMP problem [12]. It is worth noting that

for cloud infrastructures, solutions obtained using heuristics or meta-heuristics techniques

are sufﬁciently good for most practical cases.

4 VMP Taxonomy: Research Opportunities

Based on an universe of 84 studied publications systematically chosen [13], Table 2 summa-

rizes the taxonomies presented in this work (see Figure 2.1 and Table 1). Considering that

5 studied articles [6, 38, 55, 56, 69] proposed 2 different solution techniques for the same

VMP formulation, statistics are based on 89 different VMP formulations (see Table 2).

Considering a detailed analysis of the information provided by Table 2, this section

summarizes research opportunities as well as on going research work by the authors in

order to further advance in this very active research area. It should be noted that statistics

presented in the following subsections are deﬁned following the studied VMP literature (of

89 different VMP formulations) and considered only as a simple reference, given that more

articles may be considered and rapidly being published. Additionally, different systematic

reviews of the VMP literature (e.g. considering different keywords) or extended systematic

reviews (e.g. considering additional keywords) could result in different statistics.

Taking into account the diversity of existing environments, formulations and solution

techniques (see Table 2), it is important to provide a general vision on the main approaches

for the VMP problem, enabling interested readers to deﬁne speciﬁc research alternatives.

The following subsections describe research opportunities identiﬁed in this work as a main

result of the proposed VMP taxonomies (see Figure 2.1 and Tables 1 and 2).

4.1 Unexplored Environments, Formulations and Solution Techniques

Considering the studied VMP literature [13], unexplored VMP environments were identiﬁed

(see Research Opportunities (RO) in Figure 2.1). First, provider-oriented VMP problems

in federated-cloud deployments were not considered with ofﬂine formulations. Second,

broker-oriented VMP problems in multi-cloud deployments were not considered with online

formulations. No formulation or solution technique was studied for the mentioned unex-

plored VMP environments. Additionally, unexplored formulations and solutions techniques

were also identiﬁed in this work (see Research Opportunities (RO) in Table 1).

Cloud Computing Resource Allocation Taxonomies 13

Table 2 Virtual Machine Placement Taxonomy. Elements of column % represent the percentage

of articles in the studied universe [13]. For simplicity, MAM and PMO consider only the

number of considered objectives in column f(x). See Section 3.2 for f(x)details.

Oriented Deployment

Architecture Formulation Optimization

Approach f(x)Solution

Technique %

Provider

Single-cloud

Ofﬂine

MOP

f1(x)Heuristic 3.37%

Meta-Heuristic 2.25%

f2(x)Heuristic 2.25%

f4(x)Heuristic 1.12%

MAM 3Heuristic 2.25%

Meta-Heuristic 3.37%

2 Deterministic 1.12%

PMO 3 Meta-Heuristic 2.25%

2 Meta-Heuristic 1.12%

Online

MOP

f1(x)

Deterministic 4.49%

Heuristic 6.74%

Meta-Heuristic 1.12%

Approximation 1.12%

f2(x)Deterministic 2.25%

Heuristic 5.62%

f3(x)Deterministic 1.12%

Heuristic 5.62%

f4(x)Heuristic 7.87%

Meta-Heuristic 1.12%

f5(x)Deterministic 1.12%

Heuristic 4.49%

MAM

3Heuristic 3.37%

Meta-Heuristic 1.12%

2

Deterministic 3.37%

Heuristic 16.85%

Meta-Heuristic 1.12%

Approximation 1.12%

Distributed-cloud

Ofﬂine MOP f4(x)Deterministic 1.12%

Online

MOP

f2(x)Heuristic 1.12%

Deterministic 1.12%

f3(x)Heuristic 1.12%

f5(x)Heuristic 1.12%

MAM 3 Deterministic 1.12%

2 Meta-Heuristic 1.12%

Federated-cloud Online MOP f1(x)Heuristic 1.12%

Broker Multi-cloud Ofﬂine MOP f3(x)Deterministic 1.12%

Meta-Heuristic 1.12%

Total 100%

For MOP optimization approach, f2(x)(network trafﬁc) and f5(x)(resource utilization)

were not studied considering meta-heuristics as solution technique. Additionally, approxi-

mation algorithms were studied as a solution technique only for f1(x)(energy consumption),

representing unexplored alternatives for the remaining studied objective function groups.

For MAM optimization approach, the resolution of VMP formulations considering

f3(x)(economical revenue) and f4(x)(performance) with deterministic algorithms were

14 Fabio López-Pires and Benjamín Barán

not studied. Similar to the MOP approach, approximation algorithms were studied as a so-

lution technique only for f1(x)(energy consumption), representing unexplored alternatives

for the remaining studied objective functions (see Research Opportunities (RO) in Table 1).

For PMO optimization approach, f4(x)(performance) was not studied considering

meta-heuristics as solution technique, while neither deterministic algorithms, heuristics nor

approximation algorithms studied PMO formulations of the VMP problem, representing

unexplored alternatives (see Research Opportunities (RO) in Table 1).

Taking into account a complete understanding of the VMP problem composed by VMP

environments and VMP formulations and solution techniques (see Table 2), several unex-

plored VMP problems could also be identiﬁed. In this context, Table 2 could guide interested

readers to identify existing research on VMP problems according to the studied VMP lit-

erature [13]. It should be noted that all possible VMP problems that are not presented in

Table 2 could be considered as a research opportunity. As an example, MAM and PMO

optimization approaches are not presented in Table 2 for the following VMP environments:

(1) provider-oriented VMP in distributed-clouds with ofﬂine formulations, (2) provider-

oriented VMP in federated-clouds with online formulations and (3) broker-oriented VMP

in multi-clouds with ofﬂine formulations.

4.2 Broker-oriented VMP considering Online Formulations

According to the proposed VMP Environment Taxonomy (see Figure 2.1), a broker-oriented

VMP problem could be studied in a multi-cloud deployment architecture considering ofﬂine

or online formulations. A broker-oriented VMP problem could also be studied conside-

ring a mono-objective approach (MOP) or multi-objective approaches (MAM or PMO),

optimizing different objective functions (see Table 1).

Considering that only 2.24% of the considered VMP literature studied a broker-oriented

VMP problem as an ofﬂine optimization problem (see Table 2), on going research by the

authors focus on exploring broker-oriented VMP formulations. In this context, an extended

systematic review of research articles demonstrate that broker-oriented VMP includes se-

veral research opportunities. The authors of this work are already working on novel math-

ematical formulations, specially for online broker-oriented VMP problems.

Several dynamic parameters could be studied for online broker-oriented VMP problems

such as: (1) pricing schemes, (2) VM offers or (3) user requirements, but a detailed survey on

these parameters and formulations is still needed (research opportunity). It should be noted

that the implementation of possible migrations of VMs across different CSPs is limited by

cloud interoperability factors and it is still out of the scope of this work.

4.3 Provider-oriented VMP considering Online Formulations

Considering the on-demand model of cloud computing, a provider-oriented VMP problem

should be solved dynamically to efﬁciently attend typical workload of modern applications.

According to the studied articles, 77.53% considered this particular type of VMP problem

with several different formulations and dynamic parameters.

On going research by the authors include detailed studies of the provider-oriented VMP

literature in order to understand possible challenges for CSPs in dynamic environments,

based on the most relevant dynamic parameters studied so far in the VMP literature. Pre-

liminary results identiﬁed that resource capacities of VMs (associated to vertical elasticity),

number of VMs of a cloud service (associated to horizontal elasticity) and utilization of

Cloud Computing Resource Allocation Taxonomies 15

resources of VMs (related to overbooking) are the most relevant dynamic parameters in

the specialized literature. Consequently, dynamic environments for online formulations of

the provider-oriented VMP problem could be classiﬁed by one or more of the following

classiﬁcation criteria: (1) elasticity and (2) overbooking.

According to preliminary results obtained by the authors of this work, dynamic environ-

ments could be formulated considering one of the following elasticity values: no elasticity,

horizontal elasticity, vertical elasticity or both horizontal and vertical elasticity. Addition-

ally, identiﬁed dynamic environments may also consider one of the following overbooking

values: no overbooking, server resources overbooking, network resources overbooking or

both server and network overbooking.

Research opportunities for online formulations of a provider-oriented VMP include

detailed studies on complex dynamic environments (e.g. VMP considering both types of

elasticity and both types of overbooking) in order to enable CSPs to efﬁciently support

modern cloud applications and services. Considering that modern cloud services are often

composed by several inter-related VMs, the authors of this work are already working on

modeling techniques for these complex cloud services to consider any type of deployment

architecture (i.e. single-cloud, distributed-cloud, federated-cloud or multi-cloud).

4.4 Provider-oriented VMP considering PMO Optimization

According to the studied articles, only 3.37% considered a PMO approach for an ofﬂine

formulation of provider-oriented VMP problems simultaneously optimizing 2 or 3 objective

functions. PMO approaches could result in more realistic formulations of a VMP problem,

optimizing more than just one objective function at a time.

Taking into account that existing PMO formulations of a provider-oriented VMP pro-

blem consider at most 3 objective functions and more than 60 different objective functions

were identiﬁed in [12], several formulations could still be considered, specially for PMO

approaches. In this context, it is important to remember that PMOs optimizing more than

three objective functions are known as MaOPs. As described in Section 3.1.3, several issues

should be considered when solving optimization problems with more than three objective

functions [58]. Considering the mentioned challenges for solving MaOPs, López-Pires and

Barán have recently proposed in [27] a general many-objective optimization framework

that is able to consider as many objective functions as needed when solving an ofﬂine VMP

problem in a PMO context. In order to converge to a treatable number of non-dominated

solutions, the authors proposed the utilization of interactive lower and upper bounds asso-

ciated to each objective function to reduce the number of possible solutions of the Pareto

set approximation Pknown , following a decision maker needs.

Additionally, online formulations of a provider-oriented VMP considering a PMO

approach should be explored. For this particular type of VMP problem, PMO approaches

should include strategies for an appropriate solution selection from the Pareto set approxi-

mation Pknown , composed by non-dominated solutions. In this context, Ihara et al. already

proposed a ﬁrst formulation of a Many-Objective VMP problem (MaVMP) for dynamic

environments, presenting studies on the evaluation of several automatic selection strategies

for provider-oriented VMP problems formulated as MaOPs [70].

16 Fabio López-Pires and Benjamín Barán

4.5 Provider-oriented VMP in Distributed and Federated Clouds

According to the studied articles, only 7.87% proposed the provider-oriented VMP problem

in a distributed-cloud deployment architecture, while only [20] studied the provider-oriented

VMP problem considering a federated-cloud deployment architecture (see Table 2).

Resource allocation in distributed and federated cloud environments is an active research

area [71]. In a provider-oriented VMP context, several particular constraints and objective

functions may still be studied and evaluated in order to fulﬁll the requirements of a CSP

with geo-distributed cloud computing datacenters or different CSPs in a cloud federation.

It should be noted that a broker-oriented VMP in a distributed-cloud deploy-

ment architecture may be adapted as a multi-cloud deployment architecture by con-

sidering each datacenter as a different CSP in order to fulﬁll requirements of high-

availability or legal issues (e.g. Amazon’s us-east-1, us-west-2 or eu-west-1 datacenters)

[https://aws.amazon.com/about-aws/global-infrastructure].

For a VMP problem in a distributed or federated-cloud deployment architecture, several

research opportunities may be still proposed considering unexplored objective functions,

novel formulations in MOP, MAM or PMO approaches or even experimental evaluation of

different solution techniques.

5 Conclusions and Future Directions

Based on an universe of 84 studied articles with 89 different VMP formulations [13], this

work presented general taxonomies of the VMP problem (see Table 2) considering possible

environments where the VMP problem could be studied (see Research Opportunities (RO)

in Figure 2.1) as well as different possible formulations and techniques for the resolution

of the VMP problem (see Table 1).

Depending on the particular environment where a VMP problem will be studied, se-

veral different considerations should be taken into account before considering a particular

formulation or technique for a VMP problem resolution. In this context, different possible

environments were identiﬁed by classifying research articles in the VMP literature by: (1)

orientation, (2) deployment architecture and (3) type of formulation.

A VMP problem could be studied from both provider or broker perspectives (see Section

2.1). A provider-oriented VMP problem could be studied considering one of the follo-

wing deployment architectures: single-cloud, distributed-cloud or federated-cloud, while

a broker-oriented VMP problem could be studied considering a multi-cloud deployment

architecture (see Section 2.2). Additionally, both a provider-oriented and a broker-oriented

VMP problem in any of the possible deployment architectures could be studied considering

two different types of formulation: ofﬂine or online formulations (see Section 2.3).

Considering each environment where a VMP problem was studied (see Figure 2.1),

several possible formulations of the problem could be proposed. In this context, formulations

of a VMP problem was classiﬁed by: (1) optimization approach, (2) objective function

and (3) solution technique. First, a VMP problem could be formulated considering one

of the following optimization approaches: (1) mono-objective (MOP), (2) multi-objective

solved as mono-objective (MAM) or (3) pure multi-objective (PMO). Once the optimization

approach is deﬁned, formulations were classiﬁed by the objective function(s) studied, both

in minimization and maximization contexts. These objective functions could be optimized

Cloud Computing Resource Allocation Taxonomies 17

separately or simultaneously, depending on the selected optimization approach. Finally,

solution techniques for solving a VMP problem were used as a classiﬁcation criterion [12].

According to Table 2, the VMP problem have been mostly studied as an online pro-

blem from the provider perspective, considering a single-cloud deployment architecture,

representing 69.66% of the studied articles. For this particular VMP environment, 42,70%

considered a MOP approach. It could be said that online formulations of provider-oriented

VMP problems in single-cloud deployment architecture is a well studied problem and ex-

isting literature should guide CSPs to solve VMP problems with these considerations.

Based on the proposed taxonomies, several research opportunities were identiﬁed in

the following research directions (see Section 4): (1) unexplored environments, formula-

tions and solution techniques, (2) broker-oriented VMP considering online formulations,

(3) provider-oriented VMP considering online formulations, (4) provider-oriented VMP

considering PMO optimization and (5) provider-oriented VMP in distributed and feder-

ated clouds. It should be mentioned that other relevant research opportunities could also be

identiﬁed with the proposed taxonomies.

Focusing on the large number of identiﬁed objective functions [12], the following ques-

tions still have no answer considering the studied VMP literature [13]: (1) can CSPs efﬁ-

ciently optimize more than 3 objective functions for the VMP problem in a reasonable time?,

(2) which solution technique is the most appropriate for solving VMP problems considering

PMO approaches considering real scenarios?, (3) are PMO approaches the best choice for

online formulations or MAM approaches have the potential of performing even better?.

Additionally, the following questions focus on the variety of possible deployment archi-

tectures where the VMP problem could be studied: (1) how can CSPs model and formulate

and solve VMP problem in distributed or federated clouds?, (2) which solution technique

is the most appropriate for broker-oriented VMP for large scale customers?, (3) which

objective functions should a CSP consider in different deployment architectures?.

In order to answer the above mentioned relevant questions, research should focus on

unexplored formulations of the VMP problem, developing novel techniques and providing

methods and accepted benchmarks to compare and evaluate different approaches. Address-

ing this unexplored formulations may start applying an extended review of the VMP litera-

ture, considering that this work studied a relevant sub-set of the existing VMP literature in

order to guide interested readers, providing a general vision on this research area.

Additionally, future directions may include detailed studies on the VMP problem con-

sidering dynamic pricing schemes or auction-based pricing. These type of pricing schemes

represent open challenges for both CSPs and CSBs in the VMP research and emerging cloud

computing markets. Techniques for avoiding an excessive number of possible placement

combinations is also an open challenge for the VMP problem [52].

It is important to mention that actual cloud markets are mostly composed by thousands

to millions of VMs which are dynamically created and destroyed, so experimental tests for

VMP problem should consider: (1) large number of VMs and PMs, (2) heterogeneity in

PMs and VMs conﬁgurations, (3) diverse types and workload distribution, and (4) trending

dynamic parameters. Considering the studied articles, there is no test problems for the VMP

that could be used as a world accepted benchmark. A test problem generator could be very

useful at the time of composing experimental alternatives, giving the difﬁculties to access

real traces of world class cloud computing companies. As a general conclusion, it could be

said that different methods and algorithms should be still evaluated before a real good tool

is ready for massive use in commercial cloud computing datacenters.

18 Fabio López-Pires and Benjamín Barán

References

[1] Peter Mell and Tim Grance. The NIST deﬁnition of cloud computing. National Institute of

Standards and Technology, 53(6):50, 2009.

[2] Rajkumar Buyya, Chee Shin Yeo, Srikumar Venugopal, James Broberg, and Ivona Brandic.

Cloud computing and emerging it platforms: Vision, hype, and reality for delivering computing

as the 5th utility. Future Generation computer systems, 25(6):599–616, 2009.

[3] Sunilkumar S Manvi and Gopal Krishna Shyam. Resource management for infrastructure as a

service (iaas) in cloud computing: A survey. Journal of Network and Computer Applications,

41:424–440, 2014.

[4] Erik Elmroth, Johan Tordsson, Francisco Hernández, Ahmed Ali-Eldin, Petter Svärd, Mina

Sedaghat, and Wubin Li. Self-management challenges for multi-cloud architectures. In Towards

a Service-Based Internet, pages 38–49. Springer, 2011.

[5] Anton Beloglazov, Jemal Abawajy, and Rajkumar Buyya. Energy-aware resource allocation

heuristics for efﬁcient management of data centers for cloud computing. Future Generation

Computer Systems, 28(5):755–768, 2012.

[6] Ankit Anand, J Lakshmi, and SK Nandy. Virtual machine placement optimization supporting

performance SLAs. In Cloud Computing Technology and Science (CloudCom), 2013 IEEE 5th

International Conference on, volume 1, pages 298–305. IEEE, 2013.

[7] Rajkumar Buyya, Chee Shin Yeo, and Srikumar Venugopal. Market-oriented cloud computing:

Vision, hype, and reality for delivering it services as computing utilities. In High Performance

Computing and Communications, 2008. HPCC’08. 10th IEEE International Conference on,

pages 5–13. Ieee, 2008.

[8] Anton Beloglazov and Rajkumar Buyya. Optimal online deterministic algorithms and adaptive

heuristics for energy and performance efﬁcient dynamic consolidation of virtual machines in

cloud data centers. Concurrency and Computation: Practice and Experience, 24(13):1397–1420,

2012.

[9] Leili Salimian and Faramarz Saﬁ. Survey of energy efﬁcient data centers in cloud comput-

ing. In Proceedings of the 2013 IEEE/ACM 6th International Conference on Utility and Cloud

Computing, pages 369–374. IEEE Computer Society, 2013.

[10] Monica Gahlawat and Priyanka Sharma. Survey of virtual machine placement in federated

clouds. In Advance Computing Conference (IACC), 2014 IEEE International, pages 735–738.

IEEE, 2014.

[11] Kevin Mills, J Filliben, and Christopher Dabrowski. Comparing vm-placement algorithms for

on-demand clouds. In Cloud Computing Technology and Science (CloudCom), 2011 IEEE Third

International Conference on, pages 91–98. IEEE, 2011.

[12] Fabio López-Pires and Benjamín Barán. A virtual machine placement taxonomy. In Proceedings

of the 2015 IEEE/ACM 15th International Symposium on Cluster, Cloud and Grid Computing.

IEEE Computer Society, 2015.

[13] Fabio López-Pires and Benjamín Barán. Virtual machine placement literature review. Technical

report, Polytechnic School, National University of Asunción, 2015.

[14] Ryan Jansen and Paul R Brenner. Energy efﬁcient virtual machine allocation in the cloud. In

Green Computing Conference and Workshops (IGCC), 2011 International, pages 1–8. IEEE,

2011.

[15] Johan Tordsson, Rubén S Montero, Rafael Moreno-Vozmediano, and Ignacio M Llorente. Cloud

brokering mechanisms for optimized placement of virtual machines across multiple providers.

Future Generation Computer Systems, 28(2):358–367, 2012.

Cloud Computing Resource Allocation Taxonomies 19

[16] Fabio López-Pires and Benjamín Barán. Multi-objective virtual machine placement with service

level agreement: A memetic algorithm approach. In Proceedings of the 2013 IEEE/ACM 6th

International Conference on Utility and Cloud Computing, pages 203–210. IEEE Computer

Society, 2013.

[17] Katsunori Sato, Masaki Samejima, and Norihisa Komoda. Dynamic optimization of virtual

machine placement by resource usage prediction. In Industrial Informatics (INDIN), 2013 11th

IEEE International Conference on, pages 86–91. IEEE, 2013.

[18] Kuan-yin Chen, Yang Xu, Kang Xi, and H Jonathan Chao. Intelligent virtual machine placement

for cost efﬁciency in geo-distributed cloud systems. In Communications (ICC), 2013 IEEE

International Conference on, pages 3498–3503. IEEE, 2013.

[19] Eyal Bin, Ofer Biran, Odellia Boni, Erez Hadad, Elliot K Kolodner, Yosef Moatti, and Dean H

Lorenz. Guaranteeing high availability goals for virtual machine placement. In Distributed

Computing Systems, 2011 31st International Conference on, pages 700–709. IEEE, 2011.

[20] Corentin Dupont, Giovanni Giuliani, Fabien Hermenier, Thomas Schulze, and Andrey Somov.

An energy aware framework for virtual machine placement in cloud federated data centres. In

Future Energy Systems: Where Energy, Computing and Communication Meet (e-Energy), 2012

Third International Conference on, pages 1–10. IEEE, 2012.

[21] Ching Chuen Teck Mark, Dusit Niyato, and Tham Chen-Khong. Evolutionary optimal virtual

machine placement and demand forecaster for cloud computing. In Advanced Information

Networking and Applications (AINA), 2011 IEEE International Conference on, pages 348–355.

IEEE, 2011.

[22] Zhe Huang and Danny HK Tsang. Sla guaranteed virtual machine consolidation for computing

clouds. In Communications (ICC), 2012 IEEE International Conference on, pages 1314–1319.

IEEE, 2012.

[23] Kuan Lu, Ramin Yahyapour, Philipp Wieder, Constantinos Kotsokalis, Edwin Yaqub, and Ali Im-

ran Jehangiri. Qos-aware vm placement in multi-domain service level agreements scenarios.

In Cloud Computing (CLOUD), 2013 IEEE Sixth International Conference on, pages 661–668.

IEEE, 2013.

[24] Kamal Bouyoucef, I Limam-Bedhiaf, and Omar Cherkaoui. Optimal allocation approach of

virtual servers in cloud computing. In Next Generation Internet (NGI), 2010 6th EURO-NF

Conference on, pages 1–6. IEEE, 2010.

[25] D. Espling, L. Larsson, W. Li, J. Tordsson, and E. Elmroth. Modeling and placement of cloud

services with internal structure. Cloud Computing, IEEE Transactions on, PP(99):1–1, 2014.

[26] Fabio López-Pires, Elias Melgarejo, and Benjamín Barán. Virtual machine placement. a multi-

objective approach. In Computing Conference (CLEI), 2013 XXXIX Latin American, pages 1–8.

IEEE, 2013.

[27] Fabio López-Pires and Benjamín Barán. A many-objective optimization framework for virtual-

ized datacenters. In Proceedings of the 2015 5th International Conference on Cloud Computing

and Service Science, pages 439–450, 2015.

[28] Jammily Ortigoza, Fabio López Pires, and Benjamín Barán. A taxonomy on dynamic environ-

ments for provider-oriented virtual machine placement. In Proceedings of the 2016 IEEE 4th

International Conference on Cloud Engineering. IEEE Computer Society, 2016.

[29] Wubin Li, Johan Tordsson, and Erik Elmroth. Modeling for dynamic cloud scheduling via

migration of virtual machines. In Cloud Computing Technology and Science (CloudCom), 2011

IEEE Third International Conference on, pages 163–171. IEEE, 2011.

[30] José Luis Lucas Simarro, Rafael Moreno-Vozmediano, Ruben S Montero, and Ignacio Martín

Llorente. Dynamic placement of virtual machines for cost optimization in multi-cloud environ-

ments. In High Performance Computing and Simulation (HPCS), 2011 International Conference

on, pages 1–7. IEEE, 2011.

20 Fabio López-Pires and Benjamín Barán

[31] Huynh Tu Dang and Fabien Hermenier. Higher SLA satisfaction in datacenters with continuous

VM placement constraints. In Proceedings of the 9th Workshop on Hot Topics in Dependable

Systems, page 1. ACM, 2013.

[32] Zhe Huang, Danny HK Tsang, and James She. A virtual machine consolidation framework

for mapreduce enabled computing clouds. In Proceedings of the 24th International Teletrafﬁc

Congress, page 26. International Teletrafﬁc Congress, 2012.

[33] Weiming Shi and Bo Hong. Towards proﬁtable virtual machine placement in the data center.

In Utility and Cloud Computing (UCC), 2011 Fourth IEEE International Conference on, pages

138–145. IEEE, 2011.

[34] Benjamín Barán, Christian von Lücken, and Aldo Sotelo. Multi-objective pump scheduling

optimisation using evolutionary strategies. Advances in Engineering Software, 36(1):39–47,

2005.

[35] Carlos Coello Coello, Gary B Lamont, and David A Van Veldhuizen. Evolutionary algorithms

for solving multi-objective problems. Springer, 2007.

[36] Hadi Goudarzi and Massoud Pedram. Energy-efﬁcient virtual machine replication and placement

in a cloud computing system. In Cloud Computing (CLOUD), 2012 IEEE 5th International

Conference on, pages 750–757. IEEE, 2012.

[37] Jing Tai Piao and Jun Yan. A network-aware virtual machine placement and migration approach

in cloud computing. In Grid and Cooperative Computing (GCC), 2010 9th International Con-

ference on, pages 87–92. IEEE, 2010.

[38] Wubin Li, Johan Tordsson, and Erik Elmroth. Virtual machine placement for predictable and

time-constrained peak loads. In Economics of Grids, Clouds, Systems, and Services, pages

120–134. Springer, 2012.

[39] Meng Sun, Weidong Gu, Xinchang Zhang, Huiling Shi, and Wei Zhang. A matrix transformation

algorithm for virtual machine placement in cloud. In Trust, Security and Privacy in Computing

and Communications (TrustCom), 2013 12th IEEE International Conference on, pages 1778–

1783. IEEE, 2013.

[40] Fei Song, Daochao Huang, Huachun Zhou, Hongke Zhang, and Ilsun You. An optimization-based

scheme for efﬁcient virtual machine placement. International Journal of Parallel Programming,

42(5):853–872, 2014.

[41] Daniel S Dias and Luıs Henrique MK Costa. Online trafﬁc-aware virtual machine placement in

data center networks. In Global Information Infrastructure and Networking Symposium (GIIS),

2012, pages 1–8. IEEE, 2012.

[42] Abhishek Gupta, Laxmikant V Kalé, Dejan Milojicic, Paolo Faraboschi, and Susanne M Balle.

Hpc-aware vm placement in infrastructure clouds. In Cloud Engineering (IC2E), 2013 IEEE

International Conference on, pages 11–20. IEEE, 2013.

[43] Jiankang Dong, Hongbo Wang, Xing Jin, Yangyang Li, Peng Zhang, and Shiduan Cheng. Virtual

machine placement for improving energy efﬁciency and network performance in iaas cloud. In

Distributed Computing Systems Workshops (ICDCSW), 2013 IEEE 33rd International Confer-

ence on, pages 238–243. IEEE, 2013.

[44] Hua-Jun Hong, De-Yu Chen, Chun-Ying Huang, Kuan-Ta Chen, and Cheng-Hsin Hsu. Qoe-

aware virtual machine placement for cloud games. In Network and Systems Support for Games

(NetGames), 2013 12th Annual Workshop on, pages 1–2. IEEE, 2013.

[45] Aissan Dalvandi, Mohan Gurusamy, and Kee Chaing Chua. Time-aware vm-placement and

routing with bandwidth guarantees in green cloud data centers. In Cloud Computing Technology

and Science (CloudCom), 2013 IEEE 5th International Conference on, volume 1, pages 212–217.

IEEE, 2013.

Cloud Computing Resource Allocation Taxonomies 21

[46] Wenting Wang, Haopeng Chen, and Xi Chen. An availability-aware virtual machine placement

approach for dynamic scaling of cloud applications. In Ubiquitous Intelligence & Computing

and 9th International Conference on Autonomic & Trusted Computing (UIC/ATC), 2012 9th

International Conference on, pages 509–516. IEEE, 2012.

[47] Nicolò Maria Calcavecchia, Ofer Biran, Erez Hadad, and Yosef Moatti. VM placement strategies

for cloud scenarios. In Cloud Computing (CLOUD), 2012 IEEE 5th International Conference

on, pages 852–859. IEEE, 2012.

[48] Zhibo Cao and Shoubin Dong. An energy-aware heuristic framework for virtual machine con-

solidation in cloud computing. The Journal of Supercomputing, pages 1–23, 2014.

[49] Maolin Tang and Shenchen Pan. A hybrid genetic algorithm for the energy-efﬁcient virtual

machine placement problem in data centers. Neural Processing Letters, pages 1–11, 2014.

[50] Konstantinos Tsakalozos, Mema Roussopoulos, and Alex Delis. Vm placement in non-

homogeneous iaas-clouds. In Service-Oriented Computing, pages 172–187. Springer, 2011.

[51] Jing Xu and Jose AB Fortes. Multi-objective virtual machine placement in virtualized data center

environments. In Green Computing and Communications (GreenCom), 2010 IEEE/ACM Int’l

Conference on & Int’l Conference on Cyber, Physical and Social Computing (CPSCom), pages

179–188. IEEE, 2010.

[52] Soichi Shigeta, Hiroyuki Yamashima, Tsunehisa Doi, Tsutomu Kawai, and Keisuke Fukui.

Design and implementation of a multi-objective optimization mechanism for virtual machine

placement in cloud computing data center. In Cloud Computing, pages 21–31. Springer, 2013.

[53] Amol C Adamuthe, Rupali M Pandharpatte, and Gopakumaran T Thampi. Multiobjective virtual

machine placement in cloud environment. In Cloud & Ubiquitous Computing & Emerging

Technologies (CUBE), 2013 International Conference on, pages 8–13. IEEE, 2013.

[54] Yongqiang Gao, Haibing Guan, Zhengwei Qi, Yang Hou, and Liang Liu. A multi-objective

ant colony system algorithm for virtual machine placement in cloud computing. Journal of

Computer and System Sciences, 79(8):1230–1242, 2013.

[55] Jan-Jan Wu, Pangfeng Liu, and Jyun-Shiung Yang. Workload characteristics-aware virtual ma-

chine consolidation algorithms. In Proceedings of the 2012 IEEE 4th International Conference

on Cloud Computing Technology and Science (CloudCom), pages 42–49. IEEE Computer So-

ciety, 2012.

[56] Weiwei Fang, Xiangmin Liang, Shengxin Li, Luca Chiaraviglio, and Naixue Xiong. Vmplanner:

Optimizing virtual machine placement and trafﬁc ﬂow routing to reduce network power costs in

cloud data centers. Computer Networks, 57(1):179–196, 2013.

[57] Jixiang Cheng, Gary G. Yen, and Gexiang Zhang. A many-objective evolutionary algorithm

based on directional diversity and favorable convergence. In Systems, Man and Cybernetics

(SMC), 2014 IEEE International Conference on, pages 2415–2420, Oct 2014.

[58] Marco Farina and Paolo Amato. On the optimal solution deﬁnition for many-criteria optimization

problems. In Proceedings of the NAFIPS-FLINT international conference, pages 233–238, 2002.

[59] Kalyanmoy Deb, Ankur Sinha, and Saku Kukkonen. Multi-objective test problems, linkages,

and evolutionary methodologies. In Proceedings of the 8th annual conference on Genetic and

evolutionary computation, pages 1141–1148. ACM, 2006.

[60] Christian von Lücken, Benjamín Barán, and Carlos Brizuela. A survey on multi-objective evolu-

tionary algorithms for many-objective problems. Computational Optimization and Applications,

pages 1–50, 2014.

[61] Luiz André Barroso and Urs Hölzle. The case for energy-proportional computing. IEEE com-

puter, 40(12):33–37, 2007.

[62] Jiankang Dong, Xing Jin, Hongbo Wang, Yangyang Li, Peng Zhang, and Shiduan Cheng. Energy-

saving virtual machine placement in cloud data centers. In Cluster, Cloud and Grid Computing

(CCGrid), 2013 13th IEEE/ACM International Symposium on, pages 618–624. IEEE, 2013.

22 Fabio López-Pires and Benjamín Barán

[63] Shao-Heng Wang, Patrick P-W Huang, Charles H-P Wen, and Li-Chun Wang. Eqvmp: Energy-

efﬁcient and qos-aware virtual machine placement for software deﬁned datacenter networks.

In Information Networking (ICOIN), 2014 International Conference on, pages 220–225. IEEE,

2014.

[64] Nick McKeown, Tom Anderson, Hari Balakrishnan, Guru Parulkar, Larry Peterson, Jennifer

Rexford, Scott Shenker, and Jonathan Turner. Openﬂow: enabling innovation in campus net-

works. ACM SIGCOMM Computer Communication Review, 38(2):69–74, 2008.

[65] Orna Agmon Ben-Yehuda, Muli Ben-Yehuda, Assaf Schuster, and Dan Tsafrir. Deconstructing

Amazon EC2 spot instance pricing. ACM Transactions on Economics and Computation, 1(3):16,

2013.

[66] Wubin Li, Petter Svärd, Johan Tordsson, and Erik Elmroth. Cost-optimal cloud service placement

under dynamic pricing schemes. In Proceedings of the 2013 IEEE/ACM 6th International

Conference on Utility and Cloud Computing, pages 187–194. IEEE Computer Society, 2013.

[67] Kangkang Li, Jie Wu, and Adam Blaisse. Elasticity-aware virtual machine placement for cloud

datacenters. In Cloud Networking (CloudNet), 2013 IEEE 2nd International Conference on,

pages 99–107. IEEE, 2013.

[68] Mayank Mishra and Anirudha Sahoo. On theory of vm placement: Anomalies in existing

methodologies and their mitigation using a novel vector based approach. In Cloud Computing

(CLOUD), 2011 IEEE International Conference on, pages 275–282. IEEE, 2011.

[69] Tiago Ferreto, César AF De Rose, and Hans-Ulrich Heiss. Maximum migration time guaran-

tees in dynamic server consolidation for virtualized data centers. In Euro-Par 2011 Parallel

Processing, pages 443–454. Springer, 2011.

[70] Diego Ihara, Fabio López-Pires, and Benjamín Barán. Many-objective virtual machine placement

for dynamic environments. In Proceedings of the 2015 IEEE/ACM 8th International Conference

on Utility and Cloud Computing. IEEE Computer Society, 2015.

[71] Mohammad Mehedi Hassan, M Shamim Hossain, AM Jehad Sarkar, and Eui-Nam Huh. Co-

operative game-based distributed resource allocation in horizontal dynamic cloud federation

platform. Information Systems Frontiers, 16(4):523–542, 2014.