ArticlePDF Available

Abstract and Figures

Infrastructure as a service clouds hide the complexity of maintaining the physical infrastructure with a slight disadvantage: they also hide their internal working details. Should users need knowledge about these details e.g., to increase the reliability or performance of their applications, they would need solutions to detect behavioural changes in the underlying system. Existing runtime solutions for such purposes offer limited capabilities as they are mostly restricted to revealing weekly or yearly behavioural periodicity in the infrastructure. This article proposes a technique for predicting generic background workload by means of simulations that are capable of providing additional knowledge of the underlying private cloud systems in order to support activities like cloud orchestration or workflow enactment. Our technique uses long-running scientific workflows and their behaviour discrepancies and tries to replicate these in a simulated cloud with known (trace-based) workloads. We argue that the better we can mimic the current discrepancies the better we can tell expected workloads in the near future on the real life cloud. We evaluated the proposed prediction approach with a biochemical application on both real and simulated cloud infrastructures. The proposed algorithm has shown to produce significantly (~20%) better workload predictions for the future of simulated clouds than random workload selection.
Content may be subject to copyright.
Cloud workload prediction based on workflow execution time
Gabor Kecskemeti
Zsolt Nemeth
Attila Kertesz
Rajiv Ranjan
Received: 27 October 2017 / Revised: 1 February 2018 / Accepted: 23 March 2018 / Published online: 16 October 2018
ÓThe Author(s) 2018
Infrastructure as a service clouds hide the complexity of maintaining the physical infrastructure with a slight disadvantage:
they also hide their internal working details. Should users need knowledge about these details e.g., to increase the reliability
or performance of their applications, they would need solutions to detect behavioural changes in the underlying system.
Existing runtime solutions for such purposes offer limited capabilities as they are mostly restricted to revealing weekly or
yearly behavioural periodicity in the infrastructure. This article proposes a technique for predicting generic background
workload by means of simulations that are capable of providing additional knowledge of the underlying private cloud
systems in order to support activities like cloud orchestration or workflow enactment. Our technique uses long-running
scientific workflows and their behaviour discrepancies and tries to replicate these in a simulated cloud with known (trace-
based) workloads. We argue that the better we can mimic the current discrepancies the better we can tell expected
workloads in the near future on the real life cloud. We evaluated the proposed prediction approach with a biochemical
application on both real and simulated cloud infrastructures. The proposed algorithm has shown to produce significantly
(20%) better workload predictions for the future of simulated clouds than random workload selection.
Keywords Workload prediction Cloud computing Simulation Scientific workflow
1 Introduction
Infrastructure as a Service (IaaS) clouds became the
foundations of compute/data intensive applications [2].
They provide computational and storage resources in an on
demand manner. The key mechanism of IaaS is virtuali-
sation that abstracts resource access mechanisms with the
help of Virtual Machines (VM) allowing their users to
securely share physical resources. While IaaS clouds offer
some means to control a virtual ensemble of resources (so
called virtual infrastructures), they inherently provide no
means for precise insight into the state, load, performance
of their resources, thus the physical layer is completely
hidden. Due to the multi-tenant environment of clouds,
application performance may be significantly affected by
other, (from the point of view of a particular user) unknown
and invisible processes, the so-called background work-
load. Albeit Service Level Agreements (SLA) define the
expected specifics and various Quality of Service (QoS)
methods are aimed at their fulfilment, yet they can provide
&Gabor Kecskemeti
Zsolt Nemeth
Attila Kertesz
Rajiv Ranjan
Department of Computer Science, Liverpool John Moores
University, Liverpool L3 3AF, UK
Laboratory of Parallel and Distributed Systems, MTA
SZTAKI, Budapest 1111, Hungary
Software Engineering Department, University of Szeged,
Szeged 6720, Hungary
School of Computing, Newcastle University,
Newcastle upon Tyne NE4 5TG, UK
Cluster Computing (2019) 22:737–755,-volV)(0123456789().,-volV)
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
a very broad range of performance characteristics only
This article studies performance issues related to the—
unknown—background load and proposes a methodology
for its estimation. We envision a scenario where modifi-
cations in the virtual infrastructure are necessary at runtime
and to make the right decisions and take actions the
background load cannot be omitted. As follows, we made
two assumptions: (i) the application runs long enough so
that the time taken by a potential virtual infrastructure re-
arrangement is negligible and (ii) the application is exe-
cuted repeatedly over a period of time. Both these
assumptions are valid for a considerably large class of
cloud based applications. Scientific workflows are espe-
cially good candidates for exemplifying this class as they
are executed in numerous instances by large communities
over various resources [20]. During execution, jobs of a
workflow are mapped onto various resources e.g., a parallel
computer, a cluster, a grid, a virtual infrastructure on a
cloud, etc. Efficient execution of workflows requires a
precise scheduling of tasks and resources which further-
more, requires both timely information on the resources
and the ability to control them. Thus we have chosen sci-
entific workflows as a subject and evaluation example of
our method.
The recurring nature of workflows enables the extraction
of performance data and also successive adaptation,
refinement and optimisation leading to dynamic workflow
enactment. The main motivation for our work stems from
the assumption that by extracting information from past
workflow executions, one could identify current and pre-
dict future background workloads of the resources allo-
cated for the workflow. The result of this prediction
subsequently enables to steer current and future cloud
usage accordingly, including the option of resource re-ar-
rangement if indicated. The idea is centred around a set of
past load patterns (a database of historic traces). When a
workflow is being enacted, some of its jobs have already
been executed and some others are waiting for execution.
Our workload prediction aims at finding historic traces, that
likely resemble the background of workload behind the
currently running workflow. Hence, future tasks (even
those that are completely independent from the workflow
that was used for the prediction) are enacted taking into
consideration the recent background load estimations.
The main contributions of this article are: (i) the concept
of a private-cloud level load prediction method based on
the combination of historic traces, aimed at improving
execution quality (ii) an algorithm for realising the load
prediction at runtime so that performance constraints are
observed, and (iii) an evaluation of this approach using a
biochemical application with simulations using historic
traces from a widely used archive.
The remainder of this article is as follows: Sect. 2pre-
sents related work, then Sect. 3introduces the basic ter-
minology and assumptions of our research. Section 4
introduces our new algorithm. Section 5presents its eval-
uation with a biochemical application. Finally, the contri-
butions are summarised in Sect. 6.
2 Related work
In this article, we examine past traces of certain workflows,
and predict the expected background load of the clouds
behind current workflow instances. Our technique fits in
the analyse phase of autonomous control loops (like
monitor-analyse-plan-execute [8]). Similarly, Maurer et al.
[18] investigated adaptive resource configuration from a
SLA/QoS point of view using such a loop. In their work,
actions to fine tune virtual machine (VM) performance are
categorised hierarchically as so called escalation levels.
Generally, our work addresses a similar problem (our scope
is on the background workload level instead of infras-
tructure and resource management) with a different grained
action set for the plan-execute steps of the autonomous
Concerning workload modelling, Khan et al. [12] used
data traces obtained from a data centre to characterise and
predict workload on VMs. Their goal was to explore cross-
VM workload correlations, and predict workload changes
due to dependencies among applications running in dif-
ferent VMs—while we approach the load prediction from
the workflow enactment point of view.
Li et al. [14] developed CloudProphet to predict legacy
application performance in clouds. This tool is able to trace
the workload of an application running locally, and to
replay the same workload in the cloud for further investi-
gations and prediction. In contrast, our work presents a
technique to identify load characteristics independent from
the workflow ran on cloud resources.
Fard et al. [7] also identified performance uncertainties
of multi-tenant virtual machine instances over time in
Cloud environments. They proposed a model called
R-MOHEFT that considered uncertainty intervals for
workflow activity processing times. They developed a
three-objective (i.e., makespan, monetary cost, and
robustness) optimisation for Cloud scheduling in a com-
mercial setting. In contrast to this approach our goal is to
identify patterns in earlier workloads to overcome the
uncertainty, and apply simulations to predict future back-
ground load of the infrastructure.
Calheiros et al. [5] offers cloud workload prediction
based on autoregressive integrated moving average. They
argue that proactive dynamic provisioning of resources
could achieve good quality of service. Their model’s
738 Cluster Computing (2019) 22:737–755
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
accuracy is evaluated by predicting future workloads of
real request traces to web servers. Additionally, Magalhaes
et al. [15] developed a workload model for the CloudSim
simulator using generalised extreme value/lambda distri-
butions. This model captures user behavioural patterns and
supports the simulation of resource utilisation in clouds.
They argue that user behaviour must be considered in
workload modelling to reflect realistic conditions. Our
approach share this view: we apply a runtime behaviour
analysis to find a workflow enactment plan that best mat-
ches the infrastructure load including user activities.
Caron et al. [6] used workload prediction based on
identifying similar past occurrences of the current short-
term workload history for efficient resource scaling. This
approach is the closest to ours (albeit, we have a different
focus support for on-line decision making in scientific
workflow enactors etc.), as it uses real-world traces from
clouds and grids. They examine historic data to identify
similar usage patterns to a current window of records, and
their algorithm predicts the system usage by extrapolating
beyond the identified patterns. In contrast, our work’s
specific focus on scientific workflows allows the analysis
and prediction of recently observed execution time dis-
crepancies, by introducing simulations to the prediction
and validation phases.
Pietri et al. [19] designed a prediction model for the
execution time of scientific workflows in clouds. They map
the structure of a workflow to a model based on data
dependencies between its nodes to calculate an estimated
makespan. Though the goal of this paper, i.e. to determine
the amount of resources to be provisioned for better
workflow execution based on the proposed prediction
method is the same in our article, we rely on the runtime
workflow behaviour instead of its structure. This means we
aim to predict the background load instead of the execution
time of a workflow.
Besides these approaches, Mao et al. proposed a com-
bined algorithm in [16] for prediction, arguing that a single
prediction algorithm is not able to estimate workloads in
complex cloud computing environments. Therefore they
proposed a self-adaptive prediction algorithm combining
linear regression and neural networks to predict workloads
in clouds. They evaluated their approach on public cloud
server workloads, and the accuracy of their results on
workload predictions are better compared to purely neural
network or linear regression-based approaches.
Brito and Arau
´jo [3] proposed a solution for estimating
infrastructure needs of MapReduce applications. Their
suggested model is able to estimate the size of a required
Hadoop cluster for a given timeframe in cloud environ-
ments. They also presented a comparative study using
similar applications and workloads in two production
Hadoop clusters to help researchers to understand the
workload characteristics of Hadoop clusters in production
environments. They argued that the increased sharing of
physical computing host resources reduces the accuracy of
their model. In this paper we exploit these inaccuracies in
order to provide insights into the inner working of the
utilised shared cloud resources.
´et al. [17] stated that estimating the makespan of
workflows is necessary to calculate the cost of execution,
which can be done with the use of Cloud simulators. They
argued that an ideal simulated environment produces the
same output for a workflow schedule, therefore it cannot
reproduce real cloud behavior. To address this requirement,
they proposed a method to add noise to the simulation in
order to equalise the simulated behavior with the real cloud
one. They also performed a series of experiments with
different workflows and cloud resources to evaluate their
approach. Their results showed that the inaccuracy of the
mean value of the makespan was significantly reduced
compared to executions using the normal distribution. A
similar variance based workflow scheduling technique was
investigated by Thaman and Singh [21], where they eval-
uated dynamic scheduling policies with the help of
WorkflowSim. In both cases, the authors added artificial
variance for workflow’s tasks. In contrast, we do not only
introduce realistic background workload, but expose the
matched workload as a characterisation of the used cloud.
3 Background
An enactment plan describes the jobs of a scientific
workflow, their schedule to resources, and it is processed
by a workflow enactor that does the necessary assignments
between jobs and resources. If a workflow enactor is cap-
able to handle dynamic environments [4], such as clouds,
the resources form a virtual infrastructure (crafted to serve
specific jobs). In our vision, the enactment plan also lists
the projected execution time of each job in the workflow.
Workflow enactors are expected to base the projected
execution time on historic executions to represent their
expectations wrt. the job execution speed. This enactment
extension allows the workflow enactor to offer background
knowledge on the behaviour past runs of the workflow that
combined the use of various distinct inputs and resource
characteristics. As a result, during the runtime of the
workflow, infrastructure provisioning issues could be pin-
pointed by observing deviations from the projected exe-
cution time in the enactment plan.
The virtual infrastructures created by the enactor are
often hosted at IaaS cloud providers that tend to feature
multi-tenancy and under provisioning for optimal costs and
resource utilisation. These practices, especially under pro-
visioning, could potentially hinder the virtual
Cluster Computing (2019) 22:737–755 739
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
infrastructure’s performance (and thus the execution times
of jobs allocated to them). In accordance with the first
phase (monitor) of autonomous control loops, to maintain
the quality and to meet the SLAs set out for the virtual
infrastructure in the enactment plan, the workflow enactor
or a third party service continuously monitors the beha-
viour of the applications/services/workflows running on the
virtual infrastructure. In case of deviations, actions in the
management of the virtual infrastructure should take place,
such as adding or removing new computing/storage com-
ponents, to minimize fluctuations in the quality of execu-
tion (note: these reactive actions are out of scope of this
article). We assume sufficiently small, likely private, cloud
infrastructures where the workflow instances could expe-
rience significant enough portion of the whole infrastruc-
ture allowing the exploitation of the identified deviations
for prediction purposes.
We represent workflows W2W(where Wis the set of
all possible abstract workflows) as an ordered set of jobs:
W¼fj1...jNg, where the total number of jobs in the
workflow is N2N. The job order is set by their projected
completion time on the virtual infrastructure whereas the
job inter-relations (dependencies) are kept in the domain of
the workflow enactors. The projected execution time of the
a job (jx2W)isrexðjxÞ where rex :W!Rþ. We expect
the enactor to calculate the projected execution times based
on its background knowledge about thousands of past runs.
We refer to a workflow instance (i.e., a particular exe-
cution of the abstract workflow W) with the touple:
½W;t:WT—i.e., the workflow and the start time (tand
Tdepicts the set of all time instances) of its first job ½j1;t.
Hence, all instances of jx2Ware also identified as
½jx;t:jxW;T. Once the workflow started, the enac-
tor’s monitoring facilities will collect the observed exe-
cution times for each job instance. We denote these as:
robðjx;tÞ—where rob :WT !Rþ.
Using the acquired data from the enactors and its
monitoring facilities, we define the error function of
(partial) workflow execution time to determine the devia-
tion in execution time of a particular workflow suffered
compared to the projected times in the enactment plan.
Such function is partial if the evaluated workflow instance
is split into two parts: jobs j1; :::jkalready executed whereas
jkþ1; :::jNare not yet complete or waiting for execution;
when k¼Nthe workflow instance is done and the error
function determines its final execution time error. So in
general, the error function of workflow execution time is
defined as: E:WTN!Rþ.
We require that error functions assign higher error val-
ues for workflow instances that deviate more from the
projected runtime behaviour set in their enactment plan.
These functions should also penalise both positive and
negative execution time differences ensuring that the exe-
cution strictly follows the plan. The penalties applied here
will allow us to detect if the background workload reduces/
improves the performance of the workflow instance com-
pared to the enactor’s expectations. These penalties are
exploited by the later discussed workload prediction tech-
nique: it can tell if a particular workload estimate is not
sufficiently close to its actual real life counterpart. For
example, when the execution times show improvements—
negative differences—under a particular workload esti-
mates, then the prediction technique knows it still has a
chance to improve its estimate (allowing other, not nec-
essarily long running, applications to better target the
expected background workload on the cloud of the
The penalties are also important from the point of view
of the workflow enactor. The enactment plan likely con-
tains projected values resulted from several conflicting
service level requirements (regarding the quality of the
execution). These projected values are carefully selected by
the enactor to meet the needs of the workflow’s user and
follow the capabilities of the used cloud resources. Thus,
error functions should indicate if the fulfilment of the
projected values set by the enactor are at risk (i.e., they
should penalize with higher error values even if the
observed execution times turned out to be better than
originally planned). For example, if we have jobs jxand jy
where jyis dependent on the output of jxand several other
factors, these factors could make it impossible for jyto be
ready for execution by a the time a better performing jx
completes. Thus, the enactor could make a plan relaxed for
jxand explicitly ask for its longer job execution time. If the
job is executed more rapidly in spite of this request, the
penalty of the error function would show there were some
unexpected circumstances which made the job faster.
The deviation from the projected execution times (as
indicated by the error function) could either be caused by
(i) an unforeseen reaction to a specific input, or by (ii) the
background load behind the virtual infrastructure of the
workflow. In case (i), the input set causes the observed
execution times to deviate from the projected ones. Such
deviations are rare, because job execution times on a
dedicated infrastructure (i.e., only dependent on input
values) usually follow a Pareto distribution [1]. Thus, job
execution times mostly have small variances, but there
could be jobs running several orders of magnitude slower
than usual. Figure 1exemplifies this behaviour with a
sample of over 20 k jobs ran for various cloud simulation
workflows. As it can be observed on the example figure,
the long execution times in the slowest 5 % of the jobs
cannot be mistaken for perturbations caused by background
740 Cluster Computing (2019) 22:737–755
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
On the other hand (ii), under-provisioning in IaaS clouds
can cause significant background load variation yielding
observable (but minor) perturbations in job execution
times. In this article, we focus on case (ii) only, therefore
we must filter observed execution times whether they
belong to case (i) or (ii). Consequently, when observing a
significant increase in job execution time (i.e., enactor’s
predicted execution time is a magnitude smaller than what
was actually observed), we assume that the particular job
belongs to case (i) and we do not apply our technique.
However, when we only observe minor deviations from our
execution time expectations, we assume that they are of
case (ii), caused by the under-provisioned cloud behind the
virtual infrastructure executing the observed jobs.
Below, we present a few workflow execution time error
functions that match the above criteria. Later, if we refer to
a particular function from below, we will use one of the
subscripted versions of E, otherwise, when the particular
function is not relevant, we just use Ewithout subscript.
Although the algorithm and techniques discussed later are
independent from the applied Efunction, these functions
are not interchangeable, their error values are not compa-
rable at all.
Average distance. This error function calculates the
average time discrepancy of the first kjobs.
P1ikðrexðjiÞrob ðji;tÞÞ2
Mean absolute percentage error. Here the relative error
of the observed runtime is calculated for each job, then it
is averaged for all kjobs:
jrexðjiÞrob ðji;tÞj
Time adjusted distance. The function adjusts the execu-
tion time discrepancies calculated in ESQD so that the
jobs started closer (in time) to jkwill have more weight
in the final error value.
kðrexðjiÞrob ðji;tÞÞ2
4 Workflow enactment and simultaneous
When job jkis completed during the execution (phase I in
Fig. 2), a deviation analysis is performed using one of the
error functions of Eqs. 13to compare the actual job
execution times to the ones in the enactment plan. Signif-
icant deviations—EðW;t;kÞ[E, where Eis predefined
by the workflow developer—initiate the background
workload prediction phase that corresponds to the second,
Analysis phase of autonomous control loops. This phase is
omitted, if the workflow enactor estimates the remaining
workflow execution time is smaller than required for
background workload prediction. The maximum time spent
on workload prediction is limited by a predefined T, rep-
resented as a gap in the execution in Fig. 2. Thus, workload
prediction is not performed if PN
4.1 Background workload prediction
In essence, we simulate the workflow execution on a given
cloud infrastructure while adding known workloads as
background load (phase II in Fig. 2). The workflow is sim-
ulated according to the enactment plan specified runtime
properties, like job start time, completion time, and times for
creating virtual machines. We expect the simulated work-
flow to match its real-world counterpart (in terms of runtime
properties), when the added background load closely esti-
mates the real-world load. We use Eqs. 13to find the known
workload closest to the observed one (note: only one function
should be used during the whole prediction procedure). Next,
we present the details of Algorithm 1 that implements this
background workload matching mechanism.
4.1.1 Base definitions
Before diving into the details, we provide a few important
definitions: trace fragment—used to provide a particular
background workload –, past error—determines the pre-
viously collected execution time error values regarding the
completed jobs j1; :::jk—finally, future error—defines the
previously collected execution error evaluations of the jobs
jkþ1; :::jNthat have not yet run in the current workflow
instance (i.e., what was the level of error in the ‘‘future’
that after a particular past error value was observed).
Atrace fragment is a list of activities characterised by
such runtime properties (e.g., start time, duration,
Fig. 1 Example distribution of job execution times in a scientific
workflow (where 95% of the execution time is spent on 5 % of the
Cluster Computing (2019) 22:737–755 741
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
performance, etc.) that are usable in simulators, their col-
lection is denoted as fragment database in Fig. 2. Each
fragment represents realistic workloads i.e., real-world
system behaviour. Fragments are expected to last for the
duration of the complete simulation of the workflow with
all its jobs. The fragment duration is independent from the
actual real life situation modelled—which stems from the
actual ½jk;tjob which triggered the prediction. In a worst
case, fragments should last for the completely serial exe-
cution of the workflow: Pi¼1...krobðjiÞþPi¼kþ1...Nrex ðjiÞ.
Thus fragment durations vary from workflow-to-workflow.
Apart from their duration, fragments are also characterised
and identified by their starting timestamp, i.e., the time
instance their first activity was logged, denoted as t2T
(where TT); later we will refer to particular fragments
by their identifying starting timestamp (despite these
fragments often-times contain thousands of activities). As a
result, when the algorithm receives an identifying times-
tamp, it queries the trace database for all the activities that
follow the first activity for the whole duration of the
fragment. Note, that our algorithm uses the relative posi-
tion of these timestamps. Therefore, when storing historic
traces as fragments, they are stored so that their timestamps
are consistent and continuous, this requires some dis-
placement of their starting positions. This guarantees that
we can vary the fragment boundaries (according to the
workflow level fragment duration requirements) at will.
Arbitrary selection of fragment boundaries would result
in millions of trace fragments. If we would simulate with
every possible fragment, the analysis of a single situation
would take days. However, as with any prediction, the
longer time it takes the less valuable its results become (as
the predicted future could turn past by then). Predictions
typically are only allowed to run for a few minutes as a
maximum, thus the entire simulation phase must not hinder
the real execution for more than T. In the following, we
survey the steps that are necessary to meet this requirement
i.e., reducing the analysis time from days to T. The frag-
ment database needs to be pre-filtered so only a few
fragments (Tfilt T) are used in the analysis later on.
Although this is out of scope of the current article, pre-
filtering can use approaches like pattern matching, runtime
behaviour distance minimisation (e.g., by storing past
workflow behaviour—for particular fragments—and by
comparing to the current run to find a likely start timestamp),
or even random selection. Filtering must take limited and
almost negligible time. In our experiments, we assumed it
below T=1000 (allowing most of the time to be dedicated to
the simulation based analysis of the situation). As a result of
filtering, the filtered fragment count must be reduced so that
the time needed for subsequent simulations does not exceed
the maximum time for predictions: jTfilt j\T=tsimðWÞ,
where tsim is the mean execution time of Win the simulated
cloud. Our only expectation that the pre-filtering happens
only in memory and thus the fragment database is left intact.
As a result, when we run the algorithm, it only sees a portion
of the fragment database. On the other hand, future runs of
the algorithm might get a different portion from the database
depending on the future runtime situation.
Finally, we dive into the error definitions. Alongside
fragments, several error values are stored in the fragment
database, but unlike fragments, which are independent
from workflows, these error values are stored in relation to
a particular workflow and its past instances. Later, just like
execuon plan
Phase IIIPhase II
execuon steering according
to the predicted load
Fig. 2 Phases of our workload
prediction in relation to the
workflow being ran
742 Cluster Computing (2019) 22:737–755
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
projected execution times, these stored error values are also
used to steer the algorithm. First, in terms of past errors,
we store the values from our previous partial execution
time error functions, Eqs. 13. Past errors are stored for
every possible kvalue for the particular workflow instance.
We calculate future errors similarly to past errors. Our
calculation uses the part of the workflow containing the
jobs after jk:WFðW;kÞ:¼f8ji2W:i[k^iNg,
where WF2W. Thus, the future error function determines
how a particular, previously executed workflow instance
continued after a specific past error value:
This function allows the evaluation and storage of the final
workflow execution time error for those parts of the past
workflow instances, which have not been executed in the
current workflow execution.
4.1.2 Overview of the algorithm
Algorithm 1 Fitting based prediction
Require: Tfilt T– the filtered trace fragment set
Require: SR+– the primary search window size
Require: ΠR+– the precision of the trace match
Require: IN– the maximum iteration count
Require: PR+– max evaluations for searching in func-
tion φ(x)
Require: [W, tcurr ] – the current workflow instance
Require: Rex := {rex(ji):{jiW}} – the model execu-
tion times
Require: Rob := {rob(tcurr ,j
i):{jiWik}} – the
observed execution times
Ensure: ttarget is around the approximated workload
1: tinit tTfilt
2: Tlist ←∅
3: repeat
4: I(tinit S/2,t
init +S/2)
5: R2I\{∅} – arbitrary choice
6: for all tRdo
7: for all jiW:i<k do
8: r
ex(ji)rob (tcurr,j
9: r
ob(ji,t)sim(W, Rex,i,t)
10: end for
11: end for
12: Tred 2T\Tlist :|Tred|=P
13: Tmin ←{tT|φ(t) = min
14: tmin min Tmin
15: Tlist Tlist ∪{tmin}
16: ttarget (|Tlist|)←{tlT:F(W, tl,k)E(W, tl,k)=
tminS/2<t<tmin +S/2(F(W, t, k)E(W, t, k))}
17: tinit txTfilt :|txttar get(|Tlist|)|=
mintTfilt |tttarget(|Tlist |)|
18: until (|ttarget(|Tlist |)ttarget(|Tlist |−1)|)
19: return ttareget (|Tlist|)
Algorithm 1 (also depicted functionally in Fig. 3and
structurally as phases II–III in Fig. 2) aims at finding a
timestamp so that the future estimated error is minimal,
while past error prediction for this timestamp is the closest
to the actual past error (i.e., the estimated and actual ‘‘past
errors are aligned’’). The algorithm is based on the
assumption that if past workloads are similar (similarity
measured by their error functions) then future workloads
would be similar, too.
In detail, line 1 picks randomly one of the fragments
identified by the timestamp in the filtered set Tfilt and stores
in tinit. This will be the assumed initial location of the frag-
ment that best approximates the background load. Later, in
line 17, this tinit will be kept updated so it gives a fragment
that better approximates the background load. The primary
search windowRof line 5 also shown between the dashed
lines of the lower chart in Fig. 3represents a set of times-
tamps within a S/ 2 radius from the assumed start of the
fragment specified by tinit . The algorithm uses set Tlist to
store timestamps for the approximate trace fragments as well
as to count the iterations (used after line 15).
A simulator is used to calculate observed execution times
ob for the jobs in the simulated infrastructure (see line 9).
This is expressed with simðW;Rex;i;tÞthus, each simulation
receives the workflow to be simulated, the set of execution
time expectations (Rex) that specify the original enactment
plan, the identifier of the job (1 iN) we are interested in
and the timestamp of the trace fragment (t2T) to be used in
Iteration N
eration N
n N
Iteration N
Iteration N
ation N
on N
tion N
lines 1,17 ->
lines 17,18 ->
line 12 ->
lines 13,14 ->
line 9 ->
line 4 ->
<- line 16
<- line 17
S/2 S/2
Fig. 3 Visual representation of the algorithm
Cluster Computing (2019) 22:737–755 743
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
the simulation as background load to the workflow, respec-
tively. With these parameters the simulator is expected to run
all the activities in the trace fragment identified by tin par-
allel to a simulated workflow instance. Note: the simulation
is done only once for the complete workflow for a given
infrastructure and a given fragment, later this function sim-
ply looks up the past simulated r0
ob values.
Next, we use one of the error functions of the workflow
execution time as defined in Eqs. 13. As we have simu-
lated results, we substitute the observed/expected execution
time values in the calculation with their simulated coun-
terparts. In the simulation, the expected execution times r0
are set as the real observed execution times rob (see line 8).
To denote this change in the inputs to the error function, we
use the notation of E0ðW;t;kÞ—error of simulated execu-
tion time. This function shows how the simulated workload
differs from the observed one. The evaluation of the E0
function is depicted with marks at the bottom chart of
Fig. 3.
Afterwards, lines 12–14 search through the past error
values for each timestamp (using the same error function as
we used for the evaluation of E0). With the help of function
t2RE0ðW;t;kÞEðW;xþttinit þS=2;kÞð5Þ
This function offers the difference between the simulated
and real past error functions (Fig. 3represents this with red
projection lines between the chart of Eand the marks of
E0). The algorithm uses the /ðxÞfunction to find the best
alignment between the simulated and real past error func-
tions: we set tmin as the time instance in Rwith the smallest
difference between the two error functions. The alignment
is searched over an arbitrary subset of the timestamps:
Tred—the secondary search window. The algorithm selects
aTred with a cardinality of Pin order to limit the time to
search for tmin. The arbitrary selection of Tred is used to
properly represent the complete timestamp set of T.
After finding tmin, we have a timestamp from the frag-
ment database, for which the behaviour of the future error
function is in question, this corresponds to the fragment
selection in Fig. 2. Line 16 finds the timestamp that has the
closest past and future error values in the range around tmin
within radius S/ 2—see also in the top right chart of
Fig. 3. Note, this operation utilises our assumption that past
and future errors are aligned (i.e., a trace fragment with
small past error value is more likely to result in similarly
small future error value). The timestamp with the future
error value closest to the past error is used as ttarget for the
current iteration (i.e., our current estimate for the start
timestamp of the approximate background load).
Finally, the iteration is repeated until the successive
change in ttarget is smaller than the precision Por the
iteration count reaches its maximum—I, represented as
phase III in Fig. 2. Note, Iis set so the maximum time
spent on workload prediction (T) is not violated. The
algorithm then returns with the last iteration’s ttarget value
to represent the starting timestamp of the predicted trace
fragment that most resembles the background load cur-
rently experienced on the cloud behind the workflow. This
returned value (and the rest of the trace fragment following
ttarget) then could be reused by when utilizing the real life
version of the simulated cloud. For example, the workflow
enactor could use the knowledge of the future expected
workload for the planning and execution phases of its
autonomous control loop (phase IV in Fig. 2). Note, the
precise details on the use of the predicted workload is out
of the scope of this article.
5 Evaluation with a biochemical workflow
We demonstrate our approach via a biochemical workflow
that generates conformers by unconstrained molecular
dynamics at high temperature to overcome conformational
bias then finishes each conformer by simulated annealing
and/or energy minimisation to obtain reliable structures. It
uses the TINKER library [11] for molecular modelling for
QSAR studies for drug development.
Our evaluation approach is summarized in Fig. 4.Itis
composed of three main phases: (i) data collection from a
real life environment, (ii) modelling the TCG workflow and
simulating its behaviour under various background loads,
and (iii) evaluating the algorithm based on the collected
real life and simulated data.
5.1 The tinker workflow on the LPDS cloud
The TINKER Conformer Generator (TCG) workflow [11]
consists of 6 steps (see the top left corner of Fig. 4): (i) G:
generating 50,000 input molecule conformers (taking
around 12 h, compressed into 20 zip files by grouping 2500
conformers); (ii) T1: minimising the initial conformational
states generated at high temperature; (iii) T2: performing a
short low temperature dynamics with the high temperature
conformations to simulate a low temperature thermody-
namic ensemble, and minimising the above low tempera-
ture states; (iv) T3: cooling the high temperature states by
simulated annealing, and minimising the annealed states;
(v) TC: collecting parameter study results; (vi) C: re-
compressing results to a single file. The sequential execu-
tion of the workflow on a single core 2 GHz CPU takes
around 160 h.
744 Cluster Computing (2019) 22:737–755
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Note that the parallel section of the workflow could be
partitioned arbitrarily: by changing how Gsplits the input
molecule conformers. For example, any options could be
chosen from the extreme case of each molecule becoming
an input for a separate T1/2/3/C run, to the other extreme
of having a single zip file produced and processed in a
single T1/2/3/C. Also, if a larger infrastructure is available,
more input molecule conformers could be considered in a
single run (this would result in a longer execution time for
In the period of over half a year, the workflow was ran
several times on the cloud of the Laboratory of Parallel and
Distributed Systems—LPDS cloud—, which ran OpenNe-
bula 4.10 and consisted of 216 cores, 604 GBs of memory
and 70 TBs of storage at the time of the experiments. We
used a workflow enactor without autonomous control
mechanisms. We have collected the job execution times for
all jobs in the workflow, as well as the time instance when
the workflow was started. We have calculated the expected
job execution times—rexðjiÞ—as an average of the execu-
tion times observed. This average was calculated from over
500 runs for each step of the TCG workflow. To enable a
more detailed analysis of the workflow executions, we have
generated larger input sets allowing us to repeatedly exe-
cute the parallel section with 20 virtual machines (in our
implemented workflow, the 20 machine parallel section
was executed 15 times before concluding with the final re-
compression phase—C). This allowed us to populate our
initial past and future error values in the cache (i.e., we
have calculated how particular workflow instances behaved
when expected job execution times are set to be the aver-
age of all). Not only the error cache was populated though,
the individual robðji;tÞvalues were also stored in our
database (in total, the collected data was about 320 MBs).
These data stores are shown in Fig. 4as a per instance
expected runtimes database, job execution logs) and Past/
Future error cache. The stored values acted as the foun-
dation for the simulation in the next phase of our
5.2 Modelling and simulating the workflow
This section provides an overview on how the TCG
workflow was executed in a simulator. Our choice for the
simulator was the open source DISSECT-CF.
We have
chosen it, because it is well suited for simulating resource
bottlenecks in clouds, it has shown promising performance
gains over more popular simulators (e.g., CloudSim, Sim-
Grid) and its design and development was prior work of the
authors [10]. The section also details the captured proper-
ties of the TCG workflow, which we collected in previous
phase of the evaluation. Then, as a final preparatory step
Real -l ife da ta collecti on
LPDS Cloud
Other cloud users
TCG Us ers
TCG Work ow
Work ow enactor
Simulated data generation
Error funcon evaluaon
past errors
future errors
Error func SQD
Error func TAdj-SQD
Error func MAPE
error cache
Simul. Job
192 VMs for parallel trace evaluaon
Algorithm behavior data collecon
Golden set
Predict workload
with Algorithm #1
Predict workload
with Random selecon
Predicons for
each parameter
con guraon
LJMU cloud Parametric study
for S, I, etc...
Algorithm behavior analysis
Execuon me
Error correllaon check
Algorithm execuon
me check
Fig. 4 Our detailed evaluation approach
Cluster Computing (2019) 22:737–755 745
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
for our evaluation, we present the technique we used to add
arbitrary background load to the simulated cloud that is
used by the enactor to simulate the workflow’s run.
5.2.1 The model of the workflow’s execution
The execution of the TCG workflow was simulated
according to the description presented in Fig. 5. The
description is split into three main sections, each starting
with a PSSTART tag (see lines 1, 6 and 20, which corre-
spond to the three main sections G–[T1/T2/T3–TC]–C of
the TCG workflow shown in the top left corner of Fig. 4).
This tag is used as a delimiter of parallel sections of the
workflow, thus everything that reside in between two
PSSTART lines should be simulated as if they were exe-
cuted in parallel. Before the actual execution though, every
PSSTART delimited section contains the definition of the
kind of VM that should be utilized during the entire parallel
section. The properties of these VMs are defined by the
VMDEF entry (e.g., see lines 2 or 8) following PSSTART
lines. Note, the definition of a VM is dependent on the
simulator used, so below we list the defining details
specific to DISSECT-CF:
The virtual machine image used as the VM’s disk. This
is denoted with property name VA. In this property, we
specify that the image is to be called ‘‘tinker’. Next, we
ask its boot process to last for 25 seconds. Afterwards,
we specify the VM image to be copied to its hosting PM
before starting the VM—0 (i.e., the VM should not run
on a remote filesystem). Finally, we set the image’s size
as 306 MBs.
The required resources to be allocated for the VM on its
hosting PM. These resources are depicted behind the
property name of RC in the figure. Here we provided
details for the number of cores (1), their performance
(5.0E-4—this is a relative performance metric com-
pared to one of the CPUs in LPDS cloud) and the
amount of memory (1 GB) to be associated with the
soon to be VMs.
Image origin where the VM’s disk image is downloaded
from before the virtual machine is instantiated. We used
the property name of VAST to tell the simulator the host
name of the image repository that originally stores the
VM’s image.
Data store is the source/sink of all the data the VM
produces during its runtime. This is defined with the
property called DATA. This field helps the simulation to
determine the target/source of the network activities later
depicted in the VMSEQ entries.
The real-life workflow was executed in the LPDS cloud
(see the leftmost section of Fig. 4). Thus, we needed to
model this cloud to match the simulated behaviour of the
workflow to its real life counterpart ran in phase one.
Therefore, the storage name iscsi izabel in the workflow
description (e.g., in line 2 of Fig. 5) refers to the particular
storage used on LPDS cloud, just like the VMI image name
tinker does.
Now, we are ready to describe the runtime behaviour of
the workflow observed in phase one in a format easier to
process by the simulation. This behaviour is denoted with
the VMSEQ entries (e.g., see lines 4 or 10) that reside in
each PSSTART delimited parallel section. VMSEQ entries
are used to tell the simulator a new VM needs to be
2| VMDEF VA=tinker,25,0,306176000 RC=1,5.0E-4,1073741824 VAST=iscsi-izabel DATA=iscsi-izabel
3| #This is the seq for job G
4| VMSEQ N50500 N190000 N334000 C22.333 N4578744 C0.5 N6150209 N1488 C43200
7| # VM instance description for the VMs to be used
8| VMDEF VA=tinker,25,0,306176000 RC=1,5.0E-4,1073741824 VAST=iscsi-izabel DATA=iscsi-izabel
9| # 20x the main PS part of the workflow
10| # The last four C-s are jobs T1, T2, T3 and TC from the workflow
11| VMSEQ N50500 N190000 N334000 C22.333 N4578744 C0.5 N6150209 N1488 C2145 C3573 C1886 C2
12| VMSEQ N50500 N190000 N334000 C22.333 N4578744 C0.5 N6150209 N1488 C2145 C3573 C1886 C2
16| VMSEQ N50500 N190000 N334000 C22.333 N4578744 C0.5 N6150209 N1488 C2145 C3573 C1886 C2
21| VMDEF VA=tinker,25,0,306176000 RC=1,5.0E-4,1073741824 VAST=iscsi-izabel DATA=iscsi-izabel
22| #This is the seq for job C
23| VMSEQ N50500 N190000 N334000 C22.333 N4578744 C0.5 N6150209 N1488 C10.6
Fig. 5 The description of the TCG workflow’s execution for the simulation
746 Cluster Computing (2019) 22:737–755
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
instantiated in the parallel section. Each VM requested by
the VMSEQ entries will use the definition provided in the
beginning of the parallel section. All VMs listed in the
section are requested from the simulated LPDS cloud right
before the workflow’s processing reaches the next
PSSTART entry in the description. This guarantees they
are requested and executed in parallel (note, despite
requesting the VMs simultaneously from the cloud, their
level of concurrency observed during the parallel section
will depend on the actual load of the simulated LPDS
cloud). The processing of the next parallel section, only
starts after the termination of all previously created VMs.
In the VMSEQ entries, a VM’s activities before termi-
nation. There are two kinds of activities listed: network and
compute. Network activities start with Nand then followed
by the number of bytes to be transferred between the DATA
store and the VM (this is the store defined by the VMDEF
entry at the beginning of the parallel section). Compute
activities, on the other hand, start with the letter Cand then
they list the number of seconds till the CPUs of the VM are
expected to be fully utilised by the activity. VM level
activities are executed in the simulated VM in a sequence
(i.e., one must complete before the next could start).
For example, line 12 of Fig. 5defines how job execu-
tions are performed in a VM. First, we prepare the VM to
run the tinker binaries by installing three software pack-
ages. This results in three transfers (49 KB, 186 KB and
326 KB files) and a task execution for 22 seconds. Next, we
fetch the tinker package (4.4 MBs) and decompress it (in a
half a second compute task). Then, we transfer the input
files with the 2500 conformers and the required runtime
parameters to use them (5.9 MBs and 1.5 KBs). After-
wards, we execute the T1, T2, T3 and TC jobs sequentially
taking 35, 60, 32 minutes and 2 seconds, respectively.
These values were gathered as the average execution times
for the jobs while the real life workflows were running in
the LPDS cloud. Finally, this 2 second activity concludes
the VMs operations, therefore it is terminated.
The PSSTART entry in line 6 and the virtual machine
executions, defined until line 16, represent a single exe-
cution of the parallel section of the TCG workflow.
Because of repetitions, we have omitted the several VMSEQ
entries from the parallel section, as well as several
PSSTART entries representing further parallel sections of
conformer analysis. On the other hand, the description
offered for the simulator did contain all the 14 additional
PSSTART entries which were omitted here for readability
To conclude, the description in Fig. 5provides details
for over 1800 network and computing activities to be done
for a single execution of the TCG workflow. If we consider
only those activities that are shown in the TCG workflow,
we still have over 1200 computing activities remaining.
These activities result in the creation and then destruction
of 302 virtual machines in the simulated cloud. When
calculating the error functions, we would need expected
execution details for all these activities or VMs. The rest of
the article will assume that the workflow enactor provides
details about the computing activities directly relevant for
the TCG workflow only (i.e., the jobs of G/Tx/C). Thus our
Nvalue was 1202. The partial workflow execution error
functions could be evaluated for every job done in the
simulated TCG. This, however, is barely offering any more
insight than having an error evaluation at the end of each
parallel section (i.e., when all VMs in the particular parallel
section are complete). As a result, in the rest of the article,
when we report kvalues, they are going to represent the
amount of parallel sections complete and not how many
actual activities were done so far. To transform between
activity count and the reported kvalues one can apply the
following formula:
kreal :¼
k\15 1 þ80k
k¼15 1 þ80kþ1;
where 15 is the number of parallel sections, and 80 is the
number of TCG activities per parallel section. Finally, the
kreal is the value used in the actual execution time error
formulas from Eqs. 13.
Although, the above description was presented with our
TCG workflow, the tags and their attributes of the
description were defined with more generic situations in
mind. In general, our description could be applied to
workflows and applications that have synchronisation
barriers at the end of their parallel sections.
5.2.2 Simulating the background load
In order to simulate how the workflow instances of TCG
would behave under various workload conditions, we
needed a comprehensive workload database. We have
evaluated previously published datasets: we looked for
workload traces that were collected from scientific com-
puting environments (as that is more likely to resemble the
workloads behind TCG). We only considered those traces
that have been collected over the timespan of more than 6
months (i.e., the length of our experiment with TCG on the
LPDS cloud—as detailed in Sect. 5.1). We further filtered
the candidate traces to only contain those which would not
cause significant (i.e., months) overload or idle periods in
the simulated LPDS cloud. This essentially left 4 traces
(SharcNet, Grid5000, NorduGrid and AuverGrid) from the
Grid Workloads Archive (GWA [9]), we will refer to the
summary of these traces as TGWA T. Note, it is irrelevant
Cluster Computing (2019) 22:737–755 747
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
that the traces were recorded on grids—for our purposes
the user behaviour, i.e., the variety of tasks, their arrival
rate and duration are important. Fortunately, these char-
acteristics are all independent from the actual type of
Trace fragments were created using GWA as follows:
we started a fragment from every job entry in the trace.
Then we identified the end of the fragment as follows. Each
fragment was expected to last at least for the duration of
the actual workflow instance. Unfortunately, the simulated
LPDS cloud could cause distortions to the job execution
durations (e.g., because of the different computing nodes or
because of the temporary under/over-provisioning situa-
tions compared to the trace’s original collection infras-
tructure), making it hard to determine the exact length of a
fragment without simulation. Thus, we have created frag-
ments that included all jobs within 3 times the expected
runtime of the workflow. As a result each of our fragments
was within the following time range: ½t;tþ3P0\i\N
rexðjiÞ. All these fragments were loaded in the trace
archive of Fig. 4(see its middle section titled ‘‘simulated
data generation’’). All together, our database have con-
tained more than 2 million trace fragments.
With the trace fragments in place, we had every input
data ready to evaluate the r0
ob values under various work-
load situations (i.e., represented by the fragments). Thus,
we have set out to create a large scale parametric study. For
this study, we have accumulated as many virtual machines
from various cloud infrastructures as many we could
afford. In total, we have had 3 cloud infrastructures
(SZTAKI cloud,
LPDS cloud, Amazon EC2
) involved
which hosted 192 single core virtual machines with 4 GBs
of memory and 5 GBs of hard disk storage (and the closest
equivalent on EC2). We offered the trace archive to all of
them as a network share. Each VM hosted DISSECT-CF
v0.9.6 and was acting as a slave node for our parametric
study. The master node (not shown in the figure), then
instructed the slaves to process one trace at a time as
1. Load a trace fragment as per the request of master.
2. Load the description of LPDS cloud.
3. Load the description of TCG workflow execution (i.e.,
the one shown in Fig. 5).
4. Start to submit the jobs from the loaded fragment to the
simulated LPDS cloud (for each submitted job, our
simulation asks a VM from the cloud which will last
until the job completes).
5. Wait until the 50th job—this step ensures the simu-
lated infrastructure is not under a transient load.
6. Start to submit the jobs and virtual machines of the
workflow execution specified in the previously loaded
description (the VMs here were also ran in the
simulated LPDS cloud).
7. For each task, record its observed execution time—
obðji;tÞ. Note, here trefers to the start time of the
simulated workflow.
8. After the completion of the last job and VM pair in the
workflow, terminate the simulation.
9. Send the collected job execution times to master.
The simulation of all trace fragments took less than 2 days.
The mean simulation execution time for a fragment run-
ning on our cloud’s model is tsim ðTCGÞ¼756 ms. We
have stored the details about each simulation in relation to
the particular trace fragment in our simulated job execution
log database (see Fig. 4).
To conclude our simulated data generation phase, we
populated our past and future error cache of Fig. 4. Later
our algorithm used this to represent past workflow beha-
viour. We calculated the past and future error values with
the help of all r0
ob we collected during the simulation phase.
The error values were cached from all 3 error functions we
defined in Eqs. 13as well as from their future error
counterparts from Eq. 4. This cached database allowed us
to evaluate the algorithm’s assumptions and behaviour in
the simulated environment as we discuss it in the next
5.3 Evaluation
In this section, we evaluate our algorithm using the col-
lected data about the simulated and real life TCG workflow
instance behaviour. We focused on three areas: (i) analyse
our assumption on the relation of past and future errors, (ii)
provide a performance evaluation of the algorithm, and (iii)
analyse how the various input variables to the algorithm
influence its accuracy. These are shown as the last two
phases (algorithm behaviour data collection and analysis)
in Fig. 4.
5.3.1 Relation between past and future errors
To investigate our assumption on the relation of past and
future error values, we have analysed the collected values
in the error cache. In Fig. 6, we exemplify how the simu-
lated past and future error values (using the ESQD function)
vary within a subset of the past/future error cache (which
we collected in the previous phase of our evaluation). Here,
The source code of these steps are published as part of the following
748 Cluster Computing (2019) 22:737–755
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
each dot represents a single simulation run, while the error
values were calculated after the twelfth parallel section—
k¼12, see Fig. 5. To reduce the clutter in the figure, we
present the simulation results using only a subset of the
trace fragments that we have identified in the Nordugrid
GWA trace (TGWA
nordu T). The items in the subset were
selected so every 50th Nordugrid related trace fragment is
shown in the chart: fti;tiþ1g2TGWA
nordu !ftj;tjþ50g2T. Out
of the selected fragments, only those are shown in the
figure which resulted in relatively low error values. This
allows us to better observe the relationship between past
and future errors when both error values are low. For
example, in case of Fig. 6, we have used the low error
value limit of 107. Note, the range of this error function for
all the simulated trace fragments was:
Based on the error cache, in this section, we investigated
two assumptions that are both important for the algorithm’s
success: (i) low past error values likely pair up with low
future error values; and (ii) show that as we approach the
end of the workflow, decreasing past error values would
more likely pair up with decreasing future error values (i.e.,
as past error values converge towards the final worklfow
execution error, future error values also approach a
stable hypothetical final value). As a first step, we limited
our analysis to fragments with past/future error values
below a chosen low error values—threshold:
filt :¼ft2T:EðW;t;kÞ\s_FðW;t;kÞ\s7Þ
The choice for scould ensure that this filtered set contains
the trace fragments most likely to be found by Algorithm 1
lines 1–14. For the error function ESQD , we have identified
the low error value limit as: s:¼2106.
For our first assumption, we evaluated the likelihood
that consecutive fragments with small past error values
E(W,t,k) lead to small F(W,t,k) values. We consider two
trace fragments (ta;tb2T:ðta\tbÞ) consecutive when
there are no other trace fragments with starting timestamps
in between the starting timestamps of the consecutive ones:
6 9tc2T:ðtc[ta^tc\tbÞ. First, we prepared the subsets
of Texp
filt , that hold more than 80 consecutive timestamps of
the trace. The number of consecutive timestamps is cal-
culated as T=tsim while assuming Tto be a minute to
minimise the impact of the simulation on the complete
prediction operation and its users (eg., a workflow enactor)
from the autonomous control loop. Next, we observed that
in these subsets the likelihood of having both minimal
future and past error values was 65–86%. Finally, we also
observed that the selection of a lower svalue could notably
decrease the simultaneous presence of below-threshold
error values (suggesting that a too precise match for the
past/future error leads to over fitting).
For our second assumption, we evaluated the error cache
and we observed that the higher kis the more potential the
prediction has. I.e., with increasing kthe error values tend
towards the low error values, while simultaneously their
deviation also decrease. Videos of this behaviour can be
seen in footnotes for the Sharcnet
and AuverGrid
The recordings show how the past and future error func-
tions of ESQD converge towards the optimal values as a
result of the increasing number of completed parallel sec-
tions of TCG—k.
Fig. 6 Nordugrid trace as the
background load behind TCG,
exemplified with the use of error
function ESQD
Cluster Computing (2019) 22:737–755 749
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
To conclude, we have shown that our assumption on
using past error values to indicate the tendency of future
ones is well supported by our simulated error cache. In
other words, if a trace shows similarity (in terms of an error
function) to past workloads then the same trace can be used
to estimate future workloads.
5.3.2 Behaviour analysis of the algorithm
In the final phases of our evaluation, we first executed an
extensive parametric study on the most important inputs of
the algorithm. Table 1shows all the parameters evaluated.
In our parametric study, we have ran all input parameter
combinations on resources utilizing the private OpenStack
cloud of Liverpool John Moores University. We have used
5 VM instances with equivalent configurations as follows:
(i) 20 GB disks, containing the execution logs and the error
cache, (ii) 16 GB of memory to allow efficient handling of
the cache, and (iii) an Intel Core i7 4790 processor. For
each parameter combination, these VMs ran 500 approxi-
mations using our previously collected simulated job exe-
cution log database (this is depicted in Fig. 4as the
‘‘ Algorithm behaviour data collection’ phase).
For each approximation run, we have randomly selected
an approximation target: a golden fragment, which we
denote as tg2TG, where TGTGWA denotes the set of all
randomly chosen golden fragments. We have retrieved the
execution logs related to the golden fragment (i.e., the
robðj;tgÞ). Then, we applied our algorithm to identify an
approximate GWA trace fragment for this particular
(golden) TCG workflow instance, given that the instance
has progressed to job jk(i.e., the algorithm did not receive
the complete execution log, just the observed execution
time values that occurred before job jk:
robðji;tgÞ:1i\k). The approximation resulted in a
ttarget 2TGWA trace fragment start time from one of the
GWA traces. Finally, we have analysed the relation of
ttarget and tg, as well as the execution time—d—of the
approximation algorithm under the particular input
parameter conditions.
To analyse the capabilities of the algorithm in terms of
workload approximation, we first defined the metrics to
quantify the accuracy. An ideal solution would be finding
exactly the golden fragment however, this is neither fea-
sible nor necessary (as real life traces for the background
workload would not be comparable). The aim is to see the
degree of similarity between the golden fragment and its
approximation (identified by ttarget ). We have defined two
fundamental metrics for the evaluation of our algorithm:
Execution time level metric: First, we used the EMAPE
function from Eq. 2. We wanted EMAPE to show the
average error (in percentage) between tgand ttarget , thus
during this evaluation, we assigned rex ðjiÞ robðji;tgÞ
and robðji;tÞ rob ðji;ttargetÞ. This allowed us to see the
execution time differences the algorithm’s predicted
ttarget trace fragment causes in contrast to the golden’s.
We will denote this special use of the error function as
Error level metric: We also compared how do the golden
and the approximated trace fragments relate to the real
life execution expectations of TCG—these are the rex ðjiÞ
values we have identified in the LPDS cloud according
to Sect. 5.1. For this metric, we again use the mean
absolute percentage error method, but this time to see
how the error for tgis approximated by the error of ttarget
at every kvalue.
Thus our second, error level, metrics are defined as:
100 EðW;tg;iÞ
for past errors, and
100 FðW;tg;iÞ
for future errors. Note: when evaluating our above metrics,
we used the same error function as the algorithm—e.g.,
EðW;tg;iÞcould be in both cases ESQD ðW;tg;iÞ.
As we have done 500 approximations for each input
parameter combination, we have calculated their overall
average and median values (for the following metrics:
MAPE;MAPEE;MAPEF;d) to represent the accu-
racy and performance of the approximation with a particular
input set. To put these aggregated values into context, we
have evaluated them for a random trace selection approach as
well (this served as a baseline for comparing the effective-
ness of our technique). Here, for each member of the golden
set (TG), an arbitrary trace fragment from the whole trace
Table 1 Algorithm configurations investigated in our parametric
Input Used parameters
P10, 20, 50, 100, 200, 500,
1000, 2000, 5000
S100, 200, 500, 1000, 2000
I2, 4, 8, 16, 32
ðmax Tred min TredÞ=S2, 5, 10, 20, 50, 100
* As there was no significant improvement when using values of
S=2000, their corresponding results are not included in the figures.
These experiments are only discussed in the penultimate paragraph of
Sect. 5.3.2
750 Cluster Computing (2019) 22:737–755
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
database was selected as the member’s approximation:
target 2TGWA. The calculated accuracy metrics of this
approximation approach are shown in Table 2.
Figures 7and 8show the behaviour induced by the
changes in the two most impacting parameters of the
algorithm. In total, each figure represents results of over
500 thousand approximations. A plotted point averages the
outcomes of all possible parameter configurations except
the one that is fixed for the plot. Note, the apparent impact
of a parameter is often reduced by averaging (e.g., the best
parameter configurations for S¼1000 lead to E
MAPE val-
ues in the range of 28%, while the average shown in the
figure is over 34%). Nevertheless, the major trends are still
visible. Also, in all cases the approximations of our algo-
rithm yield significantly better results than random selec-
tion. Not surprisingly, using the E
MAPE metric, we have the
best results when the algorithm also uses our MAPE based
workflow execution time error function (EMAPE ), while the
time adjusted function (ETAltSQD ) performs poorly. The
improvements over random selection range between 10 and
20% for the median of our execution time level metric.
Based on the results, one can also conclude that the
secondary search window size (shown in Fig. 8) is a more
important factor (albeit not directly configurable from the
algorithm’s inputs). Thus, this must be an exposed user
configurable parameter in the future.
In general, the dduration of the approximation is neg-
ligible (in the range of 31–2114 ms, with a median of less
than 200 ms) compared to our assumed 1 minute maximum
time for prediction. This leaves enough time for the sim-
ulation needs of the algorithm and therefore ensures timely
predictions. We have also observed that increasing any of
the parameters obviously introduces more calculations,
however they do not increase the accuracy in a uniform
way. Increasing parameter I, the number of iterations
increases accuracy in a minimal way. After investigation,
we have concluded that the algorithm’s exit condition (see
line 18 of Algorithm 1) is often fulfilled by its sub-condi-
tion on P—which was a set constant in all algorithm
executions. Increasing the number (P) of evaluations for
/ðxÞoften leads to slightly decreasing accuracy because a
higher number of evaluations lead to local minimums.
The effects on accuracy are not conclusive, because the
nature of cross-parameter averages shown in Figs. 7and 8.
Table 2 Baseline results of
random trace selection Random selection MAPEEðW;tgÞMAPEFðW;tgÞE
Average (8tg2TG) 157.874 166.166 72.243 85.893
Median (8tg2TG) 49.180 67.825 45.174 47.002
(a) ESQD
(c) ET AltSQD
Fig. 7 The influence of the input Sof the algorithm to its performance and accuracy
(a) ESQD
(c) ETAltSQD
Fig. 8 Performance and accuracy impacts of the secondary search window size—ðmax Tred min Tred Þ=S
Cluster Computing (2019) 22:737–755 751
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
To better understand the effects of particular parameters,
we have filtered the top 5% most accurate parameter con-
figurations. Figures 9,10,11 and 12 show the likelihood
for a particular parameter value to be represented in the
filtered list. The figures also analyse how the particular
error functions benefit from the other input parameters of
the algorithm. Note, the average dduration of the algo-
rithm’s evaluation with the parameter combinations from
the top 5% was 2333 ms.
These figures suggest, that selecting an Ivalue between
8 and 32 could bring slight benefits in accuracy. With
regards to S, we have seen that values over 500 are bene-
ficial. Our parametric study evaluated for S¼2000 as well,
but this increased Svalue did not bring significant enough
improvements to compensate for the additional time spent
on evaluating the algorithm (i.e., the dincreased threefold
for an average accuracy increase of less than 1%). Next, we
have analysed the effect of the number of /ðxÞevaluations
(P). As with our previous experiments, Phave had an
inconclusive effect on accuracy (which highlights that the
algorithm would greatly benefit from techniques that avoid
the traps of local minimums). In general, lower P\500
values proved more accurate, especially values of 20–50
were strongly represented in the top 5%. Finally, we have
concluded that the secondary search window size has the
biggest impact on accuracy, which is to be set between 50
and 100 times the size of the primary search window S.We
recommend the following values:
(a) ESQD (b) EMAPE (c) ETAltSQD
Fig. 9 The likelihood that the algorithm will produce predictions in the top 5% depending on input I
(a) ESQD (b) EMAP E (c) ETAltSQD
Fig. 10 The likelihood that the algorithm will produce predictions in the top 5% depending on input S
(a) ESQD (b) EMAPE (c) ETAltSQD
Fig. 11 The likelihood that the algorithm will produce predictions in the top 5% depending on input P
(a) ESQD (b) EMAPE (c) ETAltSQD
Fig. 12 The likelihood that the algorithm will produce predictions in the top 5% depending on secondary search window size—
ðmax Tred min TredÞ=S
752 Cluster Computing (2019) 22:737–755
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
max Tred min Tred
Based on the recommended input values, we have
analysed how the past and future errors for each 500 tg2
TGcorrelated with the error values acquired for the cor-
responding ttarget predicted fragments. To evaluate the level
of correlation, we used the statistical indicator called
coefficient of determination: R2. The results are shown in
Table 3. The algorithm finds strongly correlating approxi-
mations for past errors, while future errors show weaker
correlation patterns. The weakest correlation was observed
for the EMAPE error function, which shows that the function
is too focused on the particular rob values. On the other
hand, the error function of ETAltSQD leads to the best future
error predictions, we assume this performance is likely
caused by the function’s stronger reliance on job order (and
indirectly time). The best R2results we have obtained are
presented in Fig. 13. These were using slightly different
input parameter values (namely S¼500;P¼50) than we
recommended based on our statistical evaluation. Amongst
our future work, we plan to investigate techniques that
would allow the algorithm to auto tune its parameters to get
better correlating past and future error predictions for
particular workflows.
6 Conclusions
IaaS clouds hide the complexity of maintaining the phys-
ical infrastructure, but there are many application areas that
need additional knowledge of the underlying cloud systems
in order to support their activities. Workflow enactment is
one of these areas that could benefit from detecting beha-
vioural changes in the underlying system. Therefore, this
article aimed at studying performance issues related to the
background load and proposes a methodology for its
We followed the concept of a load prediction method
based on the combination of historic traces to improve
execution quality. We proposed an algorithm for realising
the load prediction at runtime so that performance con-
straints are observed. We proposed these predictions to
select more suitable execution environments for scientific
workflows, hence we evaluated this approach using a bio-
chemical application with a state of the art simulator using
historic traces from a widely used archive. We have shown
that our assumption of using past error values to indicate
the tendency of future ones is partially supported by our
simulations. Thus, if a trace shows similarity to past
workloads then the continuation of the same trace has a
potential to be used as an estimate for future workloads.
In our future work, we aim at analysing further areas to
employ our algorithm (e.g., to support cloud orchestration,
brokering). We also aim at refining our algorithm through
multiple approaches: (i) revise the method to select frag-
ments with likely better future matches; (ii) analyse the
impact of other error functions; (iii) explore, if certain sim-
ulators are better suited for modelling particular clouds and
offering better support to our prediction algorithms; (iv)
investigate if discrepancies in the cloud based execution of
Table 3 Coefficient of determination (R2) for the recommended input
combination with the various error functions
Error function Coefficient of determination
#R2ðEðtgÞ;EðttargetÞÞ R2ðFðtgÞ;Fðttarget ÞÞ
ESQD 0.824 0.176
EMAPE 0.656 0.015
ETAltSQD 0.696 0.267
(a) Past errors (b) Future errors
Fig. 13 Correllation analysis of
the ESQD values for the
corresponding items from the
golden and predicted sets
Cluster Computing (2019) 22:737–755 753
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
other scientific workflows’ could also be used to improve
prediction accuracy. Finally, we also plan to enable other
(non-workflow-like) long running applications (e.g., com-
mercial web traffic) to offer their inputs to our prediction
technique. This direction would allow us to consider
broadening the scope of our predictions from private clouds
(that could be easily modelled in feasible time with current
simulators) to some commercial clouds as well.
Acknowledgements The research leading to these results was sup-
ported by the Hungarian Government and the European Regional
Development Fund under the grant number GINOP-2.3.2-15-2016-
00037 (‘‘Internet of Living Things’’), and it was supported by the
Janos Bolyai Research Scholarship of the Hungarian Academy of
Open Access This article is distributed under the terms of the Creative
Commons Attribution 4.0 International License (http://creative, which permits unrestricted use, distri-
bution, and reproduction in any medium, provided you give appropriate
credit to the original author(s) and the source, provide a link to the
Creative Commons license, and indicate if changes were made.
1. Bacso
´, G., Visegra
´di, A
´., Kertesz, A., Ne
´meth, Z.: On efficiency
of multi-job grid allocation based on statistical trace data. J. Grid
Comput. 12(1), 169–186 (2013).
2. Bhardwaj, S., Jain, L., Jain, S.: Cloud computing: a study of
infrastructure as a service (IaaS). Int. J. Eng. Inf. Technol. 2(1),
60–63 (2010)
3. Brito, J.B.D.S., Arau
´jo, A.P.F.D.: HCEmmodel and a compara-
tive workload analysis of hadoop cluster. Int. J. Big Data Intell.
4(1), 47–60 (2017)
4. Caballer, M., Blanquer, I., Molto, G., de Alfonso, C.: Dynamic
management of virtual infrastructures. J. Grid Comput. 13, 1–18
5. Calheiros, R.N., Masoumi, E., Ranjan, R., Buyya, R.: Workload
prediction using ARIMA model and its impact on cloud appli-
cations’ QoS. IEEE Trans. Cloud Comput. 3(4), 449–458 (2015)
6. Caron, E., Desprez, F., Muresan, A.: Forecasting for grid and
cloud computing on-demand resources based on pattern match-
ing. In: Proceedings of the 2010 IEEE Second International
Conference on Cloud Computing Technology and Science
(CloudCom), pp. 456–463 (2010).
7. Fard, H.M., Ristov, S., Prodan, R.: Handling the uncertainty in
resource performance for executing workflow applications in
clouds. In: Proceedings of the 9th International Conference on
Utility and Cloud Computing, UCC ’16, pp. 89–98. ACM, New
York, NY, USA (2016).
8. Horn, P.: Autonomic Computing: IBM’s Perspective on the State
of Information Technology. IBM TJ Watson Labs, New York
9. Iosup, A., Li, H., Jan, M., Anoep, S., Dumitrescu, C., Wolters, L.,
Epema, D.H.J.: The grid workloads archive. Future Gener.
Comput. Syst. 24(7), 672–686 (2008).
10. Kecskemeti, G.: DISSECT-CF: a simulator to foster energy-
aware scheduling in infrastructure clouds. Simul. Model. Pract.
Theory 58P2, 188–218 (2015).
11. Kertesz, A., Otvos, F., Kacsuk, P.: A case study for biochemical
application porting in European grids and clouds. Special Issue
on Distributed, Parallel, and GPU-Accelerated Approaches to
Computational Biology, Concurrency and Computation: Practice
and Experience (2013)
12. Khan, A., Yan, X., Tao, S., Anerousis, N.: Workload character-
ization and prediction in the cloud: a multiple time series
approach. In: Proceedings of the 2012 IEEE Network Operations
and Management Symposium (NOMS), pp. 1287–1294. IEEE
13. Kritikos, K., Pernici, B., Plebani, P., Cappiello, C., Comuzzi, M.,
Benrernou, S., Brandic, I., Kerte
´sz, A., Parkin, M., Carro, M.: A
survey on service quality description. ACM Comput. Surv. 46(1),
1:1–1:58 (2013).
14. Li, A., Zong, X., Kandula, S., Yang, X., Zhang, M.: CloudPro-
phet: towards application performance prediction in cloud. In:
ACM SIGCOMM Computer Communication Review, vol. 41,
pp. 426–427. ACM, New York (2011)
15. Magalhaes, D., Calheiros, R.N., Buyya, R., Gomes, D.G.:
Workload modeling for resource usage analysis and simulation in
cloud computing. Comput. Electr. Eng. 47(1), 69–81 (2015).
16. Mao, L., Qi, D., Lin, W., Zhu, C.: A self-adaptive prediction
algorithm for cloud workloads. Int. J. Grid High Perform. Com-
put. (IJGHPC) 7(2), 65–76 (2015)
17. Matha
´, R., Ristov, S., Prodan, R.: Simulation of a workflow
execution as a real cloud by adding noise. Simul. Model. Pract.
Theory 79, 37–53 (2017).
18. Maurer, M., Brandic, I., Sakellariou, R.: Adaptive resource con-
figuration for cloud infrastructure management. Future Gener.
Comput. Syst. 29(2), 472–487 (2013)
19. Pietri, I., Juve, G., Deelman, E., Sakellariou, R.: A performance
model to estimate execution time of scientific workflows on the
cloud. In: Proceedings of the 9th Workshop on Workflows in
Support of Large-Scale Science, WORKS ’14, pp. 11–19. IEEE
Press, Piscataway, NJ, USA (2014).
20. Szabo, C., Sheng, Q.Z., Kroeger, T., Zhang, Y., Yu, J.: Science in
the cloud: allocation and execution of data-intensive scientific
workflows. J. Grid Comput. 12(2), 245–264 (2014)
21. Thaman, J., Singh, M.: Extending dynamic scheduling policies in
workflowsim by using variance based approach. Int. J. Grid High
Perform. Comput. (IJGHPC) 8(2), 76–93 (2016)
Gabor Kecskemeti (M.Sc.,
University of Miskolc, 2004;
Ph.D., University of Westmin-
ster, 2011) is a Lecturer within
the Department of Computer
Science at Liverpool John
Moores University. Between
2006–2016, he worked as a
Research Fellow at the Labora-
tory of Parallel and Distributed
Systems (LPDS) of MTA
SZTAKI and in parallel he also
worked as an Assistant Lecturer
at the University of Miskolc. In,
2013, he was a Postdoctoral
Researcher at the University of Innsbruck, Austria. He has got
involved in several successful Grid and SOA research projects (e.g.,
ePerSpace, EDGeS, S-Cube) funded by the EU. Recently, in 2015 and
754 Cluster Computing (2019) 22:737–755
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
early 2016, he was steering committee member and work package
leader in the ENTICE project (focusing on cloud computing) which
was financed by the EU’s Horizon2020 framework programme. He
coordinated the Cloud research sub-group of LPDS in 2011–2012. He
was member/chair of the program committees of over 30 European
conferences/workshops. He was guest editor for the Journal of Grid
Computing for a special issue on Cloud federations and he also edited
the book titled ‘‘Developing Interoperable and Federated Cloud
Architecture’’. He has published over 70 scientific papers (incl. 16
journal articles and 4 edited volumes) about cloud and grid comput-
ing, particularly from the fields of simulating distributed systems and
virtual appliance delivery in IaaS clouds.
Zsolt Nemeth received his M.Sc.
from Technical University of
Budapest in 1994 and his Ph.D.
from Budapest University of
Technology and Economics in
2001. He is a Senior Researcher
at the Laboratory of Parallel and
Distributed Systems, MTA
SZTAKI Computer and
Automation Research Institute
of the Hungarian Academy of
Sciences. His early research
work focused on parallel logic
programming, later his research
interest has covered various
aspects of parallel and distributed computing, formal modeling, grid
computing, unconventional programming paradigms. His work on
semantic modeling of grid computing met a reception. He has been a
visiting researcher at Emory University; USA; IRISA-INRIA, France
and FORTH, Greece as an ERCIM Fellow.
Attila Kertesz (M.Sc., University
of Szeged, 2004; Ph.D.,
University of Szeged, 2011) is
an Assistant Professor at the
Software Engineering Depart-
ment of the University of
Szeged, Hungary. His research
work covers interoperability
aspects of Cloud federations and
IoT. He is a research group
leader in the GINOP IoLT
European Regional Develop-
ment project. He was a Man-
agement Committee member of
the ICT COST Action IC1304
project, and participated in several successful European projects
including ENTICE EU H2020, COST IC0805, SHIWA, S-Cube EU
FP7 and the CoreGRID EU FP6 Network of Excellence projects. He
was member of numerous program committees for European con-
ferences and workshops, and has published over 80 scientific papers
having more than 500 independent citations.
Rajiv Ranjan is a Reader
(equivalent to non-distinguished
full professor in the North
American system) in Comput-
ing Science (Ranked #76 Times
Higher Education Subject
Ranking 2016/2017) at New-
castle University, United King-
dom. He is an internationally
established scientist with about
225 peer-reviewed (scientific)
publications and expertise in
cloud computing, big data, and
the Internet of Things. He has
secured more than $12 Million
AUD (£6 Million?GBP) in form of competitive research grants from
both public and private agencies. He is an innovator with strong and
sustained academic and industrial impact and a globally recognized
R&D leader with the proven track record. He serves on the editorial
boards of top quality international journals including IEEE Transac-
tions on Computers, IEEE Transactions on Cloud Computing, IEEE
Cloud Computing, and Future Generation Computer Systems.
Cluster Computing (2019) 22:737–755 755
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
Terms and Conditions
Springer Nature journal content, brought to you courtesy of Springer Nature Customer Service Center GmbH (“Springer Nature”).
Springer Nature supports a reasonable amount of sharing of research papers by authors, subscribers and authorised users (“Users”), for small-
scale personal, non-commercial use provided that all copyright, trade and service marks and other proprietary notices are maintained. By
accessing, sharing, receiving or otherwise using the Springer Nature journal content you agree to these terms of use (“Terms”). For these
purposes, Springer Nature considers academic use (by researchers and students) to be non-commercial.
These Terms are supplementary and will apply in addition to any applicable website terms and conditions, a relevant site licence or a personal
subscription. These Terms will prevail over any conflict or ambiguity with regards to the relevant terms, a site licence or a personal subscription
(to the extent of the conflict or ambiguity only). For Creative Commons-licensed articles, the terms of the Creative Commons license used will
We collect and use personal data to provide access to the Springer Nature journal content. We may also use these personal data internally within
ResearchGate and Springer Nature and as agreed share it, in an anonymised way, for purposes of tracking, analysis and reporting. We will not
otherwise disclose your personal data outside the ResearchGate or the Springer Nature group of companies unless we have your permission as
detailed in the Privacy Policy.
While Users may use the Springer Nature journal content for small scale, personal non-commercial use, it is important to note that Users may
use such content for the purpose of providing other users with access on a regular or large scale basis or as a means to circumvent access
use such content where to do so would be considered a criminal or statutory offence in any jurisdiction, or gives rise to civil liability, or is
otherwise unlawful;
falsely or misleadingly imply or suggest endorsement, approval , sponsorship, or association unless explicitly agreed to by Springer Nature in
use bots or other automated methods to access the content or redirect messages
override any security feature or exclusionary protocol; or
share the content in order to create substitute for Springer Nature products or services or a systematic database of Springer Nature journal
In line with the restriction against commercial use, Springer Nature does not permit the creation of a product or service that creates revenue,
royalties, rent or income from our content or its inclusion as part of a paid for service or for other commercial gain. Springer Nature journal
content cannot be used for inter-library loans and librarians may not upload Springer Nature journal content on a large scale into their, or any
other, institutional repository.
These terms of use are reviewed regularly and may be amended at any time. Springer Nature is not obligated to publish any information or
content on this website and may remove it or features or functionality at our sole discretion, at any time with or without notice. Springer Nature
may revoke this licence to you at any time and remove access to any copies of the Springer Nature journal content which have been saved.
To the fullest extent permitted by law, Springer Nature makes no warranties, representations or guarantees to Users, either express or implied
with respect to the Springer nature journal content and all parties disclaim and waive any implied warranties or warranties imposed by law,
including merchantability or fitness for any particular purpose.
Please note that these rights do not automatically extend to content, data or other material published by Springer Nature that may be licensed
from third parties.
If you would like to use or distribute our Springer Nature journal content to a wider audience or on a regular basis or in any other manner not
expressly permitted by these Terms, please contact Springer Nature at
... Also, a vast majority of algorithms are based on elastically acquiring VMs and many of them assumes that execution time of a task on specific VM is deterministic and can be known beforehand [7,30]. However, such estimates are very difficult to be found accurately prior to actual execution and may need a suitable prediction technique [17]. Nonetheless, most of the aforesaid algorithms have not considered the resource wastage as an optimizing parameter. ...
... If the above conditions are not satisfied, the first task in the sorted list is selected (line 16). If any of its parent tasks is found allocated in the current working row (lines [17][18][19][20], the current row is marked as full and the control sequence is moved to the beginning of the while loop. Otherwise, the task will be added to the current working row and its child task(s) are added to the child list (lines [21][22][23][24][25][26][27]. ...
Full-text available
Cloud Computing provides a promising platform for executing large scale workflow applications with enormous computational resources to offer on-demand services. Tasks in a workflow may need different type of computing resources such as storage, compute and memory type. However, inappropriate selection of these resources may lead to higher makespan and resource wastage. In this paper, we propose an effective two-phase algorithm for provisioning of cloud resources for workflow applications by using its structural features to minimize makespan and resource wastage. The proposed approach considers the nature of the tasks which may be compute intensive, memory intensive or storage intensive. We assume a realistic cloud model similar to Amazon EC2 that provides virtual machines for different types of workloads. Most importantly, the workflow model used in our approach is assumed to contain limited information about the task which is applicable for real situation. The performance of the proposed work is measured using five benchmark scientific workflows. The simulation results show that the proposed approach outperforms two existing algorithms for all these workflows.
... They introduced a novel concept named Prediction Accuracy Assurance (PAA) as a criterion to evaluate the precision of the prediction runtime provided by a specific prediction strategy. Kecskemeti et al. [53] proposed a technique for predicting generic background workload by means of simulations. The technique was capable to provide additional knowledge of the underlying private cloud systems in order to support activities like cloud orchestration or work flow enactment. ...
... There was no specific rule for determining the structure of NNs. Their network structure was achieved through experience and trial and error [53]. So, this issue reduced trust in the network. ...
Full-text available
Time sensitive virtual machines that run real-time control tasks are constrained by hard timing requirements. Optimal resource management and guarantee the hard timing requirements of virtual machines are critical goals. Basically, cloud resource usage predicting and resource reservation play a crucial role to achieve these two goals. So, we propose a predicting approach based on two-phase decomposition method and hybrid neural network to predict future resource usage. This paper uses a clustering method based on the AnYa algorithm in an on-line manner in order to obtain the number of fuzzy rules and the initial value of the premise and consequent parameters. Since cloud resource usage varies widely from time to time and server to server, extracting the best time series model for predicting cloud resource usage depend not only on time but on the cloud resource usage trend. For this, we present a recursive hybrid technique based on singular spectrum analysis and adaptively fast ensemble empirical mode decomposition to identify the hidden characteristics of the time series data. This method tries to extract seasonal and irregular components of the time series. According to the simulation results, it can be found that the proposed model can have significantly better performance than the three comparison models from one-step to six-step CPU usage predictions with the MAPE of 33.83% average performance promotion, MAE of 36.54% average performance promotion, RMSE of 36.70% average performance promotion.
... We refer to a request for service l that is submitted to edge cloud n as a "type-(l, n)" request. The average rate of type-(l, n) requests in frame f is denoted by λ f ln (unit: requests/slot), which is assumed to be predictable based on the request history [37], [38], [39]. The actual number of type-(l, n) requests in slot t is denoted by λ t ln , which is only known at the beginning of slot 1 t. ...
Full-text available
Mobile edge computing provides the opportunity for wireless users to exploit the power of cloud computing without a large communication delay. To serve data-intensive applications (e.g., video analytics, machine learning tasks) from the edge, we need, in addition to computation resources, storage resources for storing server code and data as well as network bandwidth for receiving user-provided data. Moreover, due to time-varying demands, the code and data placement needs to be adjusted over time, which raises concerns of system stability and operation cost. In this paper, we address these issues by proposing a two-time-scale framework that jointly optimizes service (code and data) placement and request scheduling, while considering storage, communication, computation, and budget constraints. First, by analyzing the hardness of various cases, we completely characterize the complexity of our problem. Next, we develop a polynomial-time service placement algorithm by formulating our problem as a set function optimization, which attains a constant-factor approximation under certain conditions. Furthermore, we develop a polynomial-time request scheduling algorithm by computing the maximum flow in a carefully constructed auxiliary graph, which satisfies hard resource constraints and is provably optimal in the special case where requests have homogeneous resource demands. Extensive synthetic and trace-driven simulations show that the proposed algorithms achieve 90% of the optimal performance.
... Simulation is an important aspect for architecture design because it integrates hardware more effectively for particular applications, models data and control flow, enables capacity planning, reviews energy requirements, and evaluates performance metrics. Popular simulation programs have been reviewed in previous studies (Kecskemeti et al., 2019). Biofabrication technology is still in its infancy and biological products, bioprinting equipment, robotics, and sensors need to mature before cloud manufacturing can be considered. ...
Advanced manufacturing and 3D printing are transformative technologies currently undergoing rapid adoption in healthcare, a traditionally non-manufacturing sector. Recent development in this field, largely enabled by merging different disciplines, has led to important clinical applications from anatomical models to regenerative bioscaffolding and devices. Although much research to-date has focussed on materials, designs, processes, and products, little attention has been given to the design and requirements of facilities for enabling clinically relevant biofabrication solutions. These facilities are critical to overcoming the major hurdles to clinical translation, including solving important issues such as reproducibility, quality control, regulations, and commercialization. To improve process uniformity and ensure consistent development and production, large-scale manufacturing of engineered tissues and organs will require standardized facilities, equipment, qualification processes, automation, and information systems. This review presents current and forward-thinking guidelines to help design biofabrication laboratories engaged in engineering model and tissue constructs for therapeutic and non-therapeutic applications.
... Kecskemeti et al. [14] proposed a technique for predicting background workload in the cloud environment. They simulated a workflow according to the plan specified runtime properties, like job start time, completion time, and times for creating virtual machines. ...
Full-text available
Abstract Hadoop MapReduce is a framework to process vast amounts of data in the cluster of machines in a reliable and fault-tolerant manner. Since being aware of the runtime of a job is crucial to subsequent decisions of this platform and being better management, in this paper we propose a new method to estimate the runtime of a job. For this purpose, after analysis the anatomy of processing a job in Hadoop MapReduce precisely, we consider two cases: when a job runs for the first time or a job has run previously. In the first case, by considering essential and efficient parameters that higher impact on runtime we formulate each phase of the Hadoop execution pipeline and state them by mathematical expressions to calculate runtime of a job. In the second case, by referring to the profile or history of a job in the database and use a weighting system the runtime is estimated. The results show the average error rate is less than 12% in the estimation of runtime for the first run and less than 8.5% when the profile or history of the job has existed.
... Kecskemeti, Gabor, et al. [14] proposed a technique for predicting background workload in the cloud environment. They simulated a workflow according to the plan specified runtime properties, like job start time, completion time, and times for creating virtual machines. ...
Full-text available
Hadoop MapReduce is a framework to process vast amounts of data in the cluster of machines in a reliable and fault-tolerant manner. To better management of this framework, estimating the runtime of a job is an important but challenging problem. In this paper, after analysis the anatomy of processing a job in Hadoop MapReduce precisely, we model each stage by relevant essential and efficient features that higher impact on runtime; then we propose two new methods to estimate the runtime, where the history of the previous run of a job is available, or a job performs for the first time. The results show less than 12% error in the estimation of runtime for the first run and less than 8.5% error when the profile or history of the job has existed.
... Kecskemeti, Gabor, et al. [14] proposed a technique for predicting background workload in the cloud environment. They simulated a workflow according to the plan specified runtime properties, like job start time, completion time, and times for creating virtual machines. ...
Full-text available
Hadoop MapReduce is a framework to process vast amounts of data in the cluster of machines in a reliable and fault-tolerant manner. Since being aware of the runtime of a job is crucial to subsequent decisions of this platform and being better management, in this paper we propose a new method to estimate the runtime of a job. For this purpose, after analysis the anatomy of processing a job in Hadoop MapReduce precisely, we consider two cases: when a job runs for the first time or a job has run previously. In the first case, by considering essential and efficient parameters that higher impact on runtime we formulate each phase of the Hadoop execution pipeline and state them by mathematical expressions to calculate runtime of a job. In the second case, by referring to the profile or history of a job in the database and use a weighting system the runtime is estimated. The results show the average error rate is less than 12% in the estimation of runtime for the first run and less than 8.5% when the profile or history of the job has existed.
... This section puts forward a comprehensive study of the predictive VM placement approaches [40][41][42][43][44][45][46][47][48][49][50][51][52][53][54][55][56][57], designed and proposed for various cloud computing environments. All of these schemes have applied some prediction techniques to enhance the VM placement process using the historical data and all of them are published since 2010. ...
Full-text available
Efficient VM management is very crucial for energy saving, increasing profit, and preventing SLA violations. VM placement schemes can be classified into reactive and proactive/predictive schemes which try to improve the VM placement results, by forecasting future workloads or resource demands using various prediction techniques. This paper puts forward an extensive survey of the proactive VM placement approaches and categorizes them according to their applied forecasting methods. It describes how each scheme has applied the prediction algorithms to conduct more effective and low overhead VM placement. Moreover, in each category, factors such as evaluation parameters, simulation software, workload data, power management method, and prediction factors are compared to illuminate more details about the investigated VM placement approaches. At last, the concluding issues and open future studies trends and area are highlighted.
This paper explores resource allocation strategy in the Baidu Over The Edge system to enable mobile edge computing (MEC) datacenters to effectively support cloud‐native services downstream to the network edge. There are many challenges to this issue. First, MEC datacenters are resource‐constrained to fully meet resource demands. Second, previous works regard the resource requirements of each service as an indivisible unit, resulting in idle MEC resources, even if the resources can meet the demands of some microservices decoupled by the service. Third, they are confined to optimize the allocation for a single slot, failing to adapt to the dynamic demands. To improve resource utilization, we propose performability‐aware resource allocation (PARA), a PARA on the edges for cloud‐native services. It takes microservices as the unit of resource allocation and allows services to perform with degraded services when only part of microservices' demands are met. It also considers dependency among microservices, dynamic resource requirements, and resource supply characteristics of MEC and cloud. Performability is a unified performance‐reliability measure for evaluating such degradable systems. To maximize the long‐term overall performability, we model the resource optimization problem and then develop an online greedy heuristic algorithm. The algorithm predicts services' resource demands and then adapts the online allocation. The experimental results show that PARA reduces the reallocation overhead by 47.7%–53.6%, and improves the long‐term overall performability by 23.14%–43.25% of existing state‐of‐the‐art works.
Cloud datacentres are turning out to be massive energy consumers and environment polluters, which necessitate the need for promoting sustainable computing approaches for achieving environment-friendly datacentre execution. Direct causes of excess energy consumption of the datacentre include running servers at low level of workloads and over-provisioning of server resources to the arriving workloads during execution. To this end, predicting the future workload demands and their respective behaviours at the datacentres are being the focus of recent research in the context of sustainable datacentres. But prediction analytics of Cloud workloads suffer various limitations imposed by the dynamic and unclear characteristics of Cloud workloads. This paper proposes a novel forecasting model named K-RVLBPNN (K-means based Rand Variable Learning Rate Back Propagation Neural Network) for predicting the future workload arrival trend, by exploiting the latency sensitivity characteristics of Cloud workloads, based on a combination of improved K-means clustering algorithm and BPNN (Back Propagation Neural Network) algorithm. Experiments conducted on real-world Cloud datasets exhibit that the proposed model exhibits better prediction accuracy, outperforming traditional Hidden Markov Model, Naive Bayes Classifier and our earlier RVLBPNN model respectively.
Full-text available
It is difficult to analyze the workload in complex cloud computing environments with a single prediction algorithm as each algorithm has its own shortcomings. A self-adaptive prediction algorithm combining the advantages of linear regression (LR) and a BP neural network to predict workloads in clouds is proposed in this paper. The main idea of the self-adaptive prediction algorithm is to choose the better prediction method of the future workload. Some experiments of prediction algorithms are conducted with workloads on the public cloud servers. The experimental results show that the proposed algorithm has a relatively high accuracy on the workload predictions compared with the BP neural network and LR. Furthermore, in order to use the proposed algorithm in a cloud data center, a dynamic scheduling architecture of cloud resources is designed to improve resource utilization and reduce energy consumption.
Full-text available
Infrastructure as a service (IaaS) systems offer on demand virtual infrastructures so reliably and flexibly that users expect a high service level. Therefore, even with regards to internal IaaS behaviour, production clouds only adopt novel ideas that are proven not to hinder established service levels. To analyse their expected behaviour, new ideas are often evaluated with simulators in production IaaS system-like scenarios. For instance, new research could enable collaboration amongst several layers of schedulers or could consider new optimisation objectives such as energy consumption. Unfortunately, current cloud simulators are hard to employ and they often have performance issues when several layers of schedulers interact in them. To target these issues, a new IaaS simulation framework (called DISSECT-CF) was designed. The new simulator's foundation has the following goals: easy extensibility, support energy evaluation of IaaSs and to enable fast evaluation of many scheduling and IaaS internal behaviour related scenarios. In response to the requirements of such scenarios, the new simulator introduces concepts such as: a unified model for resource sharing and a new energy metering framework with hierarchical and indirect metering options. Then, the comparison of several simulated situations to real-life IaaS behaviour is used to validate the simulator's functionality. Finally, a performance comparison is presented between DISSECT-CF and some currently available simulators.
Cloud computing provides a cheap and elastic platform for executing large scientific workflow applications, but it rises two challenges in prediction of makespan (total execution time): performance instability of Cloud instances and variant scheduling of dynamic schedulers. Estimating the makespan is necessary for IT managers in order to calculate the cost of execution, for which they can use Cloud simulators. However, the ideal simulated environment produces the same output for the same workflow schedule and input parameters and thus can not reproduce the Cloud variant behavior. In this paper, we define a model and a methodology to add a noise to the simulation in order to equalise its behavior with the Clouds’ one. We propose several metrics to model a Cloud fluctuating behavior and then by injecting them within the simulator, it starts to behave as close as the real Cloud. Instead of using a normal distribution naively by using mean value and standard deviation of workflow tasks’ runtime, we inject two noises in the tasks’ runtime: noisiness of tasks within a workflow (defined as average runtime deviation) and noisiness provoked by the environment over the whole workflow (defined as average environmental deviation). In order to measure the quality of simulation by quantifying the relative difference between the simulated and measured values, we introduce the parameter inaccuracy. A series of experiments with different workflows and Cloud resources were conducted in order to evaluate our model and methodology. The results show that the inaccuracy of the makespan’s mean value was reduced up to 59 times compared to naively using the normal distribution. Additionally, we analyse the impact of particular workflow and Cloud parameters, which shows that the Cloud performance instability is simulated more correctly for small instance type (inaccuracy of up to 11.5%), instead of medium (inaccuracy of up to 35%), regardless of the workflow. Since our approach requires collecting data by executing the workflow in the Cloud in order to learn its behavior, we conduct a comprehensive sensitivity analysis. We determine the minimum amount of data that needs to be collected or minimum number of test cases that needs to be repeated for each experiment in order to get less than 12% inaccuracy for our noising parameter. Additionally, in order to reduce the number of experiments and determine the dependency of our model against Cloud resource and workflow parameters, the conducted comprehensive sensitivity analysis shows that the correctness of our model is independent of workflow parallel section size. With our sensitivity analysis, we show that we can reduce the inaccuracy of the naive approach with only 40% of total number of executions per experiment in the learning phase. In our case, 20 executions per experiment instead of 50, and only half of all experiments, which means down to 20%, i.e. 120 test cases instead of 600.
Conference Paper
Execution of workflow applications in Cloud environments involves many uncertainties because of elastic resource provisioning and unstable performance of multitenant virtual machines (VM) instances over time. These uncertainties are usually either neglected by existing researches, or modeled with some probability distribution function. To address this gap, we extend a multi-objective workflow scheduling algorithm (MOHEFT) in two directions: (1) to deal with the dynamic nature of Cloud environments offering a potentially infinite amount of on-demand resources, and (2) to consider robustness as an objective that mitigates the variability in VM performance over time. Our new robust model, called R-MOHEFT, considers uncertainty in processing times of workflow activities without a precise estimation or known distribution function within an uncertainty interval. We approach this scheduling problem as a three-objective optimisation that considers makespan, monetary cost, and robustness as simultaneous objectives of a commercial Cloud environment. Our new algorithm is able to estimate the Pareto optimal set of scheduling solutions that resist against fluctuations in processing times three times better than its MOHEFT predecessor, with a tradeoff of only 15% worse Pareto frontier. R-MOHEFT’s hypervolume suffers by only 5% to 16%, compared to the MOHEFT’s drawback of 38% to surprisingly 87%, when the processing time fluctuates up to its double value.
Workflow scheduling has been around for more than two decades. With growing interest in service oriented computing architecture among researchers and corporate users, different platform like clusters computing, grid computing and most recent cloud computing, appeared on computing horizon. Cloud computing has attracted a lot of interest from all types of users. It gave rise to variety of applications and tasks with varied requirements. Heterogeneity in application's requirement catalyzes the provision of customized services for task types. Representation of tasks characteristics and intertask relationship through workflows is in use since the ages of automation. Scheduling of workflows not only maintain the hierarchical relationship between the tasks but also dictates the requirement of dynamic scheduling. This paper presents a variance based extensions of few promising dynamic scheduling policies supported by WorkflowSim. An exhaustive performance analysis presents strength and weakness of the authors' proposal. © Copyright 2016, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
Code available at: Workload modeling enables performance analysis and simulation of cloud resource management policies, which allows cloud providers to improve their systems’ Quality of Service (QoS) and researchers to evaluate new policies without deploying expensive large scale environments. However, workload modeling is challenging in the context of cloud computing due to the virtualization layer overhead, insufficient tracelogs available for analysis, and complex workloads. These factors contribute to a lack of methodologies and models to characterize applications hosted in the cloud. To tackle the above issues, we propose a web application model to capture the behavioral patterns of different user profiles and to support analysis and simulation of resources utilization in cloud environments. A model validation was performed using graphic and statistical hypothesis methods. An implementation of our model is provided as an extension of the CloudSim simulator.
Choosing the best-performing cloud for one's application is a critical problem for potential cloud customers. We propose CloudProphet, a trace-and-replay tool to predict a legacy application's performance if migrated to a cloud infrastructure. CloudProphet traces the workload of the application when running locally, and replays the same workload in the cloud for prediction. We discuss two key technical challenges in designing CloudProphet, and some preliminary results using a prototype implementation.