No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Dynamic deadline constrained multi-objective workflow scheduling in multi-cloud environments
... In recent years, the Laboratory of Complex Systems and Computational Intelligence of Taiyuan University of Science and Technology [20][21][22][23][24][25][26][27][28][29] has done a lot of research on data storage and modeling methods and has obtained a series of research results, among which the new modeling technology mentioned provides reference data for better solving the problem of damage inducement inversion model in this paper. In the follow-up research, we will further optimize the modeling method and deepen the related research of this paper. ...
Periodic change in reservoir water level will have a significant impact on berthing position, and the impact caused by irregular operation during berthing will cause damage to wharf pile foundations. However, most of the existing monitoring methods adopt irregular methods, so it is difficult to accurately identify and analyze the damage causes. Taking a high-piled wharf in the Three Gorges Reservoir area as an example, the uncertainty of reservoir water level change is quantitatively analyzed. By establishing a simplified parametric wharf calculation model, the data set of an inversion model of pile of a high-piled wharf under ship impact is obtained, and the inversion analysis of pile damage of a high-piled wharf under ship pile is carried out based on the artificial neural network model. The results show that the inversion model can accurately and efficiently identify the intensity of ship impact, and a low water level is better than a high water level in the identification of impact position. In this paper, the behavior of wharf structure before and after damage is analyzed symmetrically under the action of damage inducement. In summary, the inversion analysis method can basically meet the requirements of inversion identification of pile foundation damage of a high-pile wharf in a backwater fluctuation area under ship impact.
Deadline-aware transmission scheduling in immersive video streaming is crucial. The objective is to guarantee that at least a certain block in multi-links is fully delivered within their deadlines, which is referred to as delivery ratio. Compared with existing models that focus on maximizing throughput and ultra-low latency, which makes bandwidth resource allocation and user satisfaction locally optimized, immersive video streaming needs to guarantee more high-priority block delivery within personalized deadlines. In this paper, we propose a deadline and priority-constrained immersive video streaming transmission scheduling scheme. It builds an accurate bandwidth prediction model that can sensitively assist scheduling decisions. It divides video streaming into various media elements and performs scheduling based on the user's personalized latency sensitivity thresholds and the media element's priority. We evaluate our scheme via trace-driven simulations. Compared with existing models, the results further demonstrate the superiority of our scheme with 12{\%}-31{\%} gains in quality of experience (QoE).
bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D
ynamic
W
orkflow
S
cheduling in
F
og
C
omputing (DWSFC) is an important optimisation problem with many real-world applications. The current workflow scheduling problems only consider cloud servers but ignore the roles of mobile devices and edge servers. Some applications need to consider the mobile devices, edge, and cloud servers simultaneously, making them work together to generate an effective schedule. In this article, a new problem model for DWSFC is considered and a new simulator is designed for the new DWSFC problem model. The designed simulator takes the mobile devices, edge, and cloud servers as a whole system, where they all can execute tasks. In the designed simulator, two kinds of decision points are considered, which are the routing decision points and the sequencing decision points. To solve this problem, a new
M
ulti-
T
ree
G
enetic
P
rogramming (MTGP) method is developed to automatically evolve scheduling heuristics that can make effective real-time decisions on these decision points. The proposed MTGP method with a multi-tree representation can handle the routing decision points and sequencing decision points simultaneously. The experimental results show that the proposed MTGP can achieve significantly better test performance (reduce the makespan by up to 50%) on all the tested scenarios than existing state-of-the-art methods.
Most real-world problems involve some type of optimization problems that are often constrained. Numerous researchers have investigated several techniques to deal with constrained single-objective and multi-objective evolutionary optimization in many fields, including theory and application. This presented study provides a novel analysis of scholarly literature on constraint-handling techniques for single-objective and multi-objective population-based algorithms according to the most relevant journals and articles. As a contribution to this study, the paper reviews the main ideas of the most state-of-the-art constraint handling techniques in population-based optimization, and then the study addresses the bibliometric analysis, with a focus on multi-objective, in the field. The extracted papers include research articles, reviews, book/book chapters, and conference papers published between 2000 and 2021 for analysis. The results indicate that the constraint-handling techniques for multi-objective optimization have received much less attention compared with single-objective optimization. The most promising algorithms for such optimization were determined to be genetic algorithms, differential evolutionary algorithms, and particle swarm intelligence. Additionally, “Engineering,” “Computer Science,” and “ Mathematics” were identified as the top three research fields in which future research work is anticipated to increase.
With the fast development of mobile edge computing (MEC), there is an increasing demand for running complex applications on the edge. These complex applications can be represented as workflows where task dependencies are explicitly specified. To achieve better Quality of Service (QoS), computation offloading is widely used in the MEC environment. However, many existing computation offloading strategies only focus on independent computation tasks but overlook the task dependencies. Meanwhile, most of these strategies are based on search algorithms which are often time-consuming and hence not suitable for many delay-sensitive complex applications in MEC. Therefore, a highly efficient graph-based strategy was proposed in our recent work but it can only deal with simple workflow applications with linear (namely sequential) structure. For solving these problems, a novel graph-based strategy is proposed for workflow applications in MEC. Specifically, this strategy can deal with complex workflow applications with nonlinear (viz. parallel, selective and iterative) structures. Meanwhile, the offloading decision plan with the lowest energy consumption of the end-device under deadline constraint can be found by using the graph-based partition technique. We have comprehensively evaluated our strategy on FogWorkflowSim platform for complex workflow applications. Extensive numerical results demonstrate that the end device's energy consumption can be effectively reduced by 7.81% and 9.51% compared with PSO and GA by the proposed strategy. Meanwhile, the strategy running time is 1% and 0.2% of PSO and GA, respectively.
A workflow is an effective way for modeling complex applications and serves as a means for scientists and researchers to better understand the details of applications. Cloud computing enables the running of workflow applications on many types of computational resources which become available on-demand. As one of the most important aspects of cloud computing, workflow scheduling needs to be performed efficiently to optimize resources. Due to the existence of various resource types at different prices, workflow scheduling has evolved into an even more challenging problem on cloud computing. The present paper proposes a workflow scheduling algorithm in the cloud to minimize the execution cost of the deadline-constrained workflow. The proposed method, EDQWS, extends the current authors’ previous study (DQWS) and is a two-step scheduler based on divide and conquer. In the first step, the workflow is divided into sub-workflows by defining, scheduling, and removing a critical path from the workflow, similar to DQWS. The process continues until only chain-structured sub-workflows, called linear graphs, remain. In the second step which is linear graph scheduling, a new merging algorithm is proposed that combines the resulting linear graphs so as to reduce the number of used instances and minimize the overall execution cost. In addition, the current work introduces a scoring function to select the most efficient instances for scheduling the linear graphs. Experiments show that EDQWS outperforms its competitors, both in terms of minimizing the monetary costs of executing scheduled workflows and meeting user-defined deadlines. Furthermore, in more than 50% of the examined workflow samples, EDQWS succeeds in reducing the number of resource instances compared to the previously introduced DQWS method.
Here, we propose a fault-tolerant workflow scheduling algorithm that combines basic redundancies to reduce execution time through minimizing the redundancy overhead. We propose a graph-theory-based divide and conquer approach for selecting fault-tolerance strategies for workflow tasks. The appropriate strategy for each task is determined with respect to runtime situation and the position of the task in the graph. The main idea of the proposed algorithm is that resources are apportioned among concurrently executing tasks such that more replicas are assigned to tasks that benefit more from having extra replicas. We use the concept of concurrency graph for finding idle durations of resources which are used for processing additional task replicas. We also propose an opportunistic method for executing extra replicas of tasks in situations that some resources become idle. Furthermore, we propose a new mapping order scheme for ordering task replicas on resources. The proposed approach achieves a significant performance improvement over the existing approaches especially in situations where few resources are enrolled with the aim of cost reduction.
Multi-cloud is the use of multiple cloud computing in a single heterogeneous architecture. Workflow scheduling in multi-cloud computing is an NP-Hard problem for which many heuristics and meta-heuristics are introduced. This paper first presents a hybrid multi-objective optimization algorithm denoted as HGSOA-GOA, which combines the Seagull Optimization Algorithm (SOA) and Grasshopper Optimization Algorithm (GOA). The HGSOA-GOA applies chaotic maps for producing random numbers and achieves a good trade-off between exploitation and exploration, leading to an improvement in the convergence rate. Then, HGSOA-GOA is applied for scientific workflow scheduling problems in multi-cloud computing environments by considering factors such as makespan, cost, energy, and throughput. In this algorithm, a solution from the Pareto front is selected using a knee-point method and then is applied for assigning the scientific workflows’ tasks in a multi-cloud environment. Extensive comparisons are conducted using the CloudSim and WorkflowSim tools and the results are compared to the SPEA2 algorithm. The achieved results exhibited that the HGSOA-GOA can outperform other algorithms in terms of metrics such as IGD, coverage ratio, and so on.
Cloud computing constant evolution requires dynamic adjustments to service pricing and billing terms, considering provider infrastructures, customer requirements, and discount policies, among others. In this context, Customer Agreements (CA) are used to regulate the service provision including, among other information, the agreed service level in SLAs, pricing, and billing terms. Although many existing proposals and industrial tools support the definition of pricing and billing of cloud services, there is a lack in terms of customisation and automated monitoring tools integrated with the billing process. In this article, we provide a flexible billing proposal supporting CA that considers not only SLA terms but also customisable pricing and billing terms, possibly including compensations (i.e. discounts or overcharges) that apply when specified conditions are met. In addition, we developed an automated monitoring and analysis tool that we validate in a real-world industrial scenario. Moreover, we analyse and compare more than 50 existing industrial tools with our proposal, highlighting the advantages of its rule-based approach.
Cloud computing providers must constantly hold many idle compute instances available (e.g., for maintenance or for users with long-term contracts). A natural idea, which should intuitively increase the provider’s profit, is to sell these idle instances on a secondary market, for example, via a preemptible spot market. However, this ignores possible “market cannibalization” effects that may occur in equilibrium as well as the additional costs the provider experiences due to preemptions. To study the viability of offering a spot market, we model the provider’s profit optimization problem by combining queuing theory and game theory to analyze the equilibria of the resulting queuing system. Our main result is an easy-to-check condition under which a provider can simultaneously achieve a profit increase and create a Pareto improvement for the users by offering a spot market (using idle resources) alongside a fixed-price market. Finally, we illustrate our results numerically to demonstrate the effects that the provider’s costs and her strategy have on her profit.
This paper was accepted by Gabriel Weintraub, revenue management and market analytics.
Any application or resource that is made available to the users via an internet connection from a cloud platform is known as a cloud-based service. Any IT organization is highly dependent on its cloud architecture & platform planning that specifically caters to the needs of that specific organization.This paper work mainly focuses on top three cloud platforms that you can use to effectively manage the IT needs of your organization,namelyAmazonElastic Compute(AWS), Google Cloud Platform, and Microsoft Azure.Allthree of these major competitors in this field have their own merits which help them survive in this harsh competition. Paper work starts by a brief introduction of cloud computing followed by 3 main cloud platform architectures. Furthermore, we have shown comparative analysis of these platforms on different aspects like pricing, specification, support and administration, database, ML and AI support, storage, deployment tools, networking and security. This study will helpthecloud user to selectthebestcloud platformfor their respective purposes.
Internet of Things (IoT) aims to connect the real world made up of devices, sensors and actuators to the virtual world of Internet in order to interconnect devices with each other generating information from the gathered data. Devices, in general, have limited computational power and limited storage capacity. Cloud Computing (CC) has virtually unlimited capacity in terms of storage and computing power, and is based on sharing resources. Therefore, the integration between IoT and CC seems to be one of the most promising solutions. In fact, many of the biggest companies that offer Cloud Services are focusing on the IoT world to offer services also in this direction to their users. In this paper we compare the three main Cloud Platforms (Amazon Web Services, Google Cloud Platform and Microsoft Azure) regarding to the services made available for the IoT. After describing the typical architecture of an IoT application, we map the Cloud-IoT Platforms services with this architecture analyzing the key points for each platform. At the same time, in order to conduct a comparative analysis of performance, we focus on a service made available by all platforms (MQTT middleware) building the reference scenarios and the metrics to be taken into account. Finally, we provide an overview of platform costs based on different loads. The aim is not to declare a winner, but to provide a useful tool to developers to make an informed choice of a platform depending on the use case.
Providing resources and services from multiple clouds is becoming an increasingly promising paradigm. Workflow applications are becoming increasingly computation-intensive or data-intensive, with its resource requirement being maintained from multicloud environment in terms of pay-per-use pricing mechanism. Existing works of cloud workflow scheduling primarily target optimizing makespan or cost. However, the reliability of workflow scheduling is also a critical concern and even the most important metric of QoS (quality of service). In this paper, a multi-objective scheduling (MOS) algorithm for scientific workflow in multicloud environment is proposed, the aim of which is to minimize workflow makespan and cost simultaneously while satisfying the reliability constraint. The proposed MOS algorithm is according to particle swarm optimization (PSO) technology, and the corresponding coding strategy takes both the tasks execution location and tasks order of data transmission into consideration. On the basis of real-world scientific workflow models, extensive simulation experiments demonstrate the significant multi-objective performances improvement of MOS algorithm over the CMOHEFT algorithm and the RANDOM algorithm.
A dynamic multi-objective optimization control (DMOOC) scheme is proposed in this paper for the wastewater treatment process (WWTP), which can dynamically optimize the set-points of dissolved oxygen concentration and nitrate level with multiple performance indexes simultaneously. To overcome the difficulty of establishing multi-objective optimization (MOO) model for the WWTP, a neural network online modeling method is proposed, requiring only the process data of the plant. Then, the constructed MOO model with constraints is solved based on the NSGA-II (non-dominated sorting genetic algorithm-II), and the optimal set-point vector is selected from the Pareto set using the defined utility function. Simulation results, based on the benchmark simulation model 1 (BSM1), demonstrate that the energy consumption can be significantly reduced applying the DMOOC than the default PID control with the fixed set-points. Moreover, a tradeoff between energy consumption and effluent quality index can be considered.
Cloud applications are growing more and more complex because of user mobility, hardware heterogeneity, and multi-component nature. Today's cloud infrastructure paradigm, based on distant data centers are not able to provide consistent performance and low enough communication latency for future applications. These discrepancies can be accommodated using existing large-scale distributed cloud infrastructure , also known as Infinite Cloud, which is amalgam of several Data Centres hosted by a Telecom Network. The Infinite Cloud provides opportunity for applications with high capacity, high availability, and low latency. The Infinite Cloud infrastructure and federated cloud paradigms introduce several challenges due to the heterogeneous nature of the resources of di↵erent scale, latencies due to geographical locations and dynamic workload, to better accommodate distributed applications with increased diversity. Managing a vast heterogeneous infrastructure of this nature can no longer be done manually. Autonomous, distributed, collab-orative, and self-configuring systems need to be developed to manage the resources of the Infinite Cloud in order to meet application Service Level Agreements (SLAs), and the operators' internal management objectives, [8]. In this paper , we discuss some of the associated research challenges for such a system by formulating an optimization problem based on its constituent cost models. The decision maker takes into account the computational complexity as well as stability of the optimal solution.
Recently, with many advances in wireless sensor networks, big data, mobile and cloud computing, Cyber-Physical Systems (CPS) can tightly couple cyber space with the physical world better than ever before. Also, the cloud-based systems can provide massive storage resources and low-cost computing as well as the flexibility of customizing the operating environment to Complex Industrial Applications (CIA). In our view, Cloud-integrated CPS (CCPS) will open the door to allow previously unachievable application scenarios to be built, deployed, managed and controlled effectively. In this paper, we propose a novel architecture of CCPS (termed CCPSA) and outline the enabling technologies for CIA. Then, we dissect three potential challenges and provide solutions from the perspective of CIA, including virtualized resource management techniques, the scheduling of cloud resources, and life cycle management. We hope this paper can provide insight and a roadmap for future research efforts in the emerging field of CCPS.
The distributed hybrid flow shop scheduling problem (DHFSP), which integrates distributed manufacturing models with parallel machines, has gained significant attention. However, in actual scheduling, some adjacent machines do not have buffers between them, resulting in blocking. This paper focuses on addressing the DHFSP with blocking constraints (DBHFSP) based on the actual production conditions. To solve DBHFSP, we construct a mixed integer linear programming (MILP) model for DBHFSP and validate its correctness using the Gurobi solver. Then, an advanced iterated greedy (AIG) algorithm is designed to minimize the makespan, in which we modify the Nawaz, Enscore, and Ham (NEH) heuristic to solve blocking constraints. To balance the global and local search capabilities of AIG, two effective inter-factory neighborhood search strategies and a swap-based local search strategy are designed. Additionally, each factory is mutually independent, and the movement within one factory does not affect the others. In view of this, we specifically designed a memory-based decoding method for insertion operations to reduce the computation time of the objective. Finally, two shaking strategies are incorporated into the algorithm to mitigate premature convergence. Five advanced algorithms are used to conduct comparative experiments with AIG on 80 test instances, and experimental results illustrate that the makespan and the relative percentage increase (RPI) obtained by AIG are 1.0% and 86.1% respectively, better than the comparative algorithms.
Cloud platforms scheduling resources based on the demand of the tasks submitted by the users, is critical to the cloud provider's interest and customer satisfaction. In this paper, we propose a multi-objective cloud task scheduling algorithm based on an evolutionary multi-factorial optimization algorithm. Firstly, we choose execution time, execution cost, and virtual machines load balancing as the objective functions to construct a multi-objective cloud task scheduling model. Secondly, the multi-factor optimization (MFO) technique is applied to the task scheduling problem, and the task scheduling characteristics are combined with the multi-objective multi-factor optimization (MO-MFO) algorithm to construct an assisted optimization task. Finally, a dynamic adaptive transfer strategy is designed to determine the similarity between tasks according to the degree of overlap of the MFO problem and to control the intensity of knowledge transfer. The results of simulation experiments on the cloud task test dataset show that our method significantly improves scheduling efficiency, compared with other evolutionary algorithms (EAs), the scheduling method simplifies the decomposition of complex problems by a multi-factor approach, while using knowledge transfer to share the convergence direction among sub-populations, which can find the optimal solution interval more quickly and achieve the best results among all objective functions.
Cloud computing offers resources to users for computing and storage needs on the policy pay-what-you-use model. A private cloud is an excellent cost-saving option because the user owns the resources and can be used free to execute workflow applications. However, when private cloud resources are insufficient to fulfill the needs of workflow applications, the public cloud is the only option. To this end, a hybrid cloud (HC) is an elegant way to combine public and private cloud resources to meet the requirements of workflow applications. Therefore, large enterprises prefer the HC, but task scheduling in the HC is complicated. Therefore, this study proposed a Deadline-constrained Cost-aware Workflow Scheduling (DCWS) algorithm to address task scheduling problems in the HC. Firstly, we will execute maximum workflow tasks on the private cloud owned by users under the deadline constraint. Secondly, send the unscheduled workflow tasks to the public cloud for execution. Lastly, we will ensure scheduling whole workflow tasks on the private cloud and the communication channels (for sending to the public cloud) to satisfy the task-precedence needs and the task deadline constraint while incurring a minimal monetary cost. Furthermore, we reduce task slack time with the help of the sub-deadline division technique and search for an ideal virtual machine (VM) to reduce further execution time and monetary cost. The experimental results prove that the DCWS algorithm outperformed existing algorithms.
It is necessary to accelerate digital development to make digital economy, society and government shine brightly in the future. Digital technology and the real economy will be deeply integrated, and a large number of new industries and new models will emerge. Cloud computing is an important industry to create further advantages in the digital economy. As the key to the development of enterprise business, cloud computing is the decisive factor. The ability to support billing for the entire cloud is a vital link, and it is the core capability of the whole system. It requires high accuracy and performance. Based on cloud billing, this paper analyzes the challenges cloud computing service support faces. It studies the architecture upgrade, builds the cloud service billing support capability based on stream native technology, meets the flexible billing needs of cloud services, and realizes the rapid and efficient operation support of cloud services. This system has been put into production and played a significant role strongly supporting the high-quality development needs of cloud business billing.KeywordsStream NativeCloud Computing BillingBilling Message
With the development of cloud computing, a growing number of workflows are deployed in cloud platform that can dynamically provide cloud resources on demand for users. In clouds, one basic problem is how to schedule workflow under the deadline constraint and minimize the execution cost. As the capability of cloud resources getting higher, the required cost is also rising. Capability of some resources exceeds the need of users, which leads to higher cost, and the budget of users should be considered. In this paper, a novel scheduling algorithm, named DB-ACO, is proposed to minimize the execution cost for the workflow with deadline and budget constraints. DB-ACO is verified on four typical scientific workflows, and the experiments results show it outperforms four state-of-the-art methods, especially for CyberShake.
Note to Practitioners
—Budget and deadline are important requirements for users in cloud computing, which are used as constraints. Extensive works have been devoted to minimize the cost of workflows execution with different scheduling strategies. However, most of them only consider one single constraint and assume the constraint is simple and loose, which is impractical in actual scenarios due to higher requirement of users. This paper investigates a novel scheduling algorithm DB-ACO to optimize cost under budget and deadline. DB-ACO combines heuristic and meta-heuristic, it uses ant colony optimization to optimize the execution cost under the deadline and budget constraints: each ant sorts tasks on the basis of the combination of the pheromone trail and heuristic information, the deadline and budget are distributed fairly to each task by a novel distribution method, then the service selection rules are introduced to build solution.
Computationally intensive applications with a wide range of requirements are advancing to cloud computing platforms. However, with the growing demands from users, cloud providers are not always able to provide all the prerequisites of the application. Hence, flexible computation and storage systems, such as multi-cloud systems, emerged as a suitable solution. Different charging mechanisms, vast resource configuration, different energy consumption, and reliability are the key issues for multi-cloud systems. To address these issues, we propose a multi-workflow scheduling framework for multi-cloud systems, intending to lower the monetary cost and energy consumption while enhancing the reliability of application execution. Our proposed platform presents different methods (utilizing resource gaps, the DVFS utilized method, and a task duplication mechanism) to ensure each application's requirement. The Weibull distribution is used to model task reliability at different resource fault rates and fault behavior. Various synthetic workflow applications are used to perform simulation experiments. The results of the performance evaluation demonstrated that our proposed algorithms outperform (in the terms of resource rental cost, efficient energy consumption, and improved reliability) state-of-the-art algorithms for multi-cloud systems.
Shared autonomous electric mobility has attracted significant interest in recent years due to its potential to save energy consumption, enhance mobility accessibility, reduce air pollution, mitigate traffic congestion, etc. Although providing convenient, low-cost, and environmentally-friendly mobility, there are still some roadblocks to achieve efficient shared autonomous electric mobility, e.g., how to enable the accessibility of shared autonomous electric vehicles in time. To overcome these roadblocks, in this paper, we design
Safari
, an efficient
S
hared
A
utonomous electric vehicle
F
leet m
A
nagement system with joint
R
epositioning and charg
I
ng based on dynamic deadlines to improve both user experience and operating profits. Our
Safari
considers not only the highly dynamic user demand for
vehicle repositioning
(i.e., where to relocate) but also many practical factors like the time-varying charging pricing for
charging scheduling
(i.e., where to charge). To perform the two tasks efficiently, in
Safari
, we design a dynamic deadline-based deep reinforcement learning algorithm, which generates dynamic deadlines via usage prediction combined with an error compensation mechanism to adaptively learn the optimal decisions for satisfying highly dynamic and unbalanced user demand in real time. More importantly, we implement and evaluate the
Safari
system with 10-month real-world shared electric vehicle data, and the extensive experimental results show that our
Safari
achieves 100% of accessibility and effectively reduces 26.2% of charging costs and reduces 31.8% of vehicle movements for energy saving with a small runtime overhead at the same time. Furthermore, the results also show
Safari
has a great potential to achieve efficient and accessible shared autonomous electric mobility during its long-term expansion and evolution process.
Dynamic Multi-objective Optimization Evolutionary Algorithm (DMOEA) is a promising approach for solving Dynamic Multi-objective Optimization Problems (DMOPs), which has attracted extensive attention owing to its wide application value recently. The focus and difficulty of DMOEA research is to design algorithms that can quickly respond to environmental changes. To achieve this goal, a novel DMOEA based on particle swarm prediction strategy and prediction adjustment strategy is proposed in this paper. After detecting environmental changes, the particle swarm prediction strategy is implemented through the population information at the moment before environmental change to form a predicted population that approximates the new Pareto optimal set (PS)/Pareto optimal front (PF) as much as possible. In addition, by comparing the individual information of the predicted population with that of the population at the moment before the environmental change, a prediction adjustment strategy is given to leave the better individuals to form the initial population in the new environment. The performance of the algorithm is evaluated on the DF test cases and compared with the advanced DMOEAs. The results show that the algorithm has competitive performance in most of the test problems.
Customers in cloud spot market choose from a set of computing resources (spot instances) some of which are same along one or more of the dimensions of hardware configuration, hardware capacity, software, and location (or zones within the same region). While prior research in IS has shown that cloud market consumers do not substitute identical spot instances across distant locations or regions due to costs associated with latency, there is a paucity of research examining spot price dynamics of similar instances which are same along only some of the dimensions (or characteristics) within a region. Using the economic theory of substitution in production and consumption as our theoretical lens, our empirical analysis finds evidence that consumers substitute similar computing resources in the spot market. Further, we find that while hardware configuration similarity between two instances is positively related to spot instance substitution, this substitution effect along hardware configuration is moderated by similarity along hardware capacity and location (zone). Our work contributes to the literature on cloud spot market by providing empirical evidence of the substitution effect among similar spot resources within a region, and thus help in understanding the choices of consumers in this market. We conclude by discussing how practitioners can leverage these findings in the procurement of computing resources from cloud spot market.
To promote research on dynamic constrained multiobjective optimization, we first propose a group of generic test problems with challenging characteristics, including different modes of the true Pareto front (e.g., convexity-concavity and connectedness-disconnectedness) and the changing feasible region. Subsequently, motivated by the challenges presented by dynamism and constraints, we design a dynamic constrained multiobjective optimization algorithm with a nondominated solution selection operator, a mating selection strategy, a population selection operator, a change detection method, and a change response strategy. The designed nondominated solution selection operator can obtain a nondominated population with diversity when the environment changes. The mating selection strategy and population selection operator can adaptively handle infeasible solutions. If a change is detected, the proposed change response strategy reuses some portion of the old solutions in combination with randomly generated solutions to reinitialize the population, and a steady-state update method is designed to improve the retained previous solutions. The experimental results show that the proposed test problems can be used to clearly distinguish the performance of algorithms, and that the proposed algorithm is very competitive for solving dynamic constrained multiobjective optimization problems in comparison with state-of-the-art algorithms.
In recent years, more and more large-scale data processing and computing workflow applications run on heterogeneous clouds. Such cloud applications with precedence-constrained tasks are usually deadline-constrained and their scheduling is an essential problem faced by cloud providers. Moreover, minimizing the workflow execution cost based on cloud billing periods is also a complex and challenging problem for clouds. In realizing this, we first model the workflow applications as I/O Data-aware Directed Acyclic Graph (DDAG), according to clouds with global storage systems. Then, we mathematically state this deadline-constrained workflow scheduling problem with the goal of minimum execution financial cost. We also prove that the time complexity of this problem is NP-hard by deducing from a multidimensional multiple-choice knapsack problem. Thirdly, we propose a heuristic cost-efficient task scheduling strategy called CETSS, which includes workflow DDAG model building, task subdeadline initialization, greedy workflow scheduling algorithm, and task adjusting method. The greedy workflow scheduling algorithm mainly consists of dynamical task renting billing period sharing method and unscheduled task subdeadline relax technique. We perform rigorous simulations on some synthetic randomly generated applications and real-world applications, such as Epigenomics, CyberShake, and LIGO. The experimental results clearly demonstrate that our proposed heuristic CETSS outperforms the existing algorithms and can effective save the total workflow execution cost. In particular, CETSS is very suitable for large workflow applications.
This paper presents distance-based immune generalised differential evolution (DIGDE), an improved algorithmic approach to tackle dynamic multi-objective optimisation problems (DMOPs). Its novelty is using the inverted generational distance (IGD) as an indicator in its selection mechanism to guide the search. DIGDE is based on the immune generalised differential evolution (Immune GDE3) algorithm which combines differential evolution (DE) fast convergence ability and artificial immune systems (AIS) principles for good diversity preservation. A thorough empirical evaluation is carried out on novel benchmark problems configured with different dynamic characteristics. DIGDE's experimental results show an overall improved statistically supported performance in terms of solutions approximation and better achieved distributions. Using IGD as a searching indicator allows DIGDE to achieve better performance and robustness in comparison to state-of-the-art methods when facing different change frequencies and severity levels.
Dynamic multi-objective optimization problems (DMOPs), in which the environments change over time, have attracted many researchers’ attention in recent years. Since the Pareto set (PS) or the Pareto front (PF) can change over time, how to track the movement of the PS or PF is a challenging problem in DMOPs. Over the past few years, lots of methods have been proposed, and the prediction based strategy has been considered the most effective way to track the new PS. However, the performance of most existing prediction strategies depends greatly on the quantity and quality of the historical information and will deteriorate due to non-linear changes, leading to poor results. In this paper, we propose a new prediction method, named MOEA/D-GMM, which incorporates the Gaussian Mixture Model (GMM) into the MOEA/D framework for the prediction of the new PS when changes occur. Since GMM is a powerful non-linear model to accurately fit various data distributions, it can effectively generate solutions with better quality according to the distributions. In the proposed algorithm, a change type detection strategy is first designed to estimate an approximate PS according to different change types. Then, GMM is employed to make a more accurate prediction by training it with the approximate PS. To overcome the shortcoming of a lack of training solutions for GMM, the Empirical Cumulative Distribution Function (ECDF) method is used to resample more training solutions before GMM training. Experimental results on various benchmark test problems and a classical real-world problem show that, compared with some state-of-the-art dynamic optimization algorithms, MOEA/D-GMM outperforms others in most cases.
Dynamic multi-objective optimization problems (DMOPs) require evolutionary algorithms (EAs) to accurately track the Pareto-optimal Front (PF) and generate the solutions along the PF in the constantly changing environment. In order to solve the DMOPs, a novel quantile-guided dual prediction strategies evolutionary algorithm (NQDPEA) is proposed in this paper. Quantiles are often employed to characterize data in statistics. In NQDPEA, the evolution of the population is guided by the quantile, which is to predict the position of the quantile in a new environment through historical quantile information. Then, a new solution set is expanded according to the location of the new quantile. Moreover, its prediction strategies not only predict Pareto-optimal set (PS) by quantile in the decision space but also predict the PF by quantile in objective space and then mapping back to decision space. Through the adaptive combination strategy, the proportion of the new solutions produced by each prediction strategy changes adaptively. To prove the performance of NQDPEA, it is compared with four powerful EAs on 13 test instances. Experimental results show that NQDPEA can effectively generate high quality solutions uniformly along PF.
Workflows are prevalent in today’s computing infrastructures as they support many domains. Different Quality of Service (QoS) requirements of both users and providers makes workflow scheduling challenging. Meeting the challenge requires an overview of state-of-art in workflow scheduling. Sifting through literature to find the state-of-art can be daunting, for both newcomers and experienced researchers. Surveys are an excellent way to address questions regarding the different techniques, policies, emerging areas, and opportunities present, yet they rarely take a systematic approach and publish their tools and data on which they are based. Moreover, the communities behind these articles are rarely studied. We attempt to address these shortcomings in this work.
We introduce and open-source an instrument used to combine and store article meta-data. Using this meta-data, we characterize and taxonomize the workflow scheduling community and four areas within workflow scheduling: 1) the workflow formalism, 2) workflow allocation, 3) resource provisioning, and 4) applications and services. In each characterization, we obtain important keywords overall and per year, identify keywords growing in importance, get insight into the structure and relations within each community, and perform a systematic literature survey per part to validate and complement our taxonomies
Cloud platforms have recently become a popular target execution environment for numerous workflow applications. Hence, effective workflow scheduling strategies in cloud environments are in high demand. However, existing scheduling algorithms are grounded on an idealized target platform model where virtual machines are fully connected, and all communications can be performed concurrently. A significant aspect neglected by them is endpoint communication contention when executing workflows, which has a large impact on workflow makespan. This article investigates how to incorporate contention awareness into cloud workflow scheduling and proposes a new practical scheduling model. Endpoint communication contention-aware List Scheduling Heuristic (ELSH) is designed to minimize workflow makespan. It uses a novel task ranking property and schedules data communications to communication resources besides scheduling tasks to computing resources. Moreover, a rescheduling technique is employed to improve the schedule. In experiments, ELSH is evaluated against the traditional contention-oblivious list scheduling algorithm, which is adapted to address contention during execution in practice. The experimental results reveal that ELSH performs more efficaciously compared with the adapted traditional ones.
Note to Practitioners
—This article aims to advance the state of the art for workflow scheduling in clouds by taking into account endpoint communication contention that can occur in practice but has largely been neglected in existing investigations. A scheduling method called Endpoint communication contention-aware List Scheduling Heuristic (ELSH) is then proposed to optimize workflow makespan. Experimental results based on synthetic and realistic workflows show that ELSH performs better than traditional scheduling algorithms that fail to consider endpoint communication contention, especially for the workflow with a large communication-to-computation-cost ratio. The proposed approach can be readily put into use and help cloud service providers to offer their customers high-quality services when executing the latter’s workflows.
Todays large-scale parallel workflows are often processed on heterogeneous distributed computing platforms. From an economic perspective, computing resource providers should minimize the cost while offering high service quality. It has become well-organized that energy consumption accounts for a large part of a computing systems total cost, and timeliness and reliability are two important service indicators. This article studies the problem of scheduling a parallel workflow that minimizes the system energy consumption under the constraints of response time and reliability. We first mathematically formulate this problem as a Non-linear Mixed Integer Programming problem. Since this problem is hard to solve directly, we present some highly-efficient heuristic solutions. Specifically, we first develop an algorithm that minimizes the schedule length while meeting reliability requirement, on top of which we propose a processor-merging algorithm and a slack time reclamation algorithm using a dynamic voltage frequency scaling (DVFS) technique to reduce energy consumption. The processor-merging algorithm tries to turn off some energy-inefficient processors such that energy consumption can be minimized. The DVFS technique is applied to scale down the processor frequency at both processor and task levels to reduce energy consumption. Experimental results on two real-life workflows and extensive synthetic parallel workflows demonstrate their effectiveness.
Internet of Things (IoT) is a huge network and establishes ubiquitous connections between smart devices and objects. The flourishing of IoT leads to an unprecedented data explosion, traditional data storing or processing techniques have the problem of low efficiency, and if the data is used maliciously, the security loss may be further caused. Multi-cloud is a highperformance secure computing platform, which combines multiple cloud providers for data processing, and the distributed multicloud platform ensures the security of data to some extent. Based on multi-cloud and task scheduling in IoT, this paper constructs a many-objective distributed scheduling model, which includes six objectives of total time, cost, cloud throughput, energy consumption, resource utilization, and balancing load. Further, this paper presents a many-objective intelligent algorithm with sine function to implement the model, which considers the variation tendency of diversity strategy in the population is similar to the sine function. The experimental results demonstrate excellent scheduling efficiency and hence enhancing the security. This work provides a new idea for addressing the difficult problem of data processing in IoT.
Microservices are widely used for flexible software development. Recently, containers have become the preferred deployment technology for microservices because of fast start-up and low overhead. However, the container layer complicates task scheduling and auto-scaling in clouds. Existing algorithms do not adapt to the two-layer structure composed of virtual machines and containers, and they often ignore streaming workloads. To this end, this paper proposes an Elastic Scheduling for Microservices (ESMS) that integrates task scheduling with auto-scaling. ESMS aims to minimize the cost of virtual machines while meeting deadline constraints. Specifically, we define the task scheduling problem of microservices as a cost optimization problem with deadline constraints and propose a statistics-based strategy to determine the configuration of containers under a streaming workload. Then, we propose an urgency-based workflow scheduling algorithm that assigns tasks and determines the type and quantity of instances for scale-up. Finally, we model the mapping of new containers to virtual machines as a variable-sized bin-packing problem and solve it to achieve integrated scaling of the virtual machines and containers. Via simulation-based experiments with well-known workflow applications, the ability of ESMS to improve the success ratio of meeting deadlines and reduce the cost is verified through comparison with existing algorithms.
Big data processing applications are becoming more and more complex. They are no more monolithic in nature but instead they are composed of decoupled analytical processes in the form of a workflow. One type of such workflow applications is stream workflow application, which integrates multiple streaming big data applications to support decision making. Each analytical component of these applications runs continuously and processes data streams whose velocity will depend on several factors such as network bandwidth and processing rate of parent analytical component. As a consequence, the execution of these applications on cloud environments requires advanced scheduling techniques that adhere to end user's requirements in terms of data processing and deadline for decision making. In this paper, we propose two Multicloud scheduling and resource allocation techniques for efficient execution of stream workflow applications on Multicloud environments while adhering to workflow application and user performance requirements and reducing execution cost. Results showed that the proposed genetic algorithm is an adequate and effective for all experiments.
Workflow scheduling is a largely studied research topic in cloud computing, which targets to utilize cloud resources for workflow tasks by considering the objectives specified in QoS. In this paper, we model dynamic workflow scheduling problem as a dynamic multi-objective optimization problem (DMOP) where the source of dynamism is based on both resource failures and the number of objectives which may change over time. Software faults and/or hardware faults may cause the first type of dynamism. On the other hand, confronting real-life scenarios in cloud computing may change number of objectives at runtime during the execution of a workflow. In this study, we propose a prediction-based dynamic multi-objective evolutionary algorithm, called NN-DNSGA-II algorithm, by incorporating artificial neural network with the NSGA-II algorithm. Additionally, five leading non-prediction based dynamic algorithms from the literature are adapted for the dynamic workflow scheduling problem. Scheduling solutions are found by the consideration of six objectives: minimization of makespan, cost, energy and degree of imbalance; and maximization of reliability and utilization. The empirical study based on real-world applications from Pegasus workflow management system reveals that our NN-DNSGA-II algorithm significantly outperforms the other alternatives in most cases with respect to metrics used for DMOPs with unknown true Pareto-optimal front, including the number of non-dominated solutions, Schott’s spacing and Hypervolume indicator.
The Infrastructure as a Service (IaaS) cloud industry that relies on leasing virtual machines (VMs) has significant portion of business values of finding the dynamic equilibrium between two conflicting phenomena: underutilization and surging congestion. Spot instance has been proposed as an elegant solution to overcome these challenges, with the ultimate goal to achieve greater profits. However, previous studies on recent spot pricing schemes reveal artificial pricing policies that do not comply with the dynamic nature of these phenomena. Motivated by these facts, this paper investigates dynamic pricing of stagnant resources in order to maximize cloud revenue. Specifically, our proposed approach manages multiple classes of virtual machines in order to achieve the maximum expected revenue within a finite discrete time horizon. For this sake, the proposed approach leverages the Markov decision processes with a number of properties under optimum controlling conditions that characterize a model's behaviour. Further, this approach applies approximate stochastic dynamic programming using linear programming to create a practical model. Experimental results confirm that this approach of dynamic pricing can scale up or down the price efficiently and effectively, according to the stagnant resources and the load thresholds. These results provide significant insights to maximizing the IaaS cloud revenue.
Basic science is becoming ever more computationally intensive, increasing the need for large-scale compute and storage resources, be they within a High Performance Computer cluster, or more recently within the cloud. In most cases, large scale scientific computation is represented as a workflow for scheduling and runtime provisioning. Such scheduling becomes an even more challenging problem on cloud systems due to the dynamic nature of the cloud, in particular, the elasticity, the pricing models (both static and dynamic), the non-homogeneous resource types, the vast array of services, and virtualization. This mapping of workflow tasks on to a set of provisioned instances is an example of the general scheduling problem and is NP-complete. In addition, we also need to ensure that certain runtime constraints are met - the most typical being the cost of the computation and the time which that computation requires to complete. In this article, we introduce a new heuristic scheduling algorithm, Budget Deadline Aware Scheduling (BDAS), that addresses eScience workflow scheduling under budget and deadline constraints in Infrastructure as a Service (IaaS) clouds. The novelty of our work is satisfying both budget and deadline constraints while introducing a tunable cost-time trade off over heterogeneous instances. In addition, we study the stability and robustness of our algorithm by performing sensitivity analysis. The results demonstrate that overall BDAS finds a viable schedule for more than 40000 test cases accomplishing both defined constraints: budget and . Moreover, our algorithm achieves a higher success rate when compared to state of the art algorithms. IEEE
Recently, several researchers within the evolutionary and swarm computing community have been interested in solving dynamic multi-objective problems where the objective functions, the problem's parameters, and/or the constraints may change over time. According to the related literature, most works have focused on the dynamicity of objective functions, which is insufficient since also constraints may change over time along with the objectives. For instance, a feasible solution could become infeasible after a change occurrence, and vice versa. Besides, a non-dominated solution may become dominated, and vice versa. Motivated by these observations, we devote this paper to focus on the dynamicity of both: (1) problem's constraints and (2) objective functions. To achieve our goal, we propose a new self-adaptive penalty function and a new feasibility driven strategy that are embedded within the NSGA-II and that are applied whenever a change is detected. The feasibility driven strategy is able to guide the search towards the new feasible directions according to the environment changes. The empirical results have shown that our proposal is able to handle various challenges raised by the problematic of dynamic constrained multi-objective optimization. Moreover, we have compared our new dynamic constrained NSGA-II version, denoted as DC-MOEA, against two existent dynamic constrained evolutionary algorithms. The obtained results have demonstrated the competitiveness and the superiority of our algorithm on both aspects of convergence and diversity.
Recent studies focus primarily on low energy consumption or execution time for task scheduling with precedence constraints in heterogeneous computing systems. In most cases, system reliability is more important than other performance metrics. In addition, energy consumption and system reliability are two conflicting objectives. A novel bi-objective genetic algorithm (BOGA) to pursue low energy consumption and high system reliability for workflow scheduling is presented in this paper. The proposed BOGA offers users more flexibility when jobs are submitted to a data center. On the basis of real-world and randomly generated application graphs, numerous experiments are conducted to evaluate the performance of the proposed algorithm. In comparison with excellent algorithms such as multi-objective heterogeneous earliest finish time (MOHEFT) and multi-objective differential evolution (MODE), BOGA performs significantly better in terms of finding the spread of compromise solutions.
Clouds are becoming an important platform for scientific workflow applications. However, with many nodes being deployed in clouds, managing reliability of resources becomes a critical issue, especially for the real-time scientific workflow execution where deadlines should be satisfied. Therefore, fault tolerance in clouds is extremely essential. The PB (primary backup) based scheduling is a popular technique for fault tolerance and has effectively been used in the cluster and grid computing. However, applying this technique for real-time workflows in a virtualized cloud is much more complicated and has rarely been studied. In this paper, we address this problem. We first establish a real-time workflow fault-tolerant model that extends the traditional PB model by incorporating the cloud characteristics. Based on this model, we develop approaches for task allocation and message transmission to ensure faults can be tolerated during the workflow execution. Finally, we propose a dynamic fault-tolerant scheduling algorithm, FASTER, for real-time workflows in the virtualized cloud. FASTER has three key features: 1) it employs a backward shifting method to make full use of the idle resources and incorporates task overlapping and VM migration for high resource utilization, 2) it applies the vertical/horizontal scaling-up technique to quickly provision resources for a burst of workflows, and 3) it uses the vertical scaling-down scheme to avoid unnecessary and ineffective resource changes due to fluctuated workflow requests. We evaluate our FASTER algorithm with synthetic workflows and workflows collected from the real scientific and business applications and compare it with six baseline algorithms. The experimental results demonstrate that FASTER can effectively improve the resource utilization and schedulability even in the presence of node failures in virtualized clouds.
Recently, we have witnessed workflows from science and other data-intensive applications emerging on Infrastructureas- a-Service (IaaS) clouds, and many workflow service providers offering workflow-as-a-service (WaaS). The major concern of WaaS providers is to minimize the monetary cost of executing workflows in the IaaS clouds. The selection of virtual machines (instances) types significantly affects the monetary cost and performance of running a workflow. Moreover, IaaS cloud environment is dynamic, with high performance dynamics caused by the interference from concurrent executions and price dynamics like spot prices offered by Amazon EC2. Therefore, we argue that WaaS providers should have the notion of offering probabilistic performance guarantees for individual workflows to explicitly expose the performance and cost dynamics of IaaS clouds to users. We develop a scheduling system called Dyna to minimize the expected monetary cost given the user-specified probabilistic deadline guarantees. Dyna includes an A$ -based instance configuration method for performance dynamics, and a hybrid instance configuration refinement for using spot instances. Experimental results with three scientific workflow applications on Amazon EC2 and a cloud simulator demonstrate (1) the ability of Dyna on satisfying the probabilistic deadline guarantees required by the users; (2) the effectiveness on reducing monetary cost in comparison with the existing approaches.