No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Mapping of Service Level Objectives to Performance Queries
Abstract
The concept of service level agreements (SLAs) defines the idea of a reliable contract between service providers and their users. SLAs provide information on the scope, the quality and the responsibilities of a service and its provider. Service level objectives (SLOs) define the detailed, measurable conditions of the SLAs. After service deployment, SLAs are monitored for situations, that lead to SLA violations. However, the SLA monitoring infrastructure is usually specific to the underlying system infrastructure, lacks generalization, and is often limited to measurement-based approaches. This makes it hard to apply the results from SLA monitoring in other stages of the software life-cycle. In this paper we propose the mapping of concerns defined in SLAs to the performance metrics queries using the Descartes Query Language (DQL). The benefit of our approach is that the same performance query can then be reused for evaluation of performance concerns throughout the entire life-cycle, and regardless of which approach is used for evaluation.
... Specifying performance concerns DPE requires a language to formulate concerns. We present how to query performance indicators, as well as the latest SLA language features, which we integrated into the DQL (Descartes Query Language) [2,11] (Section 2). Answering using measurements One way to answer performance concerns is interpreting measurements derived from application performance measurement (APM) tools. ...
... Thereby, DQL is a realization of the vision of Declarative Performance Engineering (DPE), which decouples the description of user concerns (performance questions and goals) from the task of selecting and applying a specific solution approach. DQL allows for specification of the following types of concerns: (i) performance indicators [2], (ii) service level agreements (SLAs) [11], and (iii) variation and what if analysis. Examples for these types of concerns can be seen in the following listings. ...
... The corresponding processing framework, depicted in Figure 2, enables to integrate performance evaluation techniques that may benefit from reusing a set of generic processing algorithms. More details on DQL and query processing can be found in [2,11]. ...
Performance is of particular relevance to software system design, operation, and evolution. However, the application of performance engineering approaches to solve a given user concern is challenging and requires expert knowledge. In this tutorial paper, we guide the reader step-by-step through the answering of performance concerns following the idea of declarative performance engineering. We explain tools available online, which can be used for automating huge parts of the software performance engineering process. In particular, we present a performance concern language, for which we provide automated answering and visualization referring to measurement-based and model-based analysis. We also detail how to derive performance models using automated extraction of architectural performance models and modeling of parametric dependencies.
... Cao [20] outlines a research agenda for an SLO-oriented management layer for cloud-edge infrastructure; Cardelli et al. [21] design an autonomous elasticity mechanism to ensure QoS in cloud-edge service chains. The authors in [22] discuss the importance of controlling distributed systems with DeepSLOs, i.e., such that span multiple abstraction layers; their vision, however, was not implemented yet, as counts for [23]. ...
... SLA provides information on the scope, the quality and the responsibilities of each microservice and its provider [19]. They include SLOs that define the detailed, measurable conditions. ...
... QoS Metrics. SLA provides information on the scope, the quality and the responsibilities of each micro service and its provider [11]. They involve QoS metrics that define the detailed, measurable conditions. ...
... There is oftentimes confusion in the use of SLO and SLA. SLOs are particular and measurable characteristics of the SLA like throughput, response time, availability, or quality [14]. Apart of SLO application domain, the objectives can be categorized as: i) performance service level objectives; ii) security service level objectives; iii) data management service level objectives; iv) personal data protection service level objectives [15] [16]. ...
The question of how to specify, provide and measure service quality for network end-users has been of utmost interest for service and network infrastructure providers and their clients as well. The Service Level Agreement (SLA) is a beneficial tool in formalising the interrelationships resulting from a negotiation among all participating actors with the target of achieving a common comprehension concerning delivery of services, its priorities, quality, responsibilities, and other relevant parameters. A horizontal SLA is an agreement between two service-providers existing at the same architectural layer (as for example two Internet Protocol (IP) or two Optical Transport Network (OTN) domains). A vertical SLA is an agreement between two individual providers at two different architectural layers (for instance, between an optical network and the core MPLS network). A service has to be defined without ambiguity utilizing Service Level Specifications (SLS) and three information types must be described: i) The QoX metrics as well as their corresponding thresholds; ii) A method of service performance measurement; iii) Service schedule. In this work we present preliminary simulation results that enable the development of a generic methodology for SLA modelling and establishment that will lead to a win-win situation for all involved actors. As an example, we put special attention in the benefits obtained by Optical Networks operators.
The question of how to specify, provide and measure service quality for network end-users has been of utmost interest for service and network infrastructure providers and their clients as well. The Service Level Agreement (SLA) is a beneficial tool in formalizing the interrelationships resulting from a negotiation among all participating actors with the target of achieving a common comprehension concerning delivery of services, its priorities, quality, responsibilities, and other relevant parameters. A horizontal SLA is an agreement between two service-providers existing at the same architectural layer (as for example two Internet Protocol (IP) or two Optical Transport Network (OTN) domains). A vertical SLA is an agreement between two individual providers at two different architectural layers (for instance, between an optical network and the core MPLS network). A service has to be defined without ambiguity utilizing Service Level Specifications (SLS) and three information types must be described: i) The QoX metrics as well as their corresponding thresholds; ii) A method of service performance measurement; iii) Service schedule.
In this work we present preliminary simulation results that enable the development of a generic methodology for SLA modeling and establishment that will lead to a win-win situation for all involved actors. As an example, we put special attention in the benefits obtained by Optical Networks operators.
Contains: runnable example instructions
Over the past decades, various methods, techniques, and tools for modeling and evaluating performance properties of software systems have been proposed covering the entire software life cycle. However, the application of performance engineering approaches to solve a given user concern is still rather challenging and requires expert knowledge and experience. There are no recipes on how to select, configure, and execute suitable methods, tools, and techniques allowing to address the user concerns. In this paper, we describe our vision of Declarative Performance Engineering (DPE), which aims to decouple the description of the user concerns to be solved (performance questions and goals) from the task of selecting and applying a specific solution approach. The strict separation of " what " versus " how " enables the development of different techniques and algorithms to automatically select and apply a suitable approach for a given scenario. The goal is to hide complexity from the user by allowing users to express their concerns and goals without requiring any knowledge about performance engineering techniques. Towards realizing the DPE vision, we discuss the different requirements and propose a reference architecture for implementing and integrating respective methods, algorithms, and tooling.
Continuous monitoring of software systems under production workload provides valuable data about application runtime behavior and usage. An adaptive monitoring infrastructure allows controlling, for instance, the overhead as well as the granularity and quality of collected data at runtime. Focusing on application-level monitoring, this paper presents the DProf approach which allows changing the instrumentation of software operations in monitored distributed applications at runtime. It simulates the process human testers employ-monitoring only such parts of an application that cause problems. DProf uses performance objectives specified in service level agreements (SLAs), along with call tree information, to detect and localize problems in application performance. As a proof-of-concept, DProf was used for adaptive monitoring of a sample distributed application.
Kieker is an extensible framework for monitoring and analyzing the runtime behavior of concurrent or distributed software systems. It provides measurement probes for application performance monitoring and control-flow tracing. Analysis plugins extract and visualize architectural models, augmented by quantitative observations. Configurable readers and writers allow Kieker to be used for online and offline analysis. This paper reviews the Kieker framework focusing on its features, its provided extension points for custom components, as well the imposed monitoring overhead.
With the introduction of services, software systems have become more flexible as new services can easily be composed from existing ones. Service composition frameworks offer corresponding functionality and hide the complexity of the underlying technologies from their users. However, possibilities for anticipating quality properties of com- posed services before their actual operation are limited so far. While existing approaches for model-based software quality prediction can be used by service composers for determining realizable Quality of Service (QoS) levels, integration of such techniques into composition frameworks is still missing. As a result, high effort and expert knowledge is required to build the system models required for prediction. In this paper, we present a novel service composition process that includes QoS prediction for composed services as an integral part. Furthermore, we describe how composition frameworks can be extended to support this process. With our approach, systematic consideration of service quality during the composition process is naturally achieved, without the need for de- tailed knowledge about the underlying prediction models. To evaluate our work and validate its applicability in different domains, we have integrated QoS prediction support according to our process in two com- position frameworks -- a large-scale SLA management framework and a service mashup platform.
Cloud Computing provides a convenient means of remote on-demand and pay-per-use access to computing resources. However, its ad hoc management of quality-of-service and SLA poses significant challenges to the performance, dependability and costs of online cloud services. The paper precisely addresses this issue and makes a threefold contribution. First, it introduces a new cloud model, the SLAaaS (SLA aware Service) model. SLAaaS enables a systematic integration of QoS levels and SLA into the cloud. It is orthogonal to other cloud models such as SaaS or PaaS, and may apply to any of them. Second, the paper introduces CSLA, a novel language to describe QoS-oriented SLA associated with cloud services. Third, the paper presents a control theoretic approach to provide performance, dependability and cost guarantees for online cloud services, with time-varying workloads. The proposed approach is validated through case studies and extensive experiments with online services hosted in clouds such as Amazon EC2. The case studies illustrate SLA guarantees for various services such as a MapReduce service, a cluster-based multi-tier e-commerce service, and a low-level locking service.
Performance predictions early in the software development process can help to detect problems before resources have been spent on implementation. The Palladio Component Model (PCM) is an example of a mature domain-specific modeling language for component-based systems enabling performance predictions at design time. PCM provides several alternative model solution methods based on analytical and simulation techniques. However, existing solution methods suffer from scalability issues and provide limited flexibility in trading-off between results accuracy and analysis overhead. Queueing Petri Nets (QPNs) are a general-purpose modeling formalism, at a lower level of abstraction, for which efficient and mature simulation-based solution techniques are available. This paper contributes a formal mapping from PCM to QPN models, implemented by means of an automated model-to-model transformation as part of a new PCM solution method based on simulation of QPNs. The limitations of the mapping and the accuracy and overhead of the new solution method compared to existing methods are evaluated in detail in the context of five case studies of different size and complexity. The new solution method proved to provide good accuracy with solution overhead up to 20 times lower compared to PCM's reference solver.
We describe a novel framework for specifying and monitoring Service Level Agreements (SLA) for Web Services. SLA monitoring and enforcement become increasingly important in a Web Service environment where enterprise applications and services rely on services that may be subscribed dynamically and on-demand. For economic and practical reasons, we want an automated provisioning process for both the service itself as well as the SLA managment system that measures and monitors the QoS parameters, checks the agreed-upon service levels, and reports violations to the authorized parties involved in the SLA management process. Our approach to these issues is presented in this paper. The Web Service Level Agreement (WSLA) framework is targeted at defining and monitoring SLAs for Web Services. Although WSLA has been designed for a Web Services environment, it is applicable as well to any inter-domain management scenario, such as business process and service management, or the management of networks, systems and applications in general. The WSLA framework consists of a flexible and extensible language based on XML Schema and a runtime architecture comprising several SLA monitoring services, which may be outsourced to third parties to ensure a maximum of objectivity. WSLA enables service customers and providers to unambiguously define a wide variety of SLAs, specify the SLA parameters and the way they are measured, and relate them to managed resource instrumentations. Upon receipt of an SLA specification, the WSLA monitoring services are automatically configured to enforce the SLA. An implementation of the WSLA framework, termed SLA Compliance Monitor, is publicly available as part of the IBM Web Services Toolkit.
The potential of communication networks and middleware to enable the composition of services across organizational boundaries remains incompletely realized. In this paper, we argue that this is in part due to outsourcing risks and describe the possible contribution of Service-Level Agreements (SLAs) to mitigating these risks. For SLAs to be effective, it should be difficult to disregard their original provisions in the event of a dispute between the parties. Properties of understandability, precision, and monitorability ensure that the original intent of an SLA can be recovered and compared to trustworthy accounts of service behavior to resolve disputes fairly and without ambiguity. We describe the design and evaluation of a domain-specific language for SLAs that tend to exhibit these properties and discuss the impact of monitorability requirements on service-provision practices.
Performance is a pervasive quality of software systems; everything affects it, from the software itself to all underlying layers, such as operating system, middleware, hardware, communication networks, etc. Software Performance Engineering encompasses efforts to describe and improve performance, with two distinct approaches: an early-cycle predictive modelbased approach, and a late-cycle measurement-based approach. Current progress and future trends within these two approaches are described, with a tendency (and a need) for them to converge, in order to cover the entire development cycle.
Providing users with Quality of Service (QoS) guarantees and the prevention of performance problems are challenging tasks for software systems. Architectural performance models can be applied to explore performance properties of a software system at design time and run time. At design time, architectural performance models support reasoning on effects of design decisions. At run time, they enable automatic reconfigurations by reasoning on the effects of changing user behavior. In this paper, we present a framework for the extraction of architectural performance models based on monitoring log files generalizing over the targeted architectural modeling language. Using the presented framework, the creation of a performance model extraction tool for a specific modeling formalism requires only the implementation of a key set of object creation routines specific to the formalism. Our framework integrates them with extraction techniques that apply to many architectural performance models, e.g., resource demand estimation techniques. This lowers the effort to implement performance model extraction tools tremendously through a high level of reuse. We evaluate our framework presenting builders for the Descartes Modeling Language (DML) and the Palladio Component Model(PCM). For the extracted models we compare simulation results with measurements receiving accurate results.
Results of a five-year research project and several industrial collaborations have produced tools that model the individual effects and complex dynamic interactions between an IT system's application workload and resource contention at multiple levels in the execution environment. An evaluation shows significant resource efficiency gains without sacrificing the performance specified in service-level agreements.
Over the past few decades, many performance modeling formalisms and prediction techniques for software architectures have been developed in the performance engineering community. However, using a performance model to predict the performance of a software system normally requires extensive experience with the respective modeling formalism and involves a number of complex and time consuming manual steps. In this paper, we propose a generic declarative interface to performance prediction techniques to simplify and automate the process of using architecture-level software performance models for performance analysis. The proposed Descartes Query Language (DQL) is a language to express the demanded performance metrics for prediction as well as the goals and constraints of the specific prediction scenario. It reduces the manual effort and learning curve in working with performance models by a unified interface independent of the employed modeling formalism. We evaluate the applicability and benefits of the proposed approach in the context of several representative case studies.
Quality of service (QoS) can be a critical element for achieving the business goals of a service provider, for the acceptance of a service by the user, or for guaranteeing service characteristics in a composition of services, where a service is defined as either a software or a software-support (i.e., infrastructural) service which is available on any type of network or electronic channel. The goal of this article is to compare the approaches to QoS description in the literature, where several models and metamodels are included. consider a large spectrum of models and metamodels to describe service quality, ranging from ontological approaches to define quality measures, metrics, and dimensions, to metamodels enabling the specification of quality-based service requirements and capabilities as well as of SLAs (Service-Level Agreements) and SLA templates for service provisioning. Our survey is performed by inspecting the characteristics of the available approaches to reveal which are the consolidated ones and which are the ones specific to given aspects and to analyze where the need for further research and investigation lies. The approaches here illustrated have been selected based on a systematic review of conference proceedings and journals spanning various research areas in computer science and engineering, including: distributed, information, and telecommunication systems, networks and security, and service-oriented and grid computing.
In this work we propose GXLA, a language for the specification of Service Level Agreements (SLA). GXLA represents the implementation of the Generalized Service Level Agreement (GSLA) information model we proposed in a previous work. It supports multi-party service relationships through a role-based mechanism. It is intended to catch up the complex nature of service interactivity in the broader range of SLA modeling of all sorts of IT business relationships. GXLA is defined as an XML schema which provides a common ground between the entities in order to automate the configuration. GXLA can be used by service providers, service customers, and third parties in order to configure their respective IT systems. Each party can use its own independent SLA interpretation and deployment technique to enforce the role it has to play in the contract. An illustrative VoIP service negotiation shows how GXLA is used for automating the process of SLA negotiation and deployment.
This paper describes SLA*, a domain-independent syntax for machine-readable Service Level Agreements (SLAs) and SLA templates. Historically, SLA* was developed as a generalisation and refinement of the web-service specific XML standards: WS-Agreement, WSLA, and WSDL. Instead of web-services, however, SLA* deals with services in general, and instead of XML, it is language independent. SLA* provides a specification of SLA(T) content at a fine-grained level of detail, which is both richly expressive and inherently extensible: supporting controlled customisation to arbitrary domain-specific requirements. The model was developed as part of the FP7 ICT Integrated Project SLA@SOI, and has been applied to a range of industrial use-cases, including; ERP hosting, Enterprise IT, live-media streaming and health-care provision. At the time of writing, the abstract syntax has been realised in concrete form as a Java API, XML-Schema, and BNF Grammar.
Service-Oriented Architectures (SOA) represent an architectural shift for building business applications based on loosely-coupled
services. In a multi-layered SOA environment the exact conditions under which services are to be delivered can be formally
specified by Service Level Agreements (SLAs). However, typical SLAs are just specified at the customer-level and do not allow
service providers to manage their IT stack accordingly as they have no insight on how customer-level SLAs translate to metrics
or parameters at the various layers of the IT stack. In this paper we present a technical architecture for a multi-level SLA
management framework. We discuss the fundamental components and interfaces in this architecture and explain the developed
integrated framework. Furthermore, we show results from a qualitative evaluation of the framework in the context of an open
reference case.
Humans should not have to grok XML
- T Parr
T. Parr. Humans should not have to grok XML.
http://www.ibm.com/developerworks/library/x-sbxml/
x-sbxml-pdf.pdf, 2001.
SLA-driven adaptive monitoring of distributed applications for performance problem localization
- D Okanović
- A Van Hoorn
- Z Konjović
- M Vidaković
Kieker4DQL: Declarative Performance Measurement
- M Blohm
- M Pahlberg
- S Vogel
- J Walter
- D Okanovic
M. Blohm, M. Pahlberg, S. Vogel, J. Walter, and
D. Okanovic. Kieker4DQL: Declarative Performance
Measurement. In Proceedings of the 2016 Symposium on
Software Performance (SSP), November 2016.
- H Ludwig
- A Keller
- A Dan
- R P King
- R Franck
H. Ludwig, A. Keller, A. Dan, R. P. King, and R. Franck.
Web Service Level Agreement (WSLA) Language
Specification, v1.0, Jan. 2003.
Automated transformation of component-based software architecture models to Queueing Petri Nets
- P Meier
- S Kounev
- H Koziolek
P. Meier, S. Kounev, and H. Koziolek. Automated
transformation of component-based software architecture
models to Queueing Petri Nets. In 19th IEEE/ACM Int.
Symposium on Modeling, Analysis and Simulation of
Computer and Telecommunication Systems, 2011.
- R Sturm
- W Morris
- M Jander
R. Sturm, W. Morris, and M. Jander. Foundations of
Service Level Management. SAMS, 2000.