Dominic Battré’s research while affiliated with Technische Universität Berlin and other places

Modern Infrastructure-as-a-Service (IaaS) clouds offer an unprecedented flexibility and elasticity in terms of resource provisioning through the use of hardware virtualization. However, for the cloud customer, this virtualization also introduces an opaqueness which imposes serious obstacles for data-intensive distributed applications. In particular, the lack of network topology information, i.e. information on how the rented virtual machines are physically interconnected, can easily cause network bottlenecks as common techniques to exploit data locality cannot be applied. In this paper we study to what extent the underlying network topology of virtual machines inside an IaaS cloud can be inferred based on end-to-end measurements. Therefore, we experimentally evaluate the impact of hardware virtualization on the measurable link characteristics packet loss and delay using the popular open source hypervisors KVM and XEN. Afterwards, we compare the accuracy of different topology inference approaches and propose an extension to improve the inference accuracy for typical network structures in datacenters. We found that common assumptions for end-to-end measurements do not hold in presence of virtualization and that RTT-based measurements in Para virtualized environments lead to the most accurate inference results.

Infrastructure as a Service (IaaS) clouds are gaining increasing popularity as a platform for distributed computations. The virtualization layers of those clouds offer new possibilities for rapid resource provisioning, but also hide aspects of the underlying IT infrastructure which have often been exploited in classic cluster environments. One of those hidden aspects is the network topology, i.e. the way the rented virtual machines are physically interconnected inside the cloud. We propose an approach to infer the network topology connecting a set of virtual machines in IaaS clouds and exploit it for data-intensive distributed applications. Our inference approach relies on delay-based end-to-end measurements and can be combined with traditional IP-level topology information, if available. We evaluate the inference accuracy using the popular hyper visors KVM as well as XEN and highlight possible performance gains for distributed applications.

In recent years, several frameworks have been introduced to facilitate massively-parallel data processing on shared-nothing architectures like compute clouds. While these frameworks generally offer good support in terms of task deployment and fault tolerance, they only provide poor assistance in finding reasonable degrees of parallelization for the tasks to be executed. However, as billing models of clouds enable the utilization of many resources for a short period of time for the same cost as utilizing few resources for a long time, proper levels of parallelization are crucial to achieve short processing times while maintaining good resource utilization and therefore good cost efficiency. In this paper, we present and evaluate a solution for detecting CPU and I/O bottlenecks in parallel DAG-based data flow programs assuming capacity constrained communication channels. The detection of bottlenecks represents an important foundation for manually or automatically scaling out and tuning parallel data flow programs in order to increase performance and cost efficiency.

Abstract The quality of the underlying network model constitutes a pivotal basis for simulating the performance of distributed systems. We present an efficient network model based on measured data and exhibiting realistic network behavior by providing location- and load-dependent latency, jitter and packet loss samples for connections between arbitrary hosts world-wide, which makes it especially well suited for the simulation and evaluation of P2P streaming systems. The model features only linear computation and memory requirements with respect to the number of hosts. It benefits from the finding that network delay/jitter/packet loss can be conveniently divided into a location- and distance-dependent but (from the user's perspective) stateless backbone part and a stateful local Internet access part. The backbone characteristics are modeled using Global Network Positioning (GNP) by embedding all hosts into an n-dimensional delay space and interpolating delays from their distance, before adding regional jitter. The local characteristics are obtained from two queues modeling the send buffers within the local up-link, e.g. DSL modem, and down-link, e.g. broadband remote access server (BRAS). The results show that the simulated delay behavior under varying load much closer approximates real network delay measurements than other models without topology modeling, such as OverSim's Simple-Underlay while still requiring only moderate resources. Both the GNP and the queuing components were integrated into the OMNeT++/INET Framework so that they can be easily used in P2P models.

The Web Services Agreement specification defines a normative language to formulate Service Level Agreements and a basic protocol to expose service-level descriptions, validate service-level requests, and come to an agreement. This protocol, often called ''take-it-or-leave-it'', allows a service provider and a service consumer to decide whether to accept or reject a service offer. Although this approach is sufficient for a number of use cases, others exist with requirements for multi-step negotiation or the adaptation of an existing agreement. In this paper, we describe the Web Services Agreement Negotiation protocol, a proposal by the Open Grid Forum to extend the existing specification. This proposal is the result of combining various research activities that have been conducted to define protocols for negotiating service levels or to supersede the existing ''take-it-or-leave-it'' protocol. The main characteristics of this proposal are the multi-round negotiation capability, re-negotiation capability, and compliance with the original specification.

Large-scale data analysis applications require processing and analyzing of Terabytes or even Petabytes of data, particularly in the areas of web analysis or scientific data management. This trend has been discussed as "web-scale data management" in a panel at VLDB 2009. Formerly, parallel data processing was the domain of parallel database systems. Today, novel requirements like scaling out to thousands of machines, improved fault-tolerance, and schema free processing have made a case for new approaches.

We present a parallel data processor centered around a programming model of so called Parallelization Contracts (PACTs) and the scalable parallel execution engine Nephele [18]. The PACT programming model is a generalization of the well-known map/reduce programming model, extending it with further second-order functions, as well as with Output Contracts that give guarantees about the behavior of a function. We describe methods to transform a PACT program into a data flow for Nephele, which executes its sequential building blocks in parallel and deals with communication, synchronization and fault tolerance. Our definition of PACTs allows to apply several types of optimizations on the data flow during the transformation. The system as a whole is designed to be as generic as (and compatible to) map/reduce systems, while overcoming several of their major weaknesses: 1) The functions map and reduce alone are not sufficient to express many data processing tasks both naturally and efficiently. 2) Map/reduce ties a program to a single fixed execution strategy, which is robust but highly suboptimal for many tasks. 3) Map/reduce makes no assumptions about the behavior of the functions. Hence, it offers only very limited optimization opportunities. With a set of examples and experiments, we illustrate how our system is able to naturally represent and efficiently execute several tasks that do not fit the map/reduce model well.

WS-Agreement describes a protocol and structure for creating and representing service level agreements. In order to remain domain independent, the authors of the WS-Agreement specification have provided many extension points for domain specific content. This creates high degrees of freedoms for programmers to implement the specification. Many attempts to do this have been made in the past. In this paper, we explain what we have learned from our own and other projects’ attempts of implementing WS-Agreement. The paper presents a set of guidelines how the features of WS-Agreement can be used in a sound way that allows transferring large parts of the WS-Agreement logic into a generic and domain-independent WS-Agreement framework.

D-Grid is a German national grid initiative founded by the German Federal Ministry of Education and Research. It provides a frame for the collaboration of researchers across Germany and offers access to distributed services and resources. The SLA4D-Grid project will enhance D-Grid by creating a service level agreement (SLA) layer which is transparently integrated into D-Grid and accessible for the D-Grid community.