Conference PaperPDF Available

Web services agreement-based resource negotiation in UNICORE


Abstract and Figures

Service Level Agreements provide the foundation to negotia te for a distinct Quality of Service level between the provider and the consumer of a service. Since the Grid com munity is adopting concepts of Service-Oriented Architectures and Web Services are capturing their space wi thin the Grid landscape, resource management within Grids increasingly evolves towards the management of resou rces represented as services. To make allowance for this the Global Grid Forum develops the Web Services Agre ement specification to support standardised cre- ation and negotiation of guarantees related to services. Th is paper illustrates the integration of a Web Services Agreement-based resource management framework into the UN ICORE Grid system, a development motivated by the system's transition towards a service-oriented Grid an d the limitations of the current solution.
Content may be subject to copyright.
Web Services Agreement-based
Resource Negotiation in UNICORE
M. Riedel, V. Sander, P. Wieder
Central Institute of Applied Mathematics
Research Centre J¨ulich
D - 52425 J¨ulich, Germany
J. Shan
Nat. High Performance Computing Center
Univ. of Science and Technology of China
230027 Hefei, China
Service Level Agreements provide the foundation to negotiate for a distinct Quality of Service level between
the provider and the consumer of a service. Since the Grid community is adopting concepts of Service-Oriented
Architecturesand Web Services are capturing their space within the Grid landscape, resource management within
Grids increasingly evolves towards the management of resources represented as services. To make allowance
for this the Global Grid Forum develops the Web Services Agreement specification to support standardised cre-
ation and negotiation of guarantees related to services. This paper illustrates the integration of a Web Services
Agreement-based resource management framework into the UNICORE Grid system, a development motivated by
the system’s transition towards a service-oriented Grid and the limitations of the current solution.
Keywords. SLA, SOA, UNICORE, UniGrids, Web Services, WS-Agreement
1 Introduction
At the present time the Grid community seems to have a clear notion of what properties and functions
future Grid systems should comprise [12]. Concerning the underlying technology and the architectural
model of forthcoming Grids, there is a trend, which propagates the usage of Web Services and the
adoption of Service-Oriented Architecture (SOA) concepts and models. The development towards the
integration of Grids, Web Services and SOA has been well-founded by the introduction of the Open
Grid Services Architecture (OGSA, [10]), an initiative advanced by the respective Global Grid Forum
working group.
Currently scientists, researchers and vendors design and develop multiple Grid architectures and sys-
tems based on the Grid Service paradigm and with OGSA concepts in mind. One of these systems is
UNICORE [5], initially designed as a Uniform Interface to Computing Resources. With the expansion
and abstraction of the term resource beyond computing and data, the focus of the UNICORE devel-
opment changed towards service-orientation. Projects and the community make efforts to enable this
transition [14], an objective supported by UNICORE’smodels and architecture which are designed in a
service-oriented way as analyses reveal [17, 16].
The work reported in this paper concentrates on one of the crucial aspects in the current Grid devel-
opment: resource management. The requirements for resource management services are described by a
group of independent experts convened by the European Commission who envisage “flexible, dynamic,
This work is partially funded by the UniGrids project under EC grant, duration: July 2004 - June 2006.
reconfigurable resources available on demand and to the level required for the application and / or end-
user” [12] for the next generation of Grids. Our work integrates appropriate models and Grid Services
into the UNICORE framework to provide resource management with properties and Quality of Service
capabilities as mentioned above.
Following the introduction the scene is set in Section 2 where we present UNICORE’s current re-
source management model and present the need for enhanced mechanisms to satisfy both end-users
and resource providers. Section 3 then introduces the underlying technology of the chosen solution,
namely Service Level Agreements, the Web Services Agreement specification and the Web Services
Resource Framework (references are provided in the respective section). How these technologies are
used to provide the requested resource management capabilities is outlined in Section 4, followed by
an analysis of the chosen solution in Section 5. The paper concludes with an outlook on the potential
of the WS-Agreement-based resource negotiation in UNICORE, especially with respect to multi-phase
2 Resource Management in UNICORE
With reference to the essential resource management properties and capabilities introduced in
Section 1, we outline more precisely the corresponding demands made on a state-of-the-art system
to manage resources within a Grid:
Dynamic information End-users (or the respective entities acting on their behalf) must operate
on up-to-date resource information in order to make a reasonable resource selection.
QoS guarantees End-users and service providers require Quality of Service (QoS) guarantees
when participating in the Grid-wide resource sharing.
Resource autonomy Service providers demand autonomous control of the resources they share
in a Grid environment. This is achieved by policy-based resource provisioning.
Economic models Economic based resource management [6] will play an important role in Grid
Computing and at least the necessary foundations should be integrated into the design of future
resource management systems.
These demands define the basis for the design and realisation of advanced UNICORE resource manage-
ment models, protocols and services.
With regard to the provision and usage of resources UNICORE offers two major models: resource
virtualisation and job abstraction [8]. In UNICORE a set of resources with a uniform user mapping
and direct access to each other is called a Virtual Site (Vsite). As a rule these resources are controlled
by a single resource management system. Furthermore, a UNICORE Site (Usite) represents a single
access point to a set of Vsites (at least one), making it the building block of a Virtual Organization of
UNICORE controlled resources. In technical terms a Usite is accessible via a unique address and port
generally virtualising the resources of one institution.
The job model distinguishes between abstract and incarnated representations of a UNICORE job. The
former defines a job description which contains no site-specific information and is processable by every
Usite, while the latter job representation comprises everything to actually execute a job on resource level.
In-between (arriving at its target Vsite) the abstract job is “translated”, a process called incarnation. For
incarnation as well as for the composition of jobs and workflowsresource-specific information is needed.
This information is currently maintained per Vsite in a static text file, the Incarnation Database (IDB).
It contains inter alia information about capability resources, such as software distributions, and about
capacity resources, such as memory or nodes available.
During initialisation a UNICORE server component named Network Job Supervisor (NJS, please re-
fer to Section 4) reads the information contained in the IDB into memory and keeps it unchanged during
its lifetime. This implies that changes of the resource situation during runtime are not reflected in the
resource information of the UNICORE system. When connected to a Usite, the client-side application
submits a GetResourceDescription request to the NJS and the corresponding handler DoGet-
Resources within the NJS reads the information from memory and returns it back to the client-side
application. After obtaining the resource information, end-users or the respective client application can
make a match between their jobs’ requirements and the available resources. Finally, the end-users will
submit their jobs, which contain the specified resource requirements, to the Usite. Although this model
supports end-users to perform their job submission, it only provides a static resource view of Vsites and
can thus not meet the requirement of providing dynamic resource information. In addition, the current
UNICORE system does neither allow resource users and providers to negotiate on QoS guarantees nor
does it include the means to manage resources based on economic models. And even though UNICORE
fulfils the “Resource autonomy” requirement with respect to the autonomous control, no framework
exists e.g. to enforce a certain policy on a single user. This gap analysis motivated the work described
here and led to the design and implementation of a WS-Agreement based resource negotiation model to
meet the requirements mentioned above.
3 Service Level Agreements for Resource Negotiation
As noted in the previous section the management of QoS guarantees is a functional requirement future
Grid systems have to satisfy. In a service-oriented context QoS management refers to the negotiation,
enforcement and monitoring of contracts between a service requestor and a service provider. The ob-
jective of such contract negotiations is to agree upon specific capabilities provided by a service, thus
to define an implicit level of service. This is of major importance during the selection of a service
provider since its actual capabilities might depend on the resource situation at the requested time of
service. By modeling the implicit level of service explicitly as a Service Level Agreement (SLA) we
formalize the relation between service requestor and provider. This leads to the problem of formally
defining agreement terms and a standard way of negotiating them.
While several SLA languages and protocols, such as the Web Service Level Agreement (WSLA)
language [13] or theBusiness Process Execution Language (BPEL, [1]), canbe used tosupport resource
negotiation in Grids, we have chosen the development of the Grid Resource Allocation Agreement
Protocol Working Group (GRAAP-WG) of the Global Grid Forum. Their proposed WS-Agreement
specification [2] defines a language and a protocol for advertising the capabilities of service providers
and creating SLAs based on agreement templates and offers. The well defined agreement structure in
combination with the straightforward negotiation process based on templates and offers is the reason
why we preferred WS-Agreement to WSLA or BPEL. The agreement creation process starts with a
WS-Resource Properties
WS-Resource Instance
Service Description Terms
Guarantee Terms
Agreement Offer
Service Description Terms
Guarantee Terms
Agreement Template
Service Description Terms
Guarantee Terms
fill out convert
Figure 1. Transforming a WS-Agreement into WS-ResourceProperties
pre-defined agreement template specifying customizable aspects of an agreement, as there are Name,
Context,Terms and Constraints (see Fig. 1). While the Constraints enforce rules that must be observed
creating an agreement, the Context contains information about the participants in the agreement. The
Terms section is split in Service Description Terms (SDT) and Guarantee Terms (GT). While SDTs, also
called functional terms, describe the essential characteristics of a service, the GTs define assurances on
service quality associated with the service described by the SDTs.
The service provider advertises the types of agreement offers it is willing to accept by means of such
agreement templates. A service requestor then fills in a template and sends it to the provider, a process
resulting in a concrete agreement offer, which is accepted or rejected dependent on the actual resource
situation of the service provider. Hence, this simple template and offer mechanism provides a dynamic
resource negotiation framework to be used in Grid systems and especially in UNICORE.
If an offer is accepted the service provider creates a stateful WS-Resource [9]wherein the state repre-
sents the structure of the agreement. The agreement offer is then converted into WS-ResourceProperties
[11] as Fig. 1 shows. Thus, the WS-Agreement specification relies on OASIS’ Web Services Resource
Framework (WSRF, [7]) that includes the WS-ResourceProperties specification. WS-ResourceProperties
defines the representation of properties inside a WS-Resource as well as a standard set of message ex-
change patterns that allow a service requestor to query resource properties. WS-Agreement uses this
specification to query agreement properties, especially SLA terms, at service runtime.
4 Dynamic Resource Management Architecture
Based on the definition of the resource management requirements and the related UNICORE gap
analysis in Section 2, we now evolve the integration of the WS-Agreement framework as described in
Section 3 into the UNICORE architecture.
4.1 Basic UNICORE architecture
(EPR 1)
Target System
Network Job Supervisor
<EPR 1>
Target system tier
Hosting Environment
Hosting Environment
Server tier
Client tier
EPR - Endpoint Reference
DB - Database containing
Resource Properties,
policies, ...
Proprietary UNICORE
Resource information
Figure 2. UNICORE components (light grey) and architecture enhancements (dark grey)
As shown in Fig. 2, UNICORE introduces a three tier Grid architecture consisting of client, server
and target system tier, an implementation of which is realized entirely in Java (except for the Target
System Interface (TSI) which is implemented in Perl). This paragraph introduces the basic concepts
and components (those in light grey) relevant for resource management, for an extensive insight please
refer to [8]. Section 4.2 then pictures the enhancements implemented to realise the dynamic resource
management architecture, while Section 4.3 depicts selected implementation details. The client tier is
either represented by an application making use of the UNICORE client API or by the UNICORE Client
which provides a graphical user interface to exploit the entire functionality offered by the server tier.
This is achieved by sending and receiving abstract representations of UNICORE jobs as introduced in
Section 2. In addition to resource information this job instantiation contains platform and site neutral
descriptions of computational and data related tasks as well as work flow specifications along with user
and security information.
The Gateway is the secure entry point to a Usite, which authenticates user requests and transfers
them to the Network Job Supervisor. The NJS translates the abstract job into a target system specific
one, making use of the Incarnation Database. Furthermore the NJS contains a work flow engine which,
among other tasks, performs pre- and post-staging of computational data and authorises the user. The
Gateway and NJS execute typically on dedicated secure systems behind a firewall.
UNICORE’s communication endpoint is the Target System Interface (TSI), a stateless daemon exe-
cuting on the target system. It interfaces with the local resource manager represented either by a batch
system like LoadLeveler, an operating system like Linux (using fork) or an entity capable of interoper-
ating with other Grid systems like Globus [18]. In a typical setup one TSI and one NJS form a Vsite
(see Section 2).
4.2 Architecture Extensions
Extending the three tier Grid architecture of UNICORE, Fig. 2 depicts several components in dark
grey that have been added to the basic architecture. These extensions realise a UNICORE-specific
implementation of the WS-Agreement framework, which itself defines a two-layered service model [2],
namely service layer and agreement layer.
The service layer is integrated inside the target system tier, represented by an Information Sensor (IS)
which exploits dynamic resource-specific information and therefore fulfils the first demand specified
in Section 2. To enforce user-specific resource management and support the provider’s autonomy we
implemented a filter that makes use of policies stored in a database (DB). This filter allows e.g. to
restrain information delivered to certain users or restrict access to certain parts of the resource.
The agreement layer consists of two components, the Agreement Manager on the target system tier
and the Agreement Consumer on the server tier. These components enable an UNICORE Client to sub-
mit a resource request and negotiate the terms of resource usage. This is accomplished by forwarding
any resource information request from the UNICORE Client via the NJS directly to the Agreement
Consumer, which in turn enquires for resource information provided by the Agreement Manager. The
Agreement Manager responds by using the service layer to get the actual resource information from
the IS. In particular, the Agreement Manager generates a WS-Agreement compliant template containing
the resource information which is transferred back to the Agreement Consumer. The requested template
contains the current, filtered information about a UNICORE resource represented as SDTs. For example
in the case of a computing resource the template comprises, among others SDTs, representing informa-
tion like nodes, processors or memory. The subsequent step in this negotiation process requires the
Agreement Consumer to fill in the template to make a concrete agreement offer to the Agreement Man-
ager which then is in charge of creating and managing the agreement. The completion of the template
is supported by the UNICORE Client. This agreement offer constitutes a concrete resource request that
leads to the creation of an UNICORE resource agreement instance if the Agreement Manager accepts
this offer. This UNICORE resource agreement instance provides access to dynamic resource informa-
tion through the UNICORE Client.
4.3 Prototype Implementation
The implementation of the afore outlined resource negotiation architecture mainly complies two
tasks: the modification of the original UNICORE software and the implementation of the current WS-
Agreement specification. Since the modifications of the UNICORE software imply primarily protocol
extensions, we will concentrate on the integration of a WSRF-compliant hosting environment and WS-
Agreement-related Grid Services into UNICORE as shown in Fig. 3.
UNICORE Site (Usite)
WSRF hosting environment
Agreement status
Grid Service
Factor y Service
EPR creates
Figure 3. Services implementation details
WSRF uses the factory pattern [7] to create WS-Resources that are capable of storing the resource’s
state. We realise this pattern as an Autonomic Factory Service [15] that implements the
AgreementFactory portType [2] and provides the createAgreement and
GetResourcePropertyoperations to the Agreement Consumer. While GetResourceProper-
ty provides access to the WS-ResourceProperties which are dynamically updated by the IS, the
createAgreement operation generates a UNICORE ResourceAgreement Instance and an Endpoint
Reference (EPR [4]) that can be used to access this WS-Resource through the UNICORE ResourceA-
greement Grid Service. In consequence the agreement state is represented by the UNICORE ResourceA-
greement Instance and manipulated via the UNICORE ResourceAgreement Grid Service that imple-
ments the Agreement portType [2]. This portType defines a Terminate,
GetResourceProperty and other domain-specific operations, like for instance
QueryResourceProperties [11]. Furthermore, the portType exposes WS-ResourceProperties,
namely Name, Context, and Terms which are converted from the Agreement offer (see Section 3) into
the WS-ResourceProperties of the UNICORE ResourceAgreement Instance. These architecture en-
hancements lay the foundations for dynamic resource management in UNICORE and, among other
functionality, enable user-specific resource management and support the provider’s autonomy through
site-specific policies managed by the Autonomic Factory Service, as the following sections reveal.
5 Architecture Evaluation
As determined in Section 2 the purpose of the work reported here is to enhance UNICORE’s resource
management capabilities with functions to advertise dynamic resource information, enable the negoti-
ation of QoS guarantees, enable economic resource management and retain resource autonomy. These
functions and their impact on UNICORE will be briefly described here.
Using up-to-date resource information UNICORE users are now in the position to make a suitable
resource selection based onthe actual resource situation on the target system. Although this still implies
jobs to be scheduled manually by the user, the transition from a static to a dynamic information service
lays the foundation for more advanced scheduling architectures as researched in projects like VIOLA
(Vertically Integrated Optical Testbed for Large Applications).
Moreover, our work provides the means for users or applications to demand a certain service level.
Currently this service level is negotiated within one step, but once the respective WS-Agreement Ne-
gotiation specification [3] is available, multi-step negotiation will be integrated into the UNICORE
Apart from carrying out an arbitrary number of steps to agree upon the modalities of resource usage,
economic resource management requires the possibility to reserve resources in advance. This capability
can be easily realised with the described architecture by including additional service description terms
which represent the reservation and are defined inside the agreement.
The introduction of site-specific policies overcomes the limitations of the current situation in which
all users have the same rights and therefore strengthens the autonomy of resource providers. As an extra
benefit, since all service description terms of an agreement are exposed as WS-Resource properties, a
user or application can monitor agreement-compliance at the run-time of the service.
6 Conclusion and Future Work
An evaluation of the UNICORE framework showed that its resource management capacity lacks cer-
tain functionality with respect to the requirements users and experts have. To fulfil these requirements
we designed and implemented a dynamic resource management architecture to extend the UNICORE
system. This architecture is based on the WS-Agreement specification and the WS Resource Frame-
work and enables users to control and to manage dynamic resource allocation through agreements.
Furthermore the proposed solution is extensible and therefore allows the integration of various different
resources and their respective QoS parameters.
The previous section shows not only that our solution improves usage and administration of the
UNICORE system, but it also provides the basis to realise functions like advance reservation, multi-step
negotiation, related agreements, automatic scheduling and support for dynamic workflows. Hence we
will exploit the potential of this work and develop additional services and components to make,as stated
in the introduction, “flexible, dynamic, reconfigurable resources available on demand”.
[1] T. Andrews et al. Business Process Execution Language for Web Services, 2003. Version 1.1.
[2] A. Andrieux et al. Web Services Agreement Specification, 2004. Version 1.1, draft 20.
[3] A. Andrieux et al. Web Services Agreement Negotiation Specification, 2005.
[4] D. Box, F. Curbera, et al. Web Services Addressing (WS-Addressing), 2004.
[5] D. Breuer, D. Erwin, D. Mallmann, R. Menday, M. Romberg, V. Sander, B. Schuller, and P. Wieder. Sci-
entific Computing with UNICORE. In D. Wolf, G. M¨unster, and M. Kremer, editors, Proc. of the NIC
Symposium 2004, volume 20, pages 429–440. John von Neumann Institute for Computing, 2004.
[6] R. Buyya and H. Stockinger. Economic Models for Management of Resources in Peer-to-Peer and Grid
Computing. In SPIE International Symposium on The Convergence of Information Technologies and Com-
munications (ITCom 2001), 2001.
[7] K. Czajkowski, D. Ferguson, I. Foster, J. Frey, S. Graham, I. Sedukhin, D. Snelling, S. Tuecke, and W. Vam-
benepe. The WebServices Resource Framework, 2004. Version 1.0.
[8] D. Erwin, editor. UNICORE Plus Final Report – Uniform Interface to Computing Resources. UNICORE
Forum e.V., 2003. ISBN 3-00-011592-7.
[9] I. Foster et al. Modeling Stateful Resources with Web Services, 2004.
[10] I. Foster, C. Kesselmann, J. M. Nick, and S. Tuecke. The Physiology of the Grid. In F. Berman, G. C. Fox,
and A. J. G. Hey, editors, Grid Computing, pages 217–249. John Wiley & Sons Ltd, 2003.
[11] S. Graham, J. Treadwell, et al. Web Services Resource Properties, 2004.
[12] K. Jeffery et al. Next Generation Grids 2 – Requirements and Options for European Grids Research 2005-
2010 and Beyond, July 2004.
[13] H. Ludwig, A. Keller, A. Dan, R. P. King, and R. Franck. Web Service Level Agreement Language Specifi-
cation, 2003. Version 1.0.
[14] R. Menday and P. Wieder. GRIP: The Evolution of UNICORE towards a Service-Oriented Grid. In Proc.
of the 3rd Cracow Grid Workshop (CGW’03), Oct. 27–29 2003.
[15] M. Riedel. Prospects and Realization of Flexible Service Offers in a Grid Environment. Technical Report
JUEL-4154, Research Centre J¨ulich, 2004.
[16] D. Snelling. UNICORE and the Open Grid Services Architecture. In F. Berman, G. C. Fox, and A. J. G.
Hey, editors, Grid Computing, pages 701–712. John Wiley & Sons Ltd, 2003.
[17] D. Snelling, S. van den Berghe, G. von Laszweski, P. Wieder, D. Breuer, J. MacLaren, D. Nicole, and H.-C.
Hoppe. A UNICORE Globus Interoperability Layer. Computing and Informatics, 21:399–411, 2002.
[18] P. Wieder and M. Rambadt. UNICORE – Globus: Interoperability of Grid Infrastructures. In Proc. of the
Cray User Group Summit 2002. Cray User Group, 2002.
... Exploiting the benefits of SLAs in distributed computing environments is on the research agenda for a decade now [29] and in the focus of our work almost since [114]. Our first work integrating SLAs into a scheduling framework was in the course of the German VIOLA project [26], an effort which led to the creation of the meta-scheduling service (MSS) [42]. ...
In state-of-the-art distributed computing infrastructures different kinds of resources are combined to offer complex services to customers. As of today, service-oriented middleware stacks are the work-horses to connect resources and their users, and to implement all functions needed to provide those services. Analysing the functionality of prominent middleware stacks, it becomes evident that common challenges, like scalability, manageability, efficiency, reliability, security, or complexity, exist, and that they constitute major research areas in information and communication technologies in general and distributed systems in particular. One core issue, touching all of the aforementioned challenges, is the question of how to distribute units of work in a distributed computing infrastructure, a task generally referred to as scheduling. Integrating a variety of resources and services while being compliant with well-defined business objectives makes the development of scheduling strategies and services a difficult venture, which, for service-oriented distributed computing infrastructures, translates to the assignment of services to activities over time aiming at the optimisation of multiple, potentially competing, quality-of-service criteria. Many concepts, methods, and tools for scheduling in distributed computing infrastructures exist, a majority of which being dedicated to provide algorithmic solutions and schedulers. We approach the problem from another angle and offer a more general answer to the question of ’how to design an automated scheduling process and an architecture supporting it’. Doing so, we take special care of the service-oriented nature of the systems we consider and of the integration of our solutions into IT service management processes. Our answer comprises a number of assets that form a comprehensive scheduling solution for distributed computing infrastructures. Based on a requirement analysis of application scenarios we provide a concept consisting of an automated scheduling process and the respective generic scheduling architecture supporting it. Process and architecture are based on four core models as there are a model to describe the activities to be executed, an information model to capture the capabilities of the infrastructure, a model to handle the life-cycle of service level agreements, which are the foundation for elaborated service management solutions, and a specific scheduling model capturing the specifics of state-of-the-art distributed systems. We deliver, in addition to concept and models, realisations of our solutions that demonstrate their applicability in different application scenarios spanning grid-like academic as well as financial service infrastructures. Last, but not least, we evaluate our scheduling model through simulations of artificial as well as realistic workload traces thus showing the feasibility of the approach and the implications of its usage. The work at hand therefore offers a blueprint for developers of scheduling solutions for state-of-the-art distributed computing infrastructures. It contributes essential building blocks to realise such solutions and provides an important step to integrate them into IT service management solutions.
... Such a Grid middleware manages the functionalities within a Grid or expose them to the user community. Enhancing Grid middleware with dynamic management capabilities as stated in [21] for the UNICORE Grid technology is one of the crucial aspects in Grid development. UNICORE evolves in various European-funded projects to a full-grown and well-tested Grid middleware [23], but lacks certain autonomic control of resources or jobs. ...
Full-text available
Recently, the Web Services Agreement work performed within the Global Grid Forum introduces a generic approach that, in principle, provides an abstraction layer that allows for the creation of distributed service man- agement approaches by the abstraction of managing service level agreements within autonomic resources. Such resources were modelled in a service-oriented way and comprise the functionality to create Multi-Protocol Label Switching tunnels as well as routing and signaling functionality for the establishment of Label Switched Paths. Self-management capabilities, such as self-creation, self-healing and self-monitoring that enable autonomic re- sources in Grids are described and thus lay the foundation for distributed network control planes in autonomic Grids and end-to-end management planes for emerging Autonomic Grid Networking.
... To satisfy the VIOLA meta-scheduling use case UNICORE needs to be at least extended to provide advance reservation capabilities. This can be realised by integrating WS-Agreement providers and consumers into UNICORE, as reported in [11], and has been implemented in the first phase of the MetaScheduling Service development as outlined in the following section. ...
Conference Paper
Full-text available
Designing, building and using Grids generally implies that the resources involved are highly dis-tributed, heterogeneous and managed by different organisations. These characteristics hamper the coordinated usage and management of resources within a Grid, which in turn motivates the devel-opment of Grid-speci c resource management and scheduling solutions. Please refer for instance to [8] which collects a large variety of contributions to research and development for this speci c topic. The use case of the work we present here requires a Grid that has exactly the properties listed above: The VIOLA Grid [15] comprises distributed resources of different types which are owned by different organisations and have to be managed in a coordinated fashion. Although the resources have different owners the aspects of authentication and authorisation are of secondary importance to this work since the underlying middleware, UNICORE, provides all necessary security means required to realise a Virtual Organisation [3]. The focal point of this paper is the orchestration of resources. In our case various compute and network resources have to be co-allocated to form a virtual machine that enables the execution of distributed parallel applications. To achieve this goal, a meta-scheduler is needed which generates a schedule based on user requirements and guarantees that the requested resources are reserved in advance. Therefore we developed the VIOLA MetaScheduling Service, which is founded on the experience with the UNICORE WS-Agreement prototype [11], the MeSch meta-scheduler [10] and the UNICORE timer process [2, chapter 3.7.2 "Distributed Applications"]. The integration of UNICORE and the MetaScheduling Service provides a framework which ful ls the use case's requirements and lays the foundation for co-allocation of arbitrary resources. Following the introduction (this section) we describe in Section 2 the resource management mech-anisms of the UNICORE software and the resulting challenges with respect to the VIOLA use case. Based on this description we introduce in Section 3 the VIOLA MetaScheduling Service including its architecture and the internal processes. We conclude the paper with an outline of future work related to scheduling in UNICORE (see Section 4) and a short summary (see Section 5).
... Work on interoperability includes Grid projects targeting resources running different Grid middlewares [3,14] and projects using (proposed) standard formats, e.g., to describe computational jobs [3,9]. The UniGrids [11] project specifically targets inter-operation between Globus and UNICORE. The difference between the GCM and these projects is that we also target the use of traditional non-Grid resources. ...
Conference Paper
Full-text available
Deploying Grid technologies by distributing an application over several machines has been widely used for scientific simulations, which have large requirements for computational resources. The Grid Configuration Manager (GCM) is a tool developed to ease the management of scientific applications in distributed environments and to hide some of the complexities of Grids from the end-user. In this paper we present an extension to the Grid Configuration Manager in order to incorporate a performance based resource brokering mechanism. Given a pool of machines and a trace file containing information about the runtime characteristics of the according application, GCM is able to select the combination of machines leading to the lowest execution time of the application, taking machine parameters as well as the network interconnect between the machines into account. The estimate of the execution time is based on the performance prediction tool Dimemas. The correctness of the decisions taken by GCM is evaluated in different scenarios.
Since the early days of mainframe based computing centres serving their users with compute cycles there was always a gap between the computing power offered by a centre and the computing power requested by the users of the centre, both in terms of capacity and performance. Replacing old hardware by new, more powerful one could only close the gap for a limited time since new capabilities also lead to new software developments making use of the new capabilities. This was especially true for research centres with often changing workload patterns. Moreover, the way of providing IT infrastructure turned out to be too inflexible regarding the changing computational load in times of ubiquitous use of computing. Procuring infrastructure resources to satisfy peak demand left resources idling during the day-to-day business. Frequent updates of the infrastructure to cope with increasing load are both expensive and difficult, leading, e.g. to service interruption. In the environment of research centres a first approach to overcome the problems was making use of external resources from other research centres for peak loads. With Amazon’s EC2 [1] offering Cloud computing entered the scene as a novel paradigm for providing and using computing resources for small and medium enterprises. Especially appreciated in the field of web-based applications like internet shops or hosting of companies’ web sites without the obligation to operate them locally. In contrast to Grid computing resource sharing is not in the focus of Cloud computing. Instead, access to a commercially provided infrastructure service is the appropriate label.
Since the early days of computing centres serving their users with mainframe compute cycles there was always a gap between the computing power offered by a centre and the computing power requested by the users of the centre, both in terms of capacity and performance. Replacing old hardware by new, more powerful one could only close the gap for a limited time since new capabilities also lead to new software developments making use of the new capabilities. This was especially true for research centres. Moreover, the way of providing IT infrastructure turned out to be too inflexible regarding the changing computational load in times of ubiquitous use of computing. Procuring infrastructure resources to satisfy peak demand left resources idling during the day-to-day business. Frequent updates of the infrastructure to cope with increasing load are both difficult, leading, e.g. to service interruption, and expensive. In the environment of research centres a first approach to overcome the problems was making use of external resources from other research centres for peak loads.
Grid Technologien stellen einen Lösungsansatz für die Verteilung von Anwendungen über mehrere Rechner dar, um Simulationen von wissenschaftlichen Problemen durchführen zu können, die hohe Anforderungen an Rechenressourcen haben. Während diese Art von Anwendungen in den letzten Jahren meistens noch zu Demonstrationszwecken eingesetzt wurde, ist die Grid Technologie heute mehr und mehr ein Werkzeug im täglichen Einsatz. Dabei ist die Heterogenität der vorhandenen Grid Software Umgebungen das größte Problem mit dem Benutzer umgehen müssen, wenn sie parallele, verteilte Anwendungen effizient im Grid ausführen wollen. Im Rahmen dieser Arbeit wird das Konzept und die Implementierung eines Grid Configuration Managers (GCM) vorgestellt, der die Komplexität der Grid Umgebungen und die damit verbundenen Probleme vor dem Benutzer verbergen soll. Das wichtigste Ziel des GCM ist die Vereinfachung des Managements von Grid Umgebungen für Endanwender und Entwickler. Dafür wurden die für die Ausführung von verteilten, parallelen Anwendungen notwendigen Schritte abstrahiert. Des Weiteren wurde ein Konzept für die Integration verschiedener Grid Software Lösungen entwickelt und implementiert. Zurzeit unterstützt der GCM Globus, UNICORE und ssh basierende Umgebungen. Der GCM soll Benutzer hauptsächlich während drei Phasen der Ausführung von Anwendungen helfen: bei der Definition einer Grid Konfiguration, beim Starten und bei der Überwachung einer Grid Anwendung. Der GCM bietet außerdem noch eine spezielle Unterstützung für verteilte Anwendungen, die auf Basis der Kommunikationsbibliothek PACX-MPI entwickelt wurden. Dafür werden die benötigten Konfigurationsdateien automatisch erstellt und auf den beteiligten Rechnern konsistent gehalten. In den Grid Configuration Manager wurde ein auf Leistungsvorhersage basierender Mechanismus zur Auswahl von Rechenressourcen integriert. Ausgehend von einer durch den Benutzer spezifizierten Vorauswahl an Rechnern kann der GCM anhand einer automatischen Abschätzung von Leistungsdaten einer Anwendung vorhersagen, was die effizienteste Umgebung für die Ausführung der Anwendung ist. Für die Leistungsvorhersage wird das Programm Dimemas benutzt. Dimemas kann eine Vorhersage für das Laufzeitverhalten einer Anwendung anhand von Tracing-Daten und Parameter zur Beschreibung der Hardware treffen. Der Grid Configuration Manager wurde in verschiedenen Szenarien getestet und eingesetzt. Dabei wurde aufgezeigt, dass die Handhabung von verteilten Anwendungen durch die Verwendung des GCM signifikant vereinfacht und die Festlegung der Ausführungsumgebung erleichtert wird.
Grids and mobile Grids can form the basis and the enabling technology for pervasive and utility computing due to their ability to be open, highly heterogeneous and scalable. However, the process of selecting the appropriate resources and initiating the execution of a job is not enough to provide quality in a dynamic environment such as a mobile Grid, where changes are numerous, highly variable and with unpredictable effects. In this paper we present a scheme for advancing and managing Quality of Service (QoS) attributes contained in Service Level Agreement (SLA) contracts of Grids that follow the Open Grid Services Architecture (OGSA). In order to achieve this, the execution environment of the Grid infrastructure establishes and exploits the synergies between the various modules of the architecture that participate in the management of the execution and the enforcement of the SLA contractual terms. We introduce an Execution Management Service which is in collaboration with both the application services and the network services in order to provide an adjustable quality of the requested services. The components that manage and control the execution in the Grid environment interact with the suit of the SLA-related services exchanging information that is used to provide the quality framework of the execution with respect to the agreed contractual terms. The described scheme has been implemented in the framework of the Akogrimo IST project.
Full-text available
started. The goal was to provide users of the German supercomputer centers with a seam- less, secure, and intuitive access to the heterogeneous computing resources at the centers consistent with the recommendations of the German Science Council 2 . A first prototype was developed in project UNICORE 3 to demonstrate the concept. The current production version was created in a follow-on project UNICORE Plus 4 which was completed in 2002. UNICORE's scope is expanded in projects like EUROGRID 5 , GRIP 6 and OpenMolGRID 7 to provide support for additional application areas and fun ctions like resource brokering. Section 2 familiarises the term Grid and illustrates its maj or ideas and concepts. UNI- CORE as a realization of these ideas is introduced in Section 3 followed by a detailed description of the benefits UNICORE offers to users and appli cation developers. Section 5 summarizes the paper and concludes with an outlook.
Full-text available
This paper describes software developed at the Research Centre Juelich to demonstrate the feasibility of Grid interoperability between UNICORE (Uniform Interface to Computer Resources) (1) and Globus (2) without changes to any of the systems. UNICORE user requests, like job submission, status query, and output retrieval had to be mapped to the corresponding Globus mechanisms. Another significant topic was the integration of the Globus security infrastructure into the UNICORE architecture. The functionality of this prototype has been demonstrated successfully on SC2001 in Denver.
The UNICORE Grid system implements a vertically integrated Grid architecture providing seamless access to resources within difierent or- ganizations. The software is developed and deployed by companies, re- search - and computing centres and projects throughout Europe. The UNICORE Forum (1) promotes the UNICORE software and interfaces, and makes it available for download. In this paper we instance UNI- CORE for the evolution of a Grid system towards a service-oriented Grid, primarily focussing on architectural concepts and models. Based on the current architecture and the enhancements provided by the project GRIP (2), we depict flrst steps already taken to integrate Web - and Grid Services into UNICORE. This includes provision of OGSI- compliant portTypes (3) parallel to the proprietary interfaces as well as the design of XML-based protocols. Furthermore we present the roadmap deflned by GRIP to achieve a consistent development towards an OGSA (4) implementation.
IntroductionImplementationLessonsConclusions and Future DirectionsReferences
IntroductionThe Need for Grid TechnologiesBackground An Open Grid Services ArchitectureApplication ExampleTechnical DetailsNetwork Protocol BindingsHigher-level ServicesRelated WorkSummaryAcknowledgmentsReferences