[show abstract][hide abstract] ABSTRACT: This report has two objectives. First, we describe a set of the production
distributed infrastructures currently available, so that the reader has a basic
understanding of them. This includes explaining why each infrastructure was
created and made available and how it has succeeded and failed. The set is not
complete, but we believe it is representative.
Second, we describe the infrastructures in terms of their use, which is a
combination of how they were designed to be used and how users have found ways
to use them. Applications are often designed and created with specific
infrastructures in mind, with both an appreciation of the existing capabilities
provided by those infrastructures and an anticipation of their future
capabilities. Here, the infrastructures we discuss were often designed and
created with specific applications in mind, or at least specific types of
applications. The reader should understand how the interplay between the
infrastructure providers and the users leads to such usages, which we call
usage modalities. These usage modalities are really abstractions that exist
between the infrastructures and the applications; they influence the
infrastructures by representing the applications, and they influence the ap-
plications by representing the infrastructures.
[show abstract][hide abstract] ABSTRACT: Today, data volumes have increased so significantly that we need to carefully consider how we interact with, share, and analyze data to avoid bottlenecks. In contexts such as eScience and scientific computing, a large emphasis is placed on collaboration, resulting in many well-known challenges in ensuring that data is in the right place at the right time and accessible by the right users. Yet these simple requirements create substantial challenges for the distribution, analysis, storage, and replication of potentially "large" datasets. Additional complexity is also added through constraints such as budget, data locality, usage, and available local storage. In this paper, we propose a "socially driven" approach to address some of the challenges within (academic) research contexts by defining a Social Data Cloud and underpinning Content Delivery Network: a Social CDN (S-CDN). Our approach leverages digitally encoded social constructs via social network platforms that we use to represent (virtual) research communities. Ultimately, they build upon the intrinsic incentives of members of a given scientific community to address their data challenges collaboratively and in proven trusted settings. We define the design and architecture of a S-CDN and demonstrate its feasibility via a DBLP-based case study as first steps to illustrate its usefulness.
Third International Workshop on Data Intensive Computing in the Clouds (DataCloud 2012) (in conjunction with SC12); 01/2012
[show abstract][hide abstract] ABSTRACT: The increasing deployment of sensor network infrastructures has led to large volumes of data becoming available, leading to new challenges in storing, processing and transmitting such data. This is especially true when data from multiple sensors is pre-processed prior to delivery to users. Where such data is processed in-transit (i.e. from data capture to delivery to a user) over a shared distributed computing infrastructure, it is necessary to provide some Quality of Service (QoS) guarantees to each user. We propose an architecture for supporting QoS for multiple concurrent scientific workflow data streams being processed (prior to delivery to a user) over a shared infrastructure. We consider such an infrastructure to be composed of a number of nodes, each of which has multiple processing units and data buffers. We utilize the ``token bucket" model for regulating, on a per workflow stream basis, the data injection rate into such a node. We subsequently demonstrate how a streaming pipeline, with intermediate data size variation (inflation/deflation), can be supported and managed using a dynamic control strategy at each node. Such a strategy supports end-to-end QoS with variations in data size between the various nodes involved in the workflow enactment process.
[show abstract][hide abstract] ABSTRACT: It is generally accepted that the ability to develop large-scale distributed applications has lagged seriously behind other developments in cyberinfrastructure. In this manuscript we provide insight into how such applications have been developed and an understanding of why developing applications for distributed infrastructure is hard. Our approach is unique in the sense that it is centered around half-a-dozen existing scientific applications; we posit that these scientific applications are representative of the characteristics, requirements, as well as the challenges of the bulk of current distributed applications on production cyberinfrastructure (such as the US TeraGrid). We provide a novel and comprehensive analysis of such distributed scientific applications. Specifically, we survey existing models, and methods for large-scale distributed applications, and identify commonalities, recurring structures, patterns, and abstractions. We find that there are many ad-hoc solutions employed to develop and execute distributed applications, which results in a lack of generality and the inability of distributed applications to be extensible and independent of infrastructure details. In our analysis, we introduce the notion of application vectors – a novel way of understanding the structure of distributed applications. Important contributions of this paper include identifying patterns that are derived from a wide range of real distributed applications, as well as an integrated approach to analyzing applications, programming systems, and patterns, resulting in the ability to provide a critical assessment of the current practice of developing, deploying and executing distributed applications. Gaps and omissions in the state of the art are identified, and directions for future research are outlined.
Concurrency and Computation Practice and Experience 01/2012; · 0.85 Impact Factor
[show abstract][hide abstract] ABSTRACT: One of the key motivations of computational and data grids is the ability to make coordinated use of heterogeneous computing resources which are geographically dispersed. Consequently, the performance of the network linking all the resources present in a grid has a significant impact on the performance of an application. It is therefore essential to consider network characteristics when carrying out tasks such as scheduling, migration or monitoring of jobs. This work focuses on an implementation of an autonomic network-aware meta-scheduling architecture that is capable of adapting its behavior to the current status of the environment, so that jobs can be efficiently mapped to computing resources. The implementation extends the widely used GridWay meta-scheduler and relies on exponential smoothing to predict the execution and transfer times of jobs. An autonomic control loop (which takes account of CPU use and network capability) is used to alter job admission and resource selection criteria to improve overall job completion times and throughput. The implementation has been tested using a real testbed involving heterogeneous computing resources distributed across different national organizations.
Future Generation Computer Systems 01/2012; 28:1058-1069. · 1.86 Impact Factor
[show abstract][hide abstract] ABSTRACT: Service level agreements (SLAs) have demonstrated to be efficacious tools for managing resources. They provide a valuable basis for establishing contractually binding interactions within peer-to-peer systems. Such SLAs are particularly useful when considering interactions in environments with limited trust between participants. In recent research, we observe a move from the use of SLAs as simple documents handled manually to the use of automated mechanisms for handling SLAs, appropriate for different business profiles.Complementary currencies, on the other hand, have proven to be useful for facilitating exchange among selfish peers. We identify how an SLA can itself be used as a complementary currency to encourage resource sharing between peers. Our work demonstrates how an SLA can be used as a medium of exchange and used to establish a market for computational resources. The value of an SLA can vary based on the demand for particular types of resource. By simulating a process of trade, we investigate several economic indicators. We demonstrate how the economic benefit (in terms of profit and loss) evolves based on varying levels of demand. We describe how a variation in value of an SLA can influence the overall “welfare” within a peer-to-peer system, and how such welfare is dependent on the overall demand for services and the redemption time associated with the SLA.
Future Generation Computer Systems 01/2012; · 1.86 Impact Factor
[show abstract][hide abstract] ABSTRACT: The increasing availability of Internet services has stimulated the development of peer-to-peer markets. These electronic markets have the potential for improving the efficiency of trading by reducing search and transaction costs. They also allow buyers to choose the best possible service for every transaction and interact with different sellers over time. In such markets, the establishment of trust is mandatory in order to ensure reliable market exchanges. We identify how trust can be established within a trading community, making use of pre-established Service Level Agreements between participants. The effectiveness of the approach in selecting trusted participants is evaluated via simulation.
7th International Conference on Collaborative Computing: Networking, Applications and Worksharing, CollaborateCom 2011, Orlando, FL, USA, 15-18 October, 2011; 01/2011
[show abstract][hide abstract] ABSTRACT: Grid computing generally involves the aggregation of geographically distributed resources in the context of a particular application. As such resources can exist within different administrative domains, requirements on the communication network must also be taken into account when performing meta-scheduling, migration or monitoring of jobs. Similarly, coordinating efficient interaction between different domains should also be considered when performing such meta-scheduling of jobs. A strategy to perform peer-to-peer-inspired meta-scheduling in Grids is presented. This strategy has three main goals: (1) it takes the network characteristics into account when performing meta-scheduling; (2) communication and query referral between domains is considered, so that efficient meta-scheduling can be performed; and (3) the strategy demonstrates scalability, making it suitable for many scientific applications that require resources on a large scale. Simulation results are presented that demonstrate the usefulness of this approach, and it is compared with other proposals from literature.
[show abstract][hide abstract] ABSTRACT: Service level agreements (SLAs) provide a means to define specific Quality of Service (QoS) guarantees between providers and
consumers of services. Negotiation and definition of these QoS characteristics is an area of significant research. However,
defining the actions that take place when an agreement is violated is a topic of more recent focus. This paper discusses recent
advances in this field and propose some additional features that can help both consumers and producers during the enactment
of services. These features include the ability to (re)negotiate penalties in an agreement, and specifically focuses on the
renegotiation of penalties during enactment to reflect ongoing violations.
[show abstract][hide abstract] ABSTRACT: A Service Level Agreement (SLA) represents an agreement between a service user and a provider in the context of a particular
service provision. SLAs contain Quality of Service properties that must be maintained by a provider, and as agreed between
a provider and a user/client. These are generally defined as a set of Service Level Objectives (SLOs). These properties need
to be measurable and must be monitored during the provision of the service that has been agreed in the SLA. The SLA must also
contain a set of penalty clauses specifying what happens when service providers fail to deliver the pre-agreed quality. Hence,
an SLA may be used by both a user and a provider – from a user perspective, an SLA defines what is required – often defined
using non-functional attributes of service provision. From a providers perspective, an SLA may be used to support capacity
planning – especially if a provider is making it’s capability available to multiple users. An SLA may be used by a client
and provider to manage their behaviour over time – for instance, to optimise their long running revenue (cost) or QoS attributes
(such as execution time), for instance. The lifecycle of an SLA is outlined, along with various uses of SLAs to support infrastructure
management. A discussion about WS-Agreement – the emerging standard for specifying SLAs – is also provided.
[show abstract][hide abstract] ABSTRACT: Service Level Agreements (SLAs) provide a basis for establishing contractually binding interactions within Peer-2-Peer systems. Such SLAs are particularly useful when considering interactions in environments with limited trust between participants. Complementary currencies, on the other hand, have proven to be useful for facilitating exchange among selfish peers. We identify how an SLA can itself be used as a complementary currency to encourage resource sharing between peers. Our work demonstrates how an SLA can be used as a medium of exchange and used to establish a market for computational resources. The value of an SLA can vary based on demand for particular types of resources. Simulate a process of trade we investigate several economic indicators qualified for their significance in small economies. We demonstrate how the economic benefit (in terms of profit and loss) evolves based on varying levels of demand.
Lecture Notes in Computer Science - LNCS. 01/2010;
[show abstract][hide abstract] ABSTRACT: Service level agreements (SLAs) provide a means to define specific Quality of Service (QoS) guarantees between providers and consumers of services. Ne-gotiation and definition of these QoS characteristics is an area of significant research. However, defining the actions that take place when an agreement is violated is a topic of more recent focus. This paper discusses recent advances in this field and propose some additional features that can help both consumers and producers during the enactment of services. These features include the abil-ity to (re)negotiate penalties in an agreement, and specifically focuses on the renegotiation of penalties during enactment to reflect ongoing violations.