Conference Paper

Nomadic Migration: A New Tool for Dynamic Grid Computing.

Albert-Einstein-Inst., Max-Planck-Inst. fur Gravitationsphys.;
DOI: 10.1109/HPDC.2001.945211 Conference: 10th IEEE International Symposium on High Performance Distributed Computing (HPDC-10 2001), 7-9 August 2001, San Francisco, CA, USA
Source: DBLP

ABSTRACT We describe the design and implementation of a technology which provides an application with the ability to seek out and exploit remote computing resources by migrating tasks from site to site, dynamically adapting the application to a changing Grid environment. The motivation for this migration framework, dubbed "The Worm", originated from the experience of having an abundance of computing time for simulations, which is distributed over multiple sites and split in time chunks by queuing systems. We describe the architecture of the Worm, describing how new or more suitable resources are located, and how the payload simulation is migrated to these resources following a trigger event. The migration technology presented here is designed to be used for any application, including large-scale HPC simulations

0 Bookmarks
 · 
46 Views
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Computational Science on large high performance computing resources is hampered by the complexity of these systems. Much of this complexity is due to low-level details on these resources that are exposed to the application and the end user. This includes (but is not limited to) mechanisms for remote access, configuring and building applications from source code, and managing simulations and their output files via batch queue systems. These challenges multiply in a modern research environment, where a research collaboration spans multiple groups, often in loosely defined international collaborations, where there is a constant influx of new students into multi-year projects, and where simulations are performed on several different resources. The Simulation Factory addresses these challenges by significantly simplifying remote access, building executables, and managing simulations. By abstracting out the low-level differences between different resources, it offers a uniform interface to these resources. At the same time, it can enforce certain standards for performing simulations that encapsulate best practices from experienced users. Furthermore, SimFactory's automation avoids many possible user errors that can in the worst case render month-long simulations worthless. The Simulation Factory is freely available under an open source license.
    Grid Computing (GRID), 2010 11th IEEE/ACM International Conference on; 01/2010
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Since the late 1990s, we have witnessed an extraordinary development of Grid tech- nologies. Nowadays, dieren t Grid infrastructures are being deployed within the context of growing national and transnational research projects. However, the co- existence of those dieren t infrastructures opens an interesting debate about the coordinated harnessing of their resources, from the end-user perspective, and the si- multaneous sharing of resources, from the resource owner perspective. In this paper we demonstrate the ecien t and simultaneous use of dieren t Grid infrastructures through a decentralized and \end-to-end" scheduling and execution system. In par- ticular, we evaluate the coordinated use of the EGEE and IRISGrid testbeds in the execution of a Bioinformatics application. Results show the feasibility of building loosely-coupled Grid environments only based on Globus services, while obtaining non trivial levels of quality of service, in terms of performance and reliability. Such approach allows a straightforward resource sharing since the resources are accessed by using de facto standard protocols and interfaces.
    Journal of Parallel and Distributed Computing 01/2006; 66:763-771. · 1.12 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Resource discovery is the process of locating shared resources on a computer network. Previously studied examples include efficiently finding files with a given title on a file sharing system. New developments in the application of networked computers raise the issue of dynamic resource discovery, the process of locating shared resources that are always changing. An example application is peer-to-peer computing, where a user wishes to locate idle CPU time anywhere on the network. Peer-to-peer computing is an exciting new computing paradigm. There are vast amounts of idle CPU resources scattered through the globe. We envision a peer-to-peer system to harness those resources, where every member of the network can both share their own CPU and utilize others' CPUs. In a network of hundreds of thousands of computers, resource discovery will play an important role. To avoid debilitating amounts of excess network traffic it is imperative that an efficient resource discovery algorithm be chosen. This paper's contribution to this topic is the use of gossip to reduce network traffic without sacrificing effectiveness. This project has investigated piggybacking gossip messages on other communications to increase the intelligence of searching protocols. The overhead of piggybacking the small amount of data needed is very small, and a case study by simulation shows that it can reduce network traffic by 71-84 percent
    Parallel Processing, 2006. ICPP 2006. International Conference on; 09/2006

Full-text (2 Sources)

View
21 Downloads
Available from
May 21, 2014