Conference Paper

Transparent Optimization of Grid Server Selection With Real-Time Passive Network Measurements

Coll. of William & Mary, Williamsburg
DOI: 10.1109/BROADNETS.2006.4374425 Conference: Broadband Communications, Networks and Systems, 2006. BROADNETS 2006. 3rd International Conference on
Source: DBLP


Grid services have tremendously simplified the programming challenges in leveraging large-scale distributed computing. At the same time, the increased level of abstraction reduces the opportunities available to the application for optimizing its performance by monitoring the system. In this paper we introduce a monitoring grid services proxy, which transparently monitors network performance and selects between several replica service providers. This approach provides optimized server selection without any modification to or even awareness of the client application or service providers. We describe how we implement the proxy and monitor the available bandwidth to the service providers using the Wren monitoring toolkit. We present analysis indicating that our monitoring has negligible overhead. Finally, we demonstrate the practicality of our approach by optimizing the server selection for INCOGEN's VIBE, a bioinformatics workflow application that uploads gene sequences for analysis by remote service providers.

Full-text preview

Available from:
  • Source
    • "Our current model ignores the time taken to transfer data to, and from, a service. The research of Zangrilli and Lowekamp [26] addresses this issue by "
    [Show abstract] [Hide abstract]
    ABSTRACT: An approach to dynamic workflow management and optimisation using near-realtime performance data is presented. Strategies are discussed for choosing an optimal service (based on user-specified criteria) from several semantically equivalent Web services. Such an approach may involve finding "similar" services, by first pruning the set of discovered services based on service metadata, and subsequently selecting an optimal service based on performance data. The current implementation of the prototype workflow framework is described, and demonstrated with a simple workflow. Performance results are presented that show the performance benefits of dynamic service selection. A statistical analysis based on the first order statistic is used to investigate the likely improvement in service response time arising from dynamic service selection.
    Full-text · Article · Jan 2007 · Scientific Programming
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Grids offer the potential to carry out difficult computing tasks and achieve superior aggregate performance. However, grids are highly complex systems. They consist of heterogeneous resources on disparate hosts from various virtual organizations interconnected via a mixture of communication standards. Monitoring grid resources allows grid schedulers to adapt to changes in the status of these remote resources and the network paths between them. This is crucial to ensuring optimum performance. In this paper we introduce a distributed solution, called GridMAP, to collect network and end-host resource measurements, analyze their performance and feed these statistics and predictions back to schedulers. At this stage, we present our implementation of a passive TCP-SYN-based technique to provide GridMAP with round trip time and throughput measurements and we evaluate our approach against ping and iperf.
    Full-text · Conference Paper · Aug 2009