Srikumar Venugopal

IBM Research

Bioinformatics pipelines depend on shared POSIX filesystems for its input, output and intermediate data storage. Containerization makes it more difficult for the workloads to access the shared file systems. In our previous study, we were able to run both ML and non-ML pipelines on Kubeflow successfully. However, the storage solutions were complex a...

Near real-time edge analytics requires dealing with the rapidly growing amount of data, limited resources, and high failure probabilities of edge nodes. Therefore, data replication is of vital importance to meet SLOs such as service availability and failure resilience. Consequently, specific input datasets, requested by on-demand analytics (e.g., o...

Over the years, developments such as cloud computing, Internet of Things, and now edge and fog computing, have probably caused paradigm fatigue among practitioners. The question arises whether adopting a specific paradigm has a fundamental effect on the development and deployment of applications. This talk will examine this question in the context...

The explosive growth of Internet-connected devices that form the Internet of Things and the flood of data they yield require new energy-efficient and error-resilient hardware and software server stacks for next-generation cloud and edge datacenters.

The ability to understand application performance and pro-actively manage system state is becoming increasingly important as infrastructure services move towards commoditisation models such as cloud computing. The complexity of systems being monitored in large corporations means that detailed component-by-component analysis and/or simulation of beh...

Information extraction (IE) is the task of automatically extracting structured information from unstructured/semi-structured machine-readable documents. Among various IE tasks, extracting actionable intelligence from an ever-increasing amount of data depends critically upon cross-document coreference resolution (CDCR) - the task of identifying enti...

Fog computing provides a conceptual approach for virtualizing and orchestrating computing, networking, and storage resources to process data. This issue helps toprogress the fog computing research field and offer solutions. It is clear that we are still in the formative phase of fog computing, and only the future will reveal which parts of the (som...

Optimizing high-performance computing applications requires understanding of both the application and its parallelization approach, the system software stack and the target architecture. Traditionally, performance tuning of parallel applications involves consideration of the underlying machine architecture, including floating point performance, mem...

Future supercomputers will need to support both traditional HPC applications and Big Data/High Performance Analysis applications seamlessly in a common environment. This motivates traditional job scheduling systems to support malleable jobs along with allocations that can dynamically change in size, in order to adapt the amount of resources to the...

Container-based cloud computing, as standard-ised and popularised by the open-source docker project has many potential opportunities for scientific application in high-performance computing. It promises highly flexible and available compute capabilities via cloud, without the resource overheads of traditional virtual machines. Further, productivity...

Elasticity is the defining feature of cloud computing. Performance analysts and adaptive system designers rely on representative benchmarks for evaluating elasticity for cloud applications under realistic reproducible workloads. A key feature of web workloads is burstiness or high variability at fine timescales. In this paper, we explore the innate...

The article is available here: http://authors.elsevier.com/a/1Qa4O_,OQCKOpe Abstract: With the advent of cloud computing, organizations are nowadays able to react rapidly to changing demands for computational resources. Not only individual applications can be hosted on virtual cloud infrastructures, but also complete business processes. This allow...

Indigenous Knowledge is important for Indigenous communities across the globe and for the advancement of our general scientific knowledge. In particular, Indigenous astronomical knowledge integrates many aspects of Indigenous Knowledge, including seasonal calendars, navigation, food economics, law, ceremony, and social structure. We aim to develop...

Distributed key-value stores (KVSs) have become an important component for data management in cloud applications. Since resources can be provisioned on demand in the cloud, there is a need for efficient node bootstrapping and decommissioning, i.e. to incorporate or eliminate the provisioned resources as a members of the KVS. It requires the data be...

Information Extraction (IE) is the task of automatically extracting structured information from unstructured/semi-structured machine-readable documents. Among various IE tasks, extracting actionable intelligence from ever-increasing amount of data depends critically upon Cross-Document Coreference Resolution (CDCR) - the task of identifying entity...

Metagenomics is the study of environments through genetic sampling of their microbiota. Metagenomic studies produce large datasets that are estimated to grow at a faster rate than the available computational capacity. A key step in the study of metagenome data is sequence similarity searching which is computationally intensive over large datasets....

Many IaaS providers allow cloud consumers to define elasticity (or auto-scaling) rules that carry out provisioning or de-provisioning actions in response to monitored variables crossing user-defined thresholds. Defining elasticity rules, however, remains as a key challenge for cloud consumers as it requires choosing appropriate threshold values to...

The IT industry needs systems management models that leverage available application information to detect quality of service, scalability and health of service. Ideally this technique would be common for varying application types with different n-tier architectures under normal production conditions of varying load, user session traffic, transactio...

Especially in large companies, business process landscapes may be made up from thousands of different process definitions and instances. As a result, a Business Process Management System (BPMS) needs to be able to handle the concurrent execution of a very large number of workflow steps. Many of these workflow steps may be resource-intensive, leadin...

Existing Grid meta-schedulers such as GridWay either target system-centric metrics, (such as utiliza-tion or throughput), or prioritize applications based on utility functions provided by the users. The system centric approach gives less importance to users' individual utility, while the user centric may have adverse effects such as poor system per...

Key-value stores such as Cassandra and HBase have gained popularity as for their scalability and high availability in the face of heavy workloads and hardware failure. Many enterprises are deploying applications backed by key-value stores on to resources leased from Infrastructure as a Service (IaaS) providers. However, current key-value stores are...

Online business processes are faced with varying workloads that require agile deployment of computing resources. Elastic processes leverage the on-demand provisioning ability of Cloud Computing to allocate and de-allocate resources as required to deal with shifting demand. To realize elastic processes, it is necessary to track the current and futur...

Resource-intensive tasks are playing an increasing role in business processes. The emergence of Cloud computing has enabled the deployment of such tasks onto resources sourced on-demand from Cloud providers. This has enabled so-called elastic processes that are able to dynamically adjust their resource usage to meet varying workloads. Traditional...

Infrastructure as a Service (IaaS) providers, such as Amazon Web Services, offer on-demand access to computing resources at pay-as-you-go prices. The key benefit of IaaS is elasticity, i.e., the ability to provision and de-provision resources at will. This feature makes IaaS infrastructure as the best platform for hosting web applications, e.g. e-b...

Virtualization has become commonplace in modern data centers, often referred as "computing clouds". The capability of virtual machine live migration brings benefits such as improved performance, manageability and fault tolerance, while allowing workload movement with a short service downtime. However, service levels of applications are likely to be...

The fixed-line and mobile telephony network is one of the crucial elements of an emergency response to a disaster event. However, frequently the phone network is overwhelmed in such situations and is disrupted. It is not cost-effective to maintain an over-provisioned IT infrastructure for such rare events. Cloud computing allows users to create res...

In this paper, we propose and implement a control mechanism that interfaces with Infrastructure as a Service (IaaS) or cloud providers to provision resources and manage instances of web applications in response to volatile and complex request patterns. We use reinforcement learning to orchestrate control actions such as provisioning servers and app...

An increasing number of providers are offering utility computing services which require users to pay only when they use them. Most of these providers currently charge users for metered usage based on fixed prices. In this paper, we analyze the pros and cons of charging fixed prices as compared to variable prices. In particular, charging fixed price...

Inspiration Grid Computing Grid Components Grid Initiatives Around the World Market-Oriented Grid Resource Management Requirements for Economy-Based Grid Systems Market-Oriented Grid Architecture Operational Flow in a Market-Oriented Grid Market-Oriented Systems in Practice Challenges of Utility Computing Models for Grids Summary and Conclusion Ack...

Introduction Architecture Grid Resource Broker Grid Market Directory (GMD) GridBank Aneka: SLA-Based Resource Provisioning Grid-Federation Conclusion and Future Directions Acknowledgments References

With the significant advances in Information and Communications Technology (ICT) over the last half century, there is an increasingly perceived vision that computing will one day be the 5th utility (after water, electricity, gas, and telephony). This computing utility, like all other four existing utilities, will provide the basic level of computin...

As users increasingly require better quality of service from grids, resource management and scheduling mechanisms have to evolve in order to satisfy competing demands on limited resources. Traditional schedulers for grids are system centric and favour system performance over increasing userpsilas utility. On the other hand market oriented scheduler...

Virtual machines (VMs) have become capable enough to emulate full-featured physical machines in all aspects. Therefore, they have become the foundation not only for flexible data center infrastructure but also for commercial Infrastructure-as-a-Service (IaaS) solutions. However, cur-rent providers of virtual infrastructure offer simple mech-anisms...

This chapter presents the design and implementation of seamless integration of two complex systems component-based distributed application framework ProActive and Gridbus Resource Broker. The integration solution provides: (i) the potential ability for componentbased distributed applications developed using ProActive framework, to leverage the econ...

In novel market-oriented resource sharing models, resource consumers pay for the resource usage and expect that non-functional requirements for the application execution, termed as quality of service (QoS), are satisfied. QoS is negotiated between two parties following the specific negotiation protocols and is recorded using service level agreement...

This keynote paper: presents a 21st century vision of computing; identifies various computing paradigms promising to deliver the vision of computing utilities; defines Cloud computing and provides the architecture for creating market-oriented Clouds by leveraging technologies such as VMs; provides thoughts on market-based resource management strate...

SUMMARY Data Grids are an emerging technology for managing large amounts of distributed data. This technology is highly-anticipated by scientic communities, such as in the area of astronomy and high energy physics, because their experiments generate massive amounts of data which need to be shared and analysed. Since it is not feasible to test diere...

Traditional resource management techniques (resource allocation, admission control and scheduling) have been found to be inadequate for many shared Grid and distributed systems, that consist of autonomous and dynamic distributed resources contributed by multiple organisations. They provide no incentive for users to request resources judiciously and...

Grids provide uniform access to aggregations of heterogeneous resources and services such as computers, networks and storage owned by multiple organizations. However, such a dynamic environment poses many challenges for application composition and deployment. In this paper, we present the design of the Gridbus Grid resource broker that allows users...

Service level agreements (SLAs) between grid users and providers have been proposed as mechanisms for ensuring that the users' quality of service (QoS) requirements are met, and that the provider is able to realise utility from its infrastructure. This paper presents a bilateral protocol for SLA negotiation using the alternate offers mechanism wher...

http://www.gridbus.org Abstract: Over the last few years, several Grids have been set up to share resources such as computers, data, and instruments to enable collaborative research. These Grids follow models and heterogeneous policies restricted by the requirements of e-Science applications for which they have been created, which has resulted in i...

Data-intensive Grid applications need access to large data sets that may each be replicated on different resources. Minimizing the overhead of transferring these data sets to the resources where the applications are executed requires that appropriate computational and data resources be selected. In this paper, we consider the problem of scheduling...

In this paper, we present the design of Aneka, a .NET based service-oriented platform for desktop grid computing that provides: (i) a configurable service container hosting pluggable services for discovering, scheduling and balancing various types of workloads and (ii) a flexible and extensible framework/API supporting various programming models in...

Effective scheduling is a key concern for the execution of performance driven grid applications. In this paper, we propose a dynamic critical path (DCP) based workflow scheduling algorithm that determines efficient mapping of tasks by calculating the critical path in the workflow task graph at every step. It assigns priority to a task in the critic...

The Department of Computer Science and Software Engineering at the University of Melbourne, Australia, is offering a master's-level Cluster and Grid Computing course for preparing trained personnel in the field. The course focuses on technologies, to realize high-performance network computing systems, programming models, and tools. The course desig...

The financial services industry today produces and consumes huge amounts of data and the processes involved in analysing these data have large and complex resource requirements. The need to analyse the data using such processes and get meaningful results in time, can be met only up to a certain extent by current computer systems. Most service provi...

IntroductionUtility GridsUtility-Based Resource Allocation at Various LevelsIndustrial Solutions for Utility ComputingConclusion AcknowledgementGlossaryCross ReferencesReferences

The next generation of scientific experiments and studies, popularly called e-Science, is carried out by large collaborations of researchers distributed around the world engaged in analysis of huge collections of data generated by scientific instruments. Grid computing has emerged as an enabler for e-Science as it permits the creation of virtual or...

The emergence of Grids as a platform for sharing and aggregation of distributed resources increases the need for mechanisms that allow an efficient management of resources. The Grid economy has been identified as one of the potential solutions as it helps in managing the supply and demand for resources and enables sustained sharing of resources by...

Data-intensive Grid applications need access to large datasets that may each be replicated on different resources. Minimizing the overhead of transferring these datasets to the resources where the applications are executed requires that appropriate computational and data resources be selected. In this paper, we introduce a heuristic for the selecti...

The distribution of knowledge (by scientists) and data sources (advanced scientific instruments), and the need for large-scale computational resources for analyzing massive scientific data are two major problems commonly observed in scientific disciplines. Two popular scientific disciplines of this nature are brain science and high-energy physics....

Several middleware for grid computing have been proposed aiming at providing means to access resources in a uniform and se- cure way, hiding the idiosyncrasies of heterogeneous resources. In this work we present the Enterprise Grids from the Xgrid per- spective and the Global Grids perspective from Gridbus. We also present the integration of Enterp...

In this paper, we present an algorithm for scheduling of distributed data intensive Bag-of-Task applications on Data Grids that have costs associated with requesting, transferring and processing datasets. The algorithm takes into ac- count the explosion of choices that result due to a job requiring multiple datasets from multiple data sources. The...

We present an algorithm for scheduling distributed data intensive bag-of-task applications on data grids that have costs associated with requesting, transferring and processing datasets. We evaluate the algorithm on a data grid testbed and present the results.

Data Grids have been adopted as the platform for scientific communities that need to share, access, transport, process and manage large data collections distributed worldwide. They combine high-end computing technologies with high-performance networking and wide-area storage management techniques. In this paper, we discuss the key concepts behind D...

Data Grids have been adopted as the platform for scientific applications that need to access, transport and process large data collections distributed worldwide. They combine high-end computing technologies with high-performance networking and wide-area storage management techniques. In this paper, we discuss the key concepts behind Data Grids and...

SUMMARY Computational Grids and peer-to-peer (P2P) networks enable the sharing, selection, and aggregation of geographically distributed resources for solving large-scale problems in science, engineering, and commerce. The management and composition of resources and services for scheduling applications, however, becomes a complex undertaking. We ha...

This work identifies challenges in managing resources in a Grid computing environment and proposes computational economy as a metaphor for effective management of resources and application scheduling. It identifies distributed resource management challenges and requirements of economy-based Grid systems, and discusses various representative economy...

Computational grids that couple geographically distributed resources are becoming the de-facto computing platform for solving large-scale problems in science, engineering, and commerce. Software to enable grid computing has been primarily written for Unix-class operating systems, thus severely limiting the ability to effectively utilize the computi...

Although Grids have been used extensively for executing applications with compute-intensive jobs, there exist several applications with a large number of lightweight jobs. The overall processing undertaking of these applications involves high overhead time and cost in terms of (i) job transmission to and from Grid resources and, (ii) job processing...

This paper presents the motivation for development and implementation of a computational portal for the processing of astrophysical and high energy physics data on global Grids. The requirements for the por- tal, its design, and choices leading to the utilisation of Globus, Gridbus Broker and GridSphere as the middleware, resource broker and portal...

In this paper we report experience in the use of computational grids in the domain of natural language processing, particularly in the area of information extraction, to create query indices for information retrieval tasks. Given the prevalence of large corpora in the natural language processing domain, computational grids offer significant utility...