-
I. Altintas
[show abstract]
[hide abstract]
ABSTRACT: Scientific workflows are representations of generally one, but sometimes more, process(es) in the scientific method. They combine data and computational procedures into a configurable, structured set of steps that implement semi-automated computational solutions of a scientific problem. Each atomic step in a scientific workflow uses a technology to carry out the computation or data processing. Thus, as technology progresses the requirements and motivations for usage of scientific workflows evolve with it. The first part of this invited talk aims to explore this evolution of how scientific workflows are used from late 1990s to date and discusses the advantages gained from this usage. In the second half we delve into current and expected advantages of scientific workflow systems as they mature from art to commodity, with a focus on provenance of scientific workflows and related products.As has been proven by recent workshops, challenges and community interest, capturing provenance information for computational experiments and simulations is a significant advantage of using scientific workflows to conduct computational studies. Many scientific workflow systems today provide provenance recording functionality. However, lack of generic tools to support usage of the collected information limits the usage of the provenance functionality by different users. This talk concludes by presenting a vision for creating a provenance framework to support a series of steps in the lifecycle of provenance information starting from data collection, which could serve multiple data and computational models, to the usages of this data.
Congress on Services - Part I, 2008. SERVICES '08. IEEE; 08/2008
-
M. A. Vouk, I. Altintas,
R. Barreto,
J. Blondin,
Z. Cheng,
T. Critchlow,
A. Khan,
S. Klasky,
J. Ligon,
B. Ludaescher,
P. A. Mouallem,
S. Parker,
N. Podhorszki,
A. Shoshani,
C. Silva
[show abstract]
[hide abstract]
ABSTRACT: Comprehensive, end-to-end, data and workflow management solutions are needed to handle the increasing complexity of processes
and data volumes associated with modern distributed scientific problem solving, such as ultrascale simulations and high-throughput
experiments. The key to the solution is an integrated network-based framework that is functional, dependable, faulttolerant,
and supports data and process provenance. Such a framework needs to make development and use of application workflows dramatically
easier so that scientists’ efforts can shift away from data management and utility software development to scientific research
and discovery. An integrated view of these activities is provided by the notion of scientific workflows - a series of structured
activities and computations that arise in scientific problem-solving. An information technology framework that supports scientific
workflows is the Ptolemy II based environment called Kepler. This paper discusses the issues associated with practical automation
of scientific processes and workflows and illustrates this with workflows developed using the Kepler framework and tools.
11/2007: pages 35-61;
-
[show abstract]
[hide abstract]
ABSTRACT: Evolving technologies, as exemplified by computational grids and Web services, have made it possible to solve new scientific problems that would not have been feasible previously. In order to make such advances available to the community in general and to be able to solve new problems, not necessarily from the same discipline, it is imperative to build tools that provide a common user interface in order that application programmers and users do not have to be concerned with particulars of Web services and their underlying code, computational platforms, or with data file formats. We will describe our efforts in creating a computational chemistry environment that encompasses a general scientific workflow environment, a domain specific example for quantum chemistry, our ongoing design of a workflow user interface, and our efforts at database integration.
Proceedings of the IEEE 04/2005; · 6.81 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: The article presents the Web services framework in the Kepler scientific workflow system and illustrates them with a real-world example.
Web Services, 2004. Proceedings. IEEE International Conference on; 08/2004
-
[show abstract]
[hide abstract]
ABSTRACT: Most scientists conduct analyses and run models in several different software and hardware environments, mentally coordinating the export and import of data from one environment to another. The Kepler scientific workflow system provides domain scientists with an easy-to-use yet powerful system for capturing scientific workflows (SWFs). SWFs are a formalization of the ad-hoc process that a scientist may go through to get from raw data to publishable results. Kepler attempts to streamline the workflow creation and execution process so that scientists can design, execute, monitor, re-run, and communicate analytical procedures repeatedly with minimal effort. Kepler is unique in that it seamlessly combines high-level workflow design with execution and runtime interaction, access to local and remote data, and local and remote service invocation. SWFs are superficially similar to business process workflows but have several challenges not present in the business workflow scenario. For example, they often operate on large, complex and heterogeneous data, can be computationally intensive and produce complex derived data products that may be archived for use in reparameterized runs or other workflows. Moreover, unlike business workflows, SWFs are often dataflow-oriented as witnessed by a number of recent academic systems (e.g., DiscoveryNet, Taverna and Triana) and commercial systems (Scitegic/Pipeline-Pilot, Inforsense). In a sense, SWFs are often closer to signal-processing and data streaming applications than they are to control-oriented business workflow applications.
Scientific and Statistical Database Management, 2004. Proceedings. 16th International Conference on; 07/2004
-
IEEE on Grid Computing. 01/2004;
-
[show abstract]
[hide abstract]
ABSTRACT: The authors present an approach for compiling "scientist-friendly" abstract workflow specifications into real-world executable workflows of Web service invocations, using a set of abstract-as-view definitions from a repository of abstract tasks. There have been a number of proposals and systems for scientific workflow management. However, our approach features unique aspects, in particular: the separation of abstract and concrete executable workflows; and the use of database mediation techniques to automatically translate AWFs into EWFs.
Scientific and Statistical Database Management, 2003. 15th International Conference on; 08/2003
-
I. Altintas,
S. Bhagwanani,
D. Buttler,
S. Chandra,
Zhengang Cheng,
M.A. Coleman,
T. Critchlow,
A. Gupta,
Wei Han,
Ling Liu,
B. Ludascher,
C. Pu,
R. Moore,
A. Shoshani,
M. Vouk
[show abstract]
[hide abstract]
ABSTRACT: We illustrate how a domain scientist can perform a complex scientific task by interleaving data access, querying, and manipulation, as well as analytical steps and computations in complex, problem specific ways. We show how our system is used by a geneticist for solving the problem of discovering the so-called "co-regulated" genes by interlinking data and computation from several Web sites, local computations, as well as local and remote databases. The main distinctive features of our system (compared, e.g., to the ZOO environment (Ioannidis et al., 1996)) include: (i) executable workflows run as Web services; (ii) abstract workflows employ concept names and semantic types that are higher-level (and thus more "scientist friendly") than executable workflows; and (iii) our system supports automatic translation of the latter into the former.
Scientific and Statistical Database Management, 2003. 15th International Conference on; 08/2003
-
M. A. Vouk, I. Altintas,
R. Barreto,
J. Blondin,
Z. Cheng,
T. Critchlow,
A Khan,
S. Klasky,
J. Ligon,
B. Ludaescher,
P. A. Mouallem,
S Parker,
N. Podhorszki,
A. Shoshani,
C Silva
[show abstract]
[hide abstract]
ABSTRACT: Comprehensive, end-to-end, data and workflow management solutions are needed to handle the increasing complexity of processes and data volumes associated with modern distributed scientific problem solving, such as ultrascale simulations and high-throughput experiments. The key to the solution is an integrated network-based framework that is functional, dependable, faulttolerant, and supports data and process provenance. Such a framework needs to make development and use of application workflows dramatically easier so that scientists efforts can shift away from data management and utility software development to scientific research and discovery. An integrated view of these activities is provided by the notion of scientific workflows - a series of structured activities and computations that arise in scientific problem-solving. An information technology framework that supports scientific workflows is the Ptolemy II based environment called Kepler. This paper discusses the issues associated with practical automation of scientific processes and workflows and illustrates this with workflows developed using the Kepler framework and tools. Full Text at Springer, may require registration or fee
International Federation for Information Processing Digital Library; Grid-Based Problem Solving Environments;.
-
M. A. Vouk,
Z. Cheng,
R. Barreto, I. Altintas,
J. Blondin,
T. Critchlow,
A Khan,
N. Podhorszki,
J. Ligon,
B. Ludaescher,
P. A. Mouallem,
S. Klasky,
A. Shoshani,
C Silva,
S Parker
[show abstract]
[hide abstract]
ABSTRACT: Comprehensive, end-to-end, data and workflow management solutions are needed to handle the increasing complexity of processes and data volumes associated with modern distributed scientific problem solving, such as ultrascale simulations and high-throughput experiments. The key to the solution is an integrated network-based framework that is functional, dependable, faulttolerant, and supports data and process provenance. Such a framework needs to make development and use of application workflows dramatically easier so that scientists efforts can shift away from data management and utility software development to scientific research and discovery. An integrated view of these activities is provided by the notion of scientific workflows - a series of structured activities and computations that arise in scientific problem-solving. An information technology framework that supports scientific workflows is the Ptolemy II based environment called Kepler. This paper discusses the issues associated with practical automation of scientific processes and workflows and illustrates this with workflows developed using the Kepler framework and tools. Full Text at Springer, may require registration or fee
International Federation for Information Processing Digital Library; Grid-Based Problem Solving Environments;.
