Conference Paper

Shelter from the Storm: Building a Safe Archive in a Hostile World

DOI: 10.1007/11575863_49 Conference: On the Move to Meaningful Internet Systems 2005: OTM 2005 Workshops, OTM Confederated International Workshops and Posters, AWeSOMe, CAMS, GADA, MIOS+INTEROP, ORM, PhDS, SeBGIS, SWWS, and WOSE 2005, Agia Napa, Cyprus, October 31 - November 4, 2005, Proceedings
Source: DBLP


The storing of data and configuration files related to scientific experiments is vital if those experiments are to remain reproducible, or if the data is to be shared easily. The prescence of historical (observed) data is also important in order to assist in model evaluation and development. This paper describes the design and implementation process for a data archive, which was required for a coastal modelling project. The construction of the archive is described in detail, from its design through to deployment and testing. As we will show, the archive has been designed to tolerate failures in its communications with external services, and also to ensure that no information is lost if the archive itself fails, i.e. upon restarting, the archive will still be in exactly the same state.

Download full-text


Available from: Gabrielle Allen,
  • Source
    • "The SCOOP archive service [5] at LSU currently stores atmospheric, wave and surge datasets, as well as observational data. These datasets are produced by the SCOOP partners, as well as by dozens of different operational entities in the US. "
    [Show abstract] [Hide abstract]
    ABSTRACT: The SURA Coastal Ocean Observing and Prediction (SCOOP) program is implementing an integrated system of regional observations and computational models to pro-vide improved coastal forecasts. As part of this system, the getdata client was developed to search and transfer data files from the SCOOP archive in a flexible, optimized and extensible manner. The Grid Application Toolkit forms the basis for this tool, providing an abstract interface to various transfer protocols including FTP, HTTP, SCP and GridFTP. The user requirements for getdata posed a num-ber of constraints on the client design and integration with Grid middlewares. This paper discusses the SCOOP use cases and the design and implementation of getdata, along with the challenges of engineering usable tools for hetero-geneous environments and diverse communities.
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: The economically important Louisiana Coastal Area (LCA) is susceptible to hurricane activity which is increasingly aggravated by the continuing erosion of wetlands. Various programs are aimed at building sophisticated models of meteorological, coastal, and ecological processes. The emerging paradigm of Dynamic Data Driven Application Systems (DDDAS) can be applied to these models leading to new scenarios for integrated, real-time simulations that include feedback control with sensors and simulations. This paper describes the motivation and components for a comprehensive DDDAS for coastal and environmental modeling and the implications this has for scientific libraries and high performance computing. Full Text at Springer, may require registration or fee
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Grid computing has emerged as a promising area for solving complex problems and sharing resources among a large number of institutions. On the other hand, GADA workshops have arisen as a forum for researchers in grid computing, and more specifically in data management and analysis on grids. This paper analyses the works presented in GADA workshops, with the aim of evaluating the current research and foreseeing the evolution of this discipline in the near future.
Show more