Article

Astro-WISE: Chaining to the Universe

02/2007;
Source: arXiv

ABSTRACT

The recent explosion of recorded digital data and its processed derivatives
threatens to overwhelm researchers when analysing their experimental data or
when looking up data items in archives and file systems. While current hardware
developments allow to acquire, process and store 100s of terabytes of data at
the cost of a modern sports car, the software systems to handle these data are
lagging behind. This general problem is recognized and addressed by various
scientific communities, e.g., DATAGRID/EGEE federates compute and storage power
over the high-energy physical community, while the astronomical community is
building an Internet geared Virtual Observatory, connecting archival data.
These large projects either focus on a specific distribution aspect or aim to
connect many sub-communities and have a relatively long trajectory for setting
standards and a common layer. Here, we report "first light" of a very different
solution to the problem initiated by a smaller astronomical IT community. It
provides the abstract "scientific information layer" which integrates
distributed scientific analysis with distributed processing and federated
archiving and publishing. By designing new abstractions and mixing in old ones,
a Science Information System with fully scalable cornerstones has been
achieved, transforming data systems into knowledge systems. This break-through
is facilitated by the full end-to-end linking of all dependent data items,
which allows full backward chaining from the observer/researcher to the
experiment. Key is the notion that information is intrinsic in nature and thus
is the data acquired by a scientific experiment. The new abstraction is that
software systems guide the user to that intrinsic information by forcing full
backward and forward chaining in the data modelling.

Download full-text

Full-text

Available from: E. Deul, May 13, 2013
  • Source
    • "urther processed using the PAn - dromeda data reduction pipeline . This pipeline is based on the difference imaging technique ( Alard & Lupton 1998 ; Gössl & Riffeser 2002 ) which provides high precision photometric measurements in highly crowded fields . The PAndromeda pipeline has been integrated by us into the AstroWISE data management system ( Valentijn et al . 2007 ) which has been devel - oped to serve surveys with large data volumes . For a detailed description of the AstroWISE integration of our software we refer to Koppenhoefer et al . ( 2011 ) . In the following we describe each processing step that is performed by the PAndromeda pipeline starting with the warped images until the creation of "
    [Show abstract] [Hide abstract]
    ABSTRACT: The Pan-STARRS 1 (PS1) survey of M31 (PAndromeda) is designed to identify gravitational microlensing events, caused by bulge and disk stars (self-lensing) and by compact matter in the halos of M31 and the Milky Way (halo lensing or lensing by massive compact halo objects). With the 7 deg2 field of view (FOV) of PS1, the entire disk of M31 can be imaged with one single pointing. Our aim is to monitor M31 with this wide FOV with daily sampling (20 minutes day–1). In the 2010 season, we acquired in total 91 nights toward M31, with 90 nights in the r P1 and 66 nights in the i P1. The total integration time in r P1 and i P1 are 70,740 s and 36,180 s, respectively. As a preliminary analysis, we study a 40' × 40' sub-field in the central region of M31, a 20' × 20' sub-field in the disk of M31, and a 20' × 20' sub-field for the investigation of astrometric precision. We demonstrate that the point-spread function is good enough to detect microlensing events. We present light curves for six candidate microlensing events. This is a competitive rate compared with previous M31 microlensing surveys. Finally, we also present one example light curve for Cepheids, novae, and eclipsing binaries in these sub-fields.
    Full-text · Article · Sep 2011 · The Astronomical Journal
  • Source
    • "In this paper we will introduce the LOFAR Information System which deals with the problems described above. The proposed solution is based on the existing data storage and processing system for wide-field astronomy (Astro-WISE 1 [1]) and extending this system for LOFAR [2]. The practical task for the developers of the LOFAR Information System is to combine three systems in one: Astro-WISE metadata storage and data processing, EGEE/BiGGrid data storage and computing nodes and the LOFAR project administration. "
    [Show abstract] [Hide abstract]
    ABSTRACT: a b s t r a c t In this paper, we present a newly designed and implemented scientific information system for the LOFAR Long Term Archive. It is a distributed multi-tier storage and data processing system that allows a number of users to process Petabytes of data. The LOFAR Information System is designed on the base of the Astro-WISE Information System. The system allows the use of the computing and storage resources of the BiGGrid combined with the metadata database along with the data access and data processing interfaces of Astro-WISE. The architecture of the system, problems solved during the implementation and scientific use cases for the system are also described.
    Full-text · Article · Mar 2011 · Future Generation Computer Systems
  • Source
    • "In this paper we will introduce the LOFAR Information System which deals with the problems described above. The proposed solution is based on the existing data storage and processing system for wide-field astronomy (Astro-WISE 1 [1]) and extending this system for LOFAR [2]. The practical task for the developers of the LOFAR Information System is to combine three systems in one: Astro-WISE metadata storage and data processing, EGEE/BiGGrid data storage and computing nodes and the LOFAR project administration. "

    Full-text · Article · Jan 2011
Show more