Astro-WISE: Chaining to the Universe

Source: arXiv

ABSTRACT The recent explosion of recorded digital data and its processed derivatives
threatens to overwhelm researchers when analysing their experimental data or
when looking up data items in archives and file systems. While current hardware
developments allow to acquire, process and store 100s of terabytes of data at
the cost of a modern sports car, the software systems to handle these data are
lagging behind. This general problem is recognized and addressed by various
scientific communities, e.g., DATAGRID/EGEE federates compute and storage power
over the high-energy physical community, while the astronomical community is
building an Internet geared Virtual Observatory, connecting archival data.
These large projects either focus on a specific distribution aspect or aim to
connect many sub-communities and have a relatively long trajectory for setting
standards and a common layer. Here, we report "first light" of a very different
solution to the problem initiated by a smaller astronomical IT community. It
provides the abstract "scientific information layer" which integrates
distributed scientific analysis with distributed processing and federated
archiving and publishing. By designing new abstractions and mixing in old ones,
a Science Information System with fully scalable cornerstones has been
achieved, transforming data systems into knowledge systems. This break-through
is facilitated by the full end-to-end linking of all dependent data items,
which allows full backward chaining from the observer/researcher to the
experiment. Key is the notion that information is intrinsic in nature and thus
is the data acquired by a scientific experiment. The new abstraction is that
software systems guide the user to that intrinsic information by forcing full
backward and forward chaining in the data modelling.

Download full-text


Available from: E. Deul, May 13, 2013