24 September 2012 Towards dynamic light-curve catalogues
Author Affiliations +
Abstract
Time-domain astronomy is becoming a fundamental aspect of the next generation of astronomical instruments. The timing properties will revolutionise the studies of all kinds of astronomical objects. Consequetially, the huge complex data volumes and high cadences of these facilities will force us to overhaul and extend current software solutions. LOFAR, laying the groundwork for this, will produce a continuously updated spectral light-curve catalogue of all detected sources, with real-time capabilities to cope with the growth of 50 - 100TB/yr, making it the largest dynamic astronomical catalogue. Automated pipelines use the column-store MonetDB as their key component. We exploit SciLens, a 300+ node, 4-tier locally distributed cluster focussed on massive I/O. Introduction of the new array-based query language, SciQL, simplifies data exploration and mining. I will demonstrate how MonetDB/SQL & SciQL on its SciLens platform manages the millions of lightcurves for LOFAR. Initial benchmark results confirm the linear scale-up performance over tens of TBs using tens of nodes.
© (2012) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Bart Scheers, Bart Scheers, Fabian Groffen, Fabian Groffen, } "Towards dynamic light-curve catalogues", Proc. SPIE 8451, Software and Cyberinfrastructure for Astronomy II, 84510M (24 September 2012); doi: 10.1117/12.926069; https://doi.org/10.1117/12.926069
PROCEEDINGS
8 PAGES


SHARE
RELATED CONTENT

NOAO data products program
Proceedings of SPIE (January 02 2002)
ESO Archive data and metadata model
Proceedings of SPIE (September 24 2012)
Astrogrid and data mining
Proceedings of SPIE (November 01 2001)
Mining the LAMOST spectral archive
Proceedings of SPIE (September 16 2004)
VLT science archive system
Proceedings of SPIE (July 03 1998)

Back to Top