Translator Disclaimer
24 September 2012 Towards dynamic light-curve catalogues
Author Affiliations +
Time-domain astronomy is becoming a fundamental aspect of the next generation of astronomical instruments. The timing properties will revolutionise the studies of all kinds of astronomical objects. Consequetially, the huge complex data volumes and high cadences of these facilities will force us to overhaul and extend current software solutions. LOFAR, laying the groundwork for this, will produce a continuously updated spectral light-curve catalogue of all detected sources, with real-time capabilities to cope with the growth of 50 - 100TB/yr, making it the largest dynamic astronomical catalogue. Automated pipelines use the column-store MonetDB as their key component. We exploit SciLens, a 300+ node, 4-tier locally distributed cluster focussed on massive I/O. Introduction of the new array-based query language, SciQL, simplifies data exploration and mining. I will demonstrate how MonetDB/SQL & SciQL on its SciLens platform manages the millions of lightcurves for LOFAR. Initial benchmark results confirm the linear scale-up performance over tens of TBs using tens of nodes.
© (2012) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Bart Scheers and Fabian Groffen "Towards dynamic light-curve catalogues", Proc. SPIE 8451, Software and Cyberinfrastructure for Astronomy II, 84510M (24 September 2012);


NOAO data products program
Proceedings of SPIE (January 01 2002)
ESO Archive data and metadata model
Proceedings of SPIE (September 23 2012)
Astrogrid and data mining
Proceedings of SPIE (October 31 2001)
Simulating instruments for mining uncalibrated archives
Proceedings of SPIE (September 15 2004)
Mining the LAMOST spectral archive
Proceedings of SPIE (September 15 2004)
VLT science archive system
Proceedings of SPIE (July 02 1998)

Back to Top