Towards dynamic light-curve catalogues

Bart Scheers; Fabian Groffen

doi:10.1117/12.926069

24 September 2012 Towards dynamic light-curve catalogues

Bart Scheers, Fabian Groffen

Proceedings Volume 8451, Software and Cyberinfrastructure for Astronomy II; 84510M (2012) https://doi.org/10.1117/12.926069
Event: SPIE Astronomical Telescopes + Instrumentation, 2012, Amsterdam, Netherlands

Abstract

Time-domain astronomy is becoming a fundamental aspect of the next generation of astronomical instruments. The timing properties will revolutionise the studies of all kinds of astronomical objects. Consequetially, the huge complex data volumes and high cadences of these facilities will force us to overhaul and extend current software solutions. LOFAR, laying the groundwork for this, will produce a continuously updated spectral light-curve catalogue of all detected sources, with real-time capabilities to cope with the growth of 50 - 100TB/yr, making it the largest dynamic astronomical catalogue. Automated pipelines use the column-store MonetDB as their key component. We exploit SciLens, a 300+ node, 4-tier locally distributed cluster focussed on massive I/O. Introduction of the new array-based query language, SciQL, simplifies data exploration and mining. I will demonstrate how MonetDB/SQL & SciQL on its SciLens platform manages the millions of lightcurves for LOFAR. Initial benchmark results confirm the linear scale-up performance over tens of TBs using tens of nodes.

Citation Download Citation

Bart Scheers and Fabian Groffen "Towards dynamic light-curve catalogues", Proc. SPIE 8451, Software and Cyberinfrastructure for Astronomy II, 84510M (24 September 2012); https://doi.org/10.1117/12.926069

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available