The HARPS-N archive through a Cassandra, NoSQL database suite?

Emilio Molinari; Jose Guerra; Avet Harutyunyan; Marcello Lodi; Adrian Martin

doi:10.1117/12.2233137

26 July 2016 The HARPS-N archive through a Cassandra, NoSQL database suite?

Emilio Molinari, Jose Guerra, Avet Harutyunyan, Marcello Lodi, Adrian Martin

Proceedings Volume 9913, Software and Cyberinfrastructure for Astronomy IV; 99132A (2016) https://doi.org/10.1117/12.2233137
Event: SPIE Astronomical Telescopes + Instrumentation, 2016, Edinburgh, United Kingdom

Abstract

The TNG-INAF is developing the science archive for the WEAVE instrument. The underlying architecture of the archive is based on a non relational database, more precisely, on Apache Cassandra cluster, which uses a NoSQL technology. In order to test and validate the use of this architecture, we created a local archive which we populated with all the HARPSN spectra collected at the TNG since the instrument's start of operations in mid-2012, as well as developed tools for the analysis of this data set. The HARPS-N data set is two orders of magnitude smaller than WEAVE, but we want to demonstrate the ability to walk through a complete data set and produce scientific output, as valuable as that produced by an ordinary pipeline, though without accessing directly the FITS files. The analytics is done by Apache Solr and Spark and on a relational PostgreSQL database. As an example, we produce observables like metallicity indexes for the targets in the archive and compare the results with the ones coming from the HARPS-N regular data reduction software. The aim of this experiment is to explore the viability of a high availability cluster and distributed NoSQL database as a platform for complex scientific analytics on a large data set, which will then be ported to the WEAVE Archive System (WAS) which we are developing for the WEAVE multi object, fiber spectrograph.

Citation Download Citation

Emilio Molinari, Jose Guerra, Avet Harutyunyan, Marcello Lodi, and Adrian Martin "The HARPS-N archive through a Cassandra, NoSQL database suite?", Proc. SPIE 9913, Software and Cyberinfrastructure for Astronomy IV, 99132A (26 July 2016); https://doi.org/10.1117/12.2233137

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
8 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Databases

Stars

Data modeling

Data archive systems

Spectrographs

Analytics

Astronomical imaging

Show All Keywords

Keywords/Phrases

Search In:

Publication Years