18 July 2014 Exploring No-SQL alternatives for ALMA monitoring system
Author Affiliations +
Abstract
The Atacama Large Millimeter /submillimeter Array (ALMA) will be a unique research instrument composed of at least 66 reconfigurable high-precision antennas, located at the Chajnantor plain in the Chilean Andes at an elevation of 5000 m. This paper describes the experience gained after several years working with the monitoring system, which has a strong requirement of collecting and storing up to 150K variables with a highest sampling rate of 20.8 kHz. The original design was built on top of a cluster of relational database server and network attached storage with fiber channel interface. As the number of monitoring points increases with the number of antennas included in the array, the current monitoring system demonstrated to be able to handle the increased data rate in the collection and storage area (only one month of data), but the data query interface showed serious performance degradation. A solution based on no-SQL platform was explored as an alternative to the current long-term storage system. Among several alternatives, mongoDB has been selected. In the data flow, intermediate cache servers based on Redis were introduced to allow faster streaming of the most recently acquired data to web based charts and applications for online data analysis.
© (2014) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Tzu-Chiang Shen, Tzu-Chiang Shen, Ruben Soto, Ruben Soto, Patricio Merino, Patricio Merino, Leonel Peña, Leonel Peña, Marcelo Bartsch, Marcelo Bartsch, Alvaro Aguirre, Alvaro Aguirre, Jorge Ibsen, Jorge Ibsen, } "Exploring No-SQL alternatives for ALMA monitoring system", Proc. SPIE 9152, Software and Cyberinfrastructure for Astronomy III, 91520F (18 July 2014); doi: 10.1117/12.2055880; https://doi.org/10.1117/12.2055880
PROCEEDINGS
6 PAGES


SHARE
RELATED CONTENT

An overview of the planned CCAT software system
Proceedings of SPIE (July 17 2014)
ALMA science operations and user support: software
Proceedings of SPIE (July 29 2010)
The ALMA software architecture
Proceedings of SPIE (September 14 2004)
Real-time data acquisition and storage system
Proceedings of SPIE (January 19 1993)
Observing control and data reduction at the UKIRT
Proceedings of SPIE (July 02 1998)

Back to Top