24 September 2012 A complete history of everything
Author Affiliations +
Abstract
This paper discusses Lick Observatory's local solution for retaining a complete history of everything. Leveraging our existing deployment of a publish/subscribe communications model that is used to broadcast the state of all systems at Lick Observatory, a monitoring daemon runs on a dedicated server that subscribes to and records all published messages. Our success with this system is a testament to the power of simple, straightforward approaches to complex problems. The solution itself is written in Python, and the initial version required about a week of development time; the data are stored in PostgreSQL database tables using a distinctly simple schema. Over time, we addressed scaling issues as the data set grew, which involved reworking the PostgreSQL database schema on the back-end. We also duplicate the data in flat files to enable recovery or migration of the data from one server to another. This paper will cover both the initial design as well as the solutions to the subsequent deployment issues, the trade-offs that motivated those choices, and the integration of this history database with existing client applications.
© (2012) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Kyle Lanclos, William T. S. Deich, "A complete history of everything", Proc. SPIE 8451, Software and Cyberinfrastructure for Astronomy II, 84510Y (24 September 2012); doi: 10.1117/12.926454; https://doi.org/10.1117/12.926454
PROCEEDINGS
12 PAGES


SHARE
RELATED CONTENT

Building archives in the virtual observatory era
Proceedings of SPIE (July 19 2010)
Gathering headers in a distributed environment
Proceedings of SPIE (July 21 2008)
Astrogrid and data mining
Proceedings of SPIE (November 01 2001)

Back to Top