19 July 2010 The Large Synoptic Survey Telescope data challenges
Author Affiliations +
Abstract
The Data Management system for the LSST will have to perform near-real-time calibration and analysis of acquired images, particularly for transient detection and alert generation; annual processing of the entire dataset for precision calibration, object detection and characterization, and catalog generation; and support of user data access and analysis. Images will be acquired at roughly a 17-second cadence, with alerts generated within one minute. The ten-year survey will result in tens of petabytes of image and catalog data and will require ~250 teraflops of processing to reduce. The LSST project is carrying out a series of Data Challenges (DC) to refine the design, evaluate the scientific and computational performance of candidate algorithms, and address the challenging scaling issues that the LSST dataset will present. This paper discusses the progress of the DCs to date and plans for future DCs. Algorithm development must address dual requirements for the efficient use of computational resources and the accurate, reliable processing of the deep and broad survey data. The DCs incorporate both existing astronomical images and image data resulting from detailed photon-level simulations. The data is used to ensure that the system can scale to the LSST field of view and 3.2 gigapixel camera scale and meet the scientific data quality requirements. Future DCs, carried out in conjunction with the LSST Science Collaborations, are planned to deliver data products verified by computeraided analysis and actual applications as suitable for high-quality science.
© (2010) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
J. Kantor, "The Large Synoptic Survey Telescope data challenges", Proc. SPIE 7740, Software and Cyberinfrastructure for Astronomy, 77401O (19 July 2010); doi: 10.1117/12.857253; https://doi.org/10.1117/12.857253
PROCEEDINGS
8 PAGES


SHARE
RELATED CONTENT

RabbitQR fast and flexible big data processing at LSST...
Proceedings of SPIE (August 08 2016)
Overview of aura MLS data processing
Proceedings of SPIE (January 18 2002)
NRAO VLA archive survey
Proceedings of SPIE (July 12 2008)
Designing a multi-petabyte database for LSST
Proceedings of SPIE (June 29 2006)

Back to Top