Translator Disclaimer
19 July 2010 The Large Synoptic Survey Telescope data challenges
Author Affiliations +
The Data Management system for the LSST will have to perform near-real-time calibration and analysis of acquired images, particularly for transient detection and alert generation; annual processing of the entire dataset for precision calibration, object detection and characterization, and catalog generation; and support of user data access and analysis. Images will be acquired at roughly a 17-second cadence, with alerts generated within one minute. The ten-year survey will result in tens of petabytes of image and catalog data and will require ~250 teraflops of processing to reduce. The LSST project is carrying out a series of Data Challenges (DC) to refine the design, evaluate the scientific and computational performance of candidate algorithms, and address the challenging scaling issues that the LSST dataset will present. This paper discusses the progress of the DCs to date and plans for future DCs. Algorithm development must address dual requirements for the efficient use of computational resources and the accurate, reliable processing of the deep and broad survey data. The DCs incorporate both existing astronomical images and image data resulting from detailed photon-level simulations. The data is used to ensure that the system can scale to the LSST field of view and 3.2 gigapixel camera scale and meet the scientific data quality requirements. Future DCs, carried out in conjunction with the LSST Science Collaborations, are planned to deliver data products verified by computeraided analysis and actual applications as suitable for high-quality science.
© (2010) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
J. Kantor "The Large Synoptic Survey Telescope data challenges", Proc. SPIE 7740, Software and Cyberinfrastructure for Astronomy, 77401O (19 July 2010);


Data products of the ALMA and NRAO archives
Proceedings of SPIE (July 14 2016)
RabbitQR fast and flexible big data processing at LSST...
Proceedings of SPIE (August 07 2016)
J-PAS data management pipeline and archiving
Proceedings of SPIE (September 23 2012)
NRAO VLA archive survey
Proceedings of SPIE (July 11 2008)

Back to Top