The DIRP framework: Flexible HPC based post-processing of TB size datasets

Andreas Wicenec; Christopher J. Harris; Kevin Vinsen; Peter J. Quinn

doi:10.1117/12.925261

24 September 2012 The DIRP framework: flexible HPC based post-processing of TB size datasets

Andreas Wicenec, Christopher J. Harris, Kevin Vinsen, Peter J. Quinn

Proceedings Volume 8451, Software and Cyberinfrastructure for Astronomy II; 84510X (2012) https://doi.org/10.1117/12.925261
Event: SPIE Astronomical Telescopes + Instrumentation, 2012, Amsterdam, Netherlands

Abstract

The immense scale of data from modern radio interferometer arrays results in processing demands requiring HPC facilities to produce scientific results. However, in the modern era such facilities are more complex than a single monolithic HPC system. The transfer and processing of scientific data must be managed across hierarchies of storage and processing architectures; including traditional HPC, heterogeneous HPC, database and visualisation systems. The ICRAR Data Intensive Research Pathfinder (DIRP) will consist of an integrated system of the hardware, middleware, tools and interfaces to support ICRAR data intensive research primarily focused on data owing from the Australian SKA Pathfinder (ASKAP¹) and the Murchison Widefield Array (MWA) telescopes.

Citation Download Citation

Andreas Wicenec, Christopher J. Harris, Kevin Vinsen, and Peter J. Quinn "The DIRP framework: flexible HPC based post-processing of TB size datasets", Proc. SPIE 8451, Software and Cyberinfrastructure for Astronomy II, 84510X (24 September 2012); https://doi.org/10.1117/12.925261

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available