Tensor representation, constrain (storage) and processing of multidimensional astronomical data over intense computing support

Humberto Farias Sr.; Camilo Nuñez Sr.; Mauricio Solar Sr.

doi:10.1117/12.2313222

6 July 2018 Tensor representation, constrain (storage) and processing of multidimensional astronomical data over intense computing support

Humberto Farias Sr., Camilo Nuñez Sr., Mauricio Solar Sr.

Author Affiliations +

Proceedings Volume 10707, Software and Cyberinfrastructure for Astronomy V; 107070M (2018) https://doi.org/10.1117/12.2313222
Event: SPIE Astronomical Telescopes + Instrumentation, 2018, Austin, Texas, United States

Abstract

The big data problem in Astronomy is a well now know issue, but in majority of the cases is constrains to volume of this data. Our propose take care of another aspect of the problem: the dimensionality problem, in the scope of multidimensional data especially Astronomical data cubes. We use tensor decompositions for two goals, first using tucker we achieve super compression rates that allowed to saving until 91% disks space and network traffic and second using a CANDECOMP/PARAFAC or Canonical Decomposition (CP) we build a system to find the multi-linear manifold in this astronomical cubes. Because this is a problem of BigData, for ours library we test three implementations: One approach over an intense use of GPU supported by PyTorch and using the traditional approach of HPC using MPI. Our proposed start from a simple but powerful idea, if we are dealing with multidimensional data (astronomical cubes), Why are we limited to use bi-dimensional techniques?. For example we use PCA for dimensionality reductions in spectral cubes instead of multidimensional approach that preserve the multi linear manifold inside this multidimensional data, we propose to pass from a linear algebra approach to a multi-linear algebra approach using tensor theory.

Conference Presentation

Citation Download Citation

Humberto Farias Sr., Camilo Nuñez Sr., and Mauricio Solar Sr. "Tensor representation, constrain (storage) and processing of multidimensional astronomical data over intense computing support", Proc. SPIE 10707, Software and Cyberinfrastructure for Astronomy V, 107070M (6 July 2018); https://doi.org/10.1117/12.2313222

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available