A new architecture paradigm for image processing pipeline applied to massive remote sensing data production

Olivier Melet; Antoine Masse; Yannick Ott; Pierre Lassalle

doi:10.1117/12.2325700

9 October 2018 A new architecture paradigm for image processing pipeline applied to massive remote sensing data production

Olivier Melet, Antoine Masse, Yannick Ott, Pierre Lassalle

Author Affiliations +

Proceedings Volume 10789, Image and Signal Processing for Remote Sensing XXIV; 107890F (2018) https://doi.org/10.1117/12.2325700
Event: SPIE Remote Sensing, 2018, Berlin, Germany

Abstract

Recent advances in very high resolution (VHR) earth observation (EO) techniques have led to a massive increase in data volumes to be processed. These remote sensing (RS) processing steps are complex and heterogeneous and an optimized use of these algorithms in both High Processing Computer (HPC) and Cloud platforms are still an important and opened study field for RS data providers with actual and future EO missions. The goal of our study is to identify and develop a new architecture to deal with large and increasing quantity of data and versatile production profiles. In this study, we took the example of the actual and complete processing pipeline used in Pleiades production to deliver perfect sensor image. This pipeline is composed of heterogeneous radiometric and geometric processing steps. In the first part, we study five main big data framework solutions. As result of this study, we identify Apache Spark as the best framework to use due to its performance, great development maturity, and data resilience certification. In the second part, and to develop the new processing pipeline, we redesign the processing pipeline with separation of the metadata management and the core processing. These good practices help us to develop and reuse legacy algorithms to an operational processing pipeline compatible with big data paradigm. As result of this development, we successfully identify a generic way to develop new processes and reuse legacy algorithms with large data paradigm and by keeping great performance and, more importantly, gaining platform flexibility.

Conference Presentation

Citation Download Citation

Olivier Melet, Antoine Masse, Yannick Ott, and Pierre Lassalle "A new architecture paradigm for image processing pipeline applied to massive remote sensing data production", Proc. SPIE 10789, Image and Signal Processing for Remote Sensing XXIV, 107890F (9 October 2018); https://doi.org/10.1117/12.2325700

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available