Two-stream deep learning architecture for action recognition by using extremely low-resolution infrared thermopile arrays

Igor Morawski; Wen-Nung Lie

doi:10.1117/12.2566315

Translator Disclaimer

1 June 2020 Two-stream deep learning architecture for action recognition by using extremely low-resolution infrared thermopile arrays

Igor Morawski, Wen-Nung Lie

Author Affiliations +

Proceedings Volume 11515, International Workshop on Advanced Imaging Technology (IWAIT) 2020; 115150Y (2020) https://doi.org/10.1117/12.2566315
Event: International Workshop on Advanced Imaging Technologies 2020 (IWAIT 2020), 2020, Yogyakarta, Indonesia

Abstract

This paper presents a new method for action recognition using an extremely low-resolution infrared imaging sensor. Thermopile arrays give users privacy but this comes at the price of limited information captured. The question of what methods are applicable to this sensor remains open. In our work, we adopt a two-stream deep learning architecture that accepts both spatial and temporal sequences, processes them based on CNN and stacked GRU layers separately, and finally fuses the features for action classification. To the best of our knowledge, this is the first optical-flow-based method used in combination with extremely low-resolution thermal image sequences. We use a dataset of 16 × 16 pixel image sequences introduced by a related work to directly compare the results and demonstrate the superiority of our method. Experiments show that we are able to achieve a gain of nearly 6% (96.98% vs. 91.07%) in recognition accuracy in 5-classes setup classification.

Citation Download Citation

Igor Morawski and Wen-Nung Lie "Two-stream deep learning architecture for action recognition by using extremely low-resolution infrared thermopile arrays", Proc. SPIE 11515, International Workshop on Advanced Imaging Technology (IWAIT) 2020, 115150Y (1 June 2020); https://doi.org/10.1117/12.2566315

ACCESS THE FULL ARTICLE