Two-stream deep learning architecture for action recognition by using extremely low-resolution infrared thermopile arrays

Igor Morawski; Wen-Nung Lie

doi:10.1117/12.2566315

1 June 2020 Two-stream deep learning architecture for action recognition by using extremely low-resolution infrared thermopile arrays

Igor Morawski, Wen-Nung Lie

Author Affiliations +

Proceedings Volume 11515, International Workshop on Advanced Imaging Technology (IWAIT) 2020; 115150Y (2020) https://doi.org/10.1117/12.2566315
Event: International Workshop on Advanced Imaging Technologies 2020 (IWAIT 2020), 2020, Yogyakarta, Indonesia

Abstract

This paper presents a new method for action recognition using an extremely low-resolution infrared imaging sensor. Thermopile arrays give users privacy but this comes at the price of limited information captured. The question of what methods are applicable to this sensor remains open. In our work, we adopt a two-stream deep learning architecture that accepts both spatial and temporal sequences, processes them based on CNN and stacked GRU layers separately, and finally fuses the features for action classification. To the best of our knowledge, this is the first optical-flow-based method used in combination with extremely low-resolution thermal image sequences. We use a dataset of 16 × 16 pixel image sequences introduced by a related work to directly compare the results and demonstrate the superiority of our method. Experiments show that we are able to achieve a gain of nearly 6% (96.98% vs. 91.07%) in recognition accuracy in 5-classes setup classification.

Citation Download Citation

Igor Morawski and Wen-Nung Lie "Two-stream deep learning architecture for action recognition by using extremely low-resolution infrared thermopile arrays", Proc. SPIE 11515, International Workshop on Advanced Imaging Technology (IWAIT) 2020, 115150Y (1 June 2020); https://doi.org/10.1117/12.2566315

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available