Real-time fusion of visible and thermal infrared images in surveillance applications on SoC hardware

Konrad Moren; Thomas Perschke

doi:10.1117/12.2325391

8 October 2018 Real-time fusion of visible and thermal infrared images in surveillance applications on SoC hardware

Konrad Moren, Thomas Perschke

Proceedings Volume 10802, Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies II; 108020Q (2018) https://doi.org/10.1117/12.2325391
Event: SPIE Security + Defence, 2018, Berlin, Germany

Abstract

Fusion of thermal infrared and visual images is an important technique for real-time surveillance applications. Since image fusion is used in many real-time night vision applications such as target detection, recognition and tracking, it is important to understand the processing requirements and provide computationally efficient methods. In this paper, we present a real-time image fusion system designed for night vision supervisory and monitoring purposes. The system is equipped with two image sensors: TV and IR (thermal infrared). We implement a processing pipeline on the NVIDIA Tegra TX2. The TX2 platform is equipped with a many-core NVIDIA GPU and multi-core ARM CPU. Additionally, we present the system architecture as well as the design process of the efficient, real-time multi-spectral signal-processing algorithm. The algorithm is based on the second-generation wavelets also called lifting scheme. We show also a novel parallelization approach to perform the calculations in place, so no auxiliary memory is needed. This enables a fast parallel and pipelined processing flow. We achieve a considerable speedup compared to an optimized CPU implementation. The experimental results show that the fusion system can realize real-time image fusion processing for dual channels images at the rate of 30 frames per second for the Full-HD images.

Citation Download Citation

Konrad Moren and Thomas Perschke "Real-time fusion of visible and thermal infrared images in surveillance applications on SoC hardware", Proc. SPIE 10802, Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies II, 108020Q (8 October 2018); https://doi.org/10.1117/12.2325391

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
8 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Image fusion

Wavelets

Discrete wavelet transforms

Infrared imaging

Image processing

Surveillance

Thermography

Show All Keywords

Keywords/Phrases

Search In:

Publication Years