TERNet: A deep learning approach for thermal face emotion recognition

Shreyas Kamath K. M.; Rahul Rajendran; Qianwen Wan; Karen Panetta; Sos S. Agaian

doi:10.1117/12.2518708

13 May 2019 TERNet: A deep learning approach for thermal face emotion recognition

Shreyas Kamath K. M., Rahul Rajendran, Qianwen Wan, Karen Panetta, Sos S. Agaian

Proceedings Volume 10993, Mobile Multimedia/Image Processing, Security, and Applications 2019; 1099309 (2019) https://doi.org/10.1117/12.2518708
Event: SPIE Defense + Commercial Sensing, 2019, Baltimore, MD, United States

Abstract

Facial emotion recognition technology finds numerous real-life applications in areas of virtual learning, cognitive psychology analysis, avatar animation, neuromarketing, human machine interactions, and entertainment systems. Most state-of-the-art techniques focus primarily on visible spectrum information for emotion recognition. This becomes very arduous as emotions of individuals vary significantly. Moreover, visible images are susceptible to variation in illumination. Low lighting, variation in poses, aging, and disguise have a substantial impact on the appearance of images and textural information. Even though great advances have been made in the field, facial emotion recognition using existing techniques is often not satisfactory when compared to human performance. To overcome these shortcomings, thermal images are preferred to visible images. Thermal images a) are less sensitive to lighting conditions, b) have consistent thermal signatures, and c) have a temperature distribution formed by the face vein branches. This paper proposes a robust emotion recognition system using thermal images- TERNet. To accomplish this, customized convolutional neural network(CNNs) is employed, which possess excellent generalization capabilities. The architecture adopts features obtained via transfer learning from the VGG-Face CNN model, which is further fine-tuned with the thermal expression face data from the TUFTS face database. Computer simulations demonstrate an accuracy of 96.2% when compared to the state-of-the-art models.

Conference Presentation

Citation Download Citation

Shreyas Kamath K. M., Rahul Rajendran, Qianwen Wan, Karen Panetta, and Sos S. Agaian "TERNet: A deep learning approach for thermal face emotion recognition", Proc. SPIE 10993, Mobile Multimedia/Image Processing, Security, and Applications 2019, 1099309 (13 May 2019); https://doi.org/10.1117/12.2518708

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available