Facial expression recognition in the wild based on multimodal texture features

Bo Sun; Liandong Li; Guoyan Zhou; Jun He

doi:10.1117/1.JEI.25.6.061407

22 June 2016 Facial expression recognition in the wild based on multimodal texture features

Bo Sun, Liandong Li, Guoyan Zhou, Jun He

Author Affiliations +

Journal of Electronic Imaging, Vol. 25, Issue 6, 061407 (June 2016). https://doi.org/10.1117/1.JEI.25.6.061407

Abstract

Facial expression recognition in the wild is a very challenging task. We describe our work in static and continuous facial expression recognition in the wild. We evaluate the recognition results of gray deep features and color deep features, and explore the fusion of multimodal texture features. For the continuous facial expression recognition, we design two temporal–spatial dense scale-invariant feature transform (SIFT) features and combine multimodal features to recognize expression from image sequences. For the static facial expression recognition based on video frames, we extract dense SIFT and some deep convolutional neural network (CNN) features, including our proposed CNN architecture. We train linear support vector machine and partial least squares classifiers for those kinds of features on the static facial expression in the wild (SFEW) and acted facial expression in the wild (AFEW) dataset, and we propose a fusion network to combine all the extracted features at decision level. The final achievement we gained is 56.32% on the SFEW testing set and 50.67% on the AFEW validation set, which are much better than the baseline recognition rates of 35.96% and 36.08%.

CC BY: © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Citation Download Citation

Bo Sun, Liandong Li, Guoyan Zhou, and Jun He "Facial expression recognition in the wild based on multimodal texture features," Journal of Electronic Imaging 25(6), 061407 (22 June 2016). https://doi.org/10.1117/1.JEI.25.6.061407

Published: 22 June 2016

ACCESS THE FULL ARTICLE

JOURNAL ARTICLE
8 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY