Semi-supervised classification of emotional pictures based on feature combination

Shuo Li; Yu-Jin Zhang

doi:10.1117/12.872188

Translator Disclaimer

17 February 2011 Semi-supervised classification of emotional pictures based on feature combination

Shuo Li; Yu-Jin Zhang

Author Affiliations +

Proceedings Volume 7881, Multimedia on Mobile Devices 2011; and Multimedia Content Access: Algorithms and Systems V; 78810X (2011) https://doi.org/10.1117/12.872188
Event: IS&T/SPIE Electronic Imaging, 2011, San Francisco Airport, California, United States

Abstract

Can the abundant emotions reflected in pictures be classified automatically by computer? Only the visual features extracted from images are considered in the previous researches, which have the constrained capability to reveal various emotions. In addition, the training database utilized by previous methods is the subset of International Affective Picture System (IAPS) that has a relatively small scale, which exerts negative effects on the discrimination of emotion classifiers. To solve the above problems, this paper proposes a novel and practical emotional picture classification approach, using semi-supervised learning scheme with both visual feature and keyword tag information. Besides the IAPS with both emotion labels and keyword tags as part of the training dataset, nearly 2000 pictures with only keyword tags that are downloaded from the website Flickr form an auxiliary training dataset. The visual feature of the latent emotional semantic factors is extracted by probabilistic Latent Semantic Analysis (pLSA) model, while the text feature is described by binary vectors on the tag vocabulary. A first Linear Programming Boost (LPBoost) classifier which is trained on the samples from IAPS combines the above two features, and aims to label the other training samples from the internet. Then the second SVM classifier which is trained on all training images using only visual feature, focuses on the test images. In the experiment, the categorization performance of our approach is better than the latest methods.

Citation Download Citation

Shuo Li and Yu-Jin Zhang "Semi-supervised classification of emotional pictures based on feature combination", Proc. SPIE 7881, Multimedia on Mobile Devices 2011; and Multimedia Content Access: Algorithms and Systems V, 78810X (17 February 2011); https://doi.org/10.1117/12.872188