Learning signer-invariant representations with adversarial training

Pedro M. Ferreira; Diogo Pernes; Ana Rebelo; Jaime S. Cardoso

doi:10.1117/12.2559534

31 January 2020 Learning signer-invariant representations with adversarial training

Pedro M. Ferreira, Diogo Pernes, Ana Rebelo, Jaime S. Cardoso

Proceedings Volume 11433, Twelfth International Conference on Machine Vision (ICMV 2019); 114333D (2020) https://doi.org/10.1117/12.2559534
Event: Twelfth International Conference on Machine Vision, 2019, Amsterdam, Netherlands

Abstract

Sign Language Recognition (SLR) has become an appealing topic in modern societies because such technology can ideally be used to bridge the gap between deaf and hearing people. Although important steps have been made towards the development of real-world SLR systems, signer-independent SLR is still one of the bottleneck problems of this research field. In this regard, we propose a deep neural network along with an adversarial training objective, specifically designed to address the signer-independent problem. Concretely speaking, the proposed model consists of an encoder, mapping from input images to latent representations, and two classifiers operating on these underlying representations: (i) the signclassifier, for predicting the class/sign labels, and (ii) the signer-classifier, for predicting their signer identities. During the learning stage, the encoder is simultaneously trained to help the sign-classifier as much as possible while trying to fool the signer-classifier. This adversarial training procedure allows learning signer-invariant latent representations that are in fact highly discriminative for sign recognition. Experimental results demonstrate the effectiveness of the proposed model and its capability of dealing with the large inter-signer variations.

Citation Download Citation

Pedro M. Ferreira, Diogo Pernes, Ana Rebelo, and Jaime S. Cardoso "Learning signer-invariant representations with adversarial training", Proc. SPIE 11433, Twelfth International Conference on Machine Vision (ICMV 2019), 114333D (31 January 2020); https://doi.org/10.1117/12.2559534

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available