21 July 2017 Continuous Chinese sign language recognition with CNN-LSTM
Author Affiliations +
Proceedings Volume 10420, Ninth International Conference on Digital Image Processing (ICDIP 2017); 104200F (2017) https://doi.org/10.1117/12.2281671
Event: Ninth International Conference on Digital Image Processing (ICDIP 2017), 2017, Hong Kong, China
Abstract
The goal of sign language recognition (SLR) is to translate the sign language into text, and provide a convenient tool for the communication between the deaf-mute and the ordinary. In this paper, we formulate an appropriate model based on convolutional neural network (CNN) combined with Long Short-Term Memory (LSTM) network, in order to accomplish the continuous recognition work. With the strong ability of CNN, the information of pictures captured from Chinese sign language (CSL) videos can be learned and transformed into vector. Since the video can be regarded as an ordered sequence of frames, LSTM model is employed to connect with the fully-connected layer of CNN. As a recurrent neural network (RNN), it is suitable for sequence learning tasks with the capability of recognizing patterns defined by temporal distance. Compared with traditional RNN, LSTM has performed better on storing and accessing information. We evaluate this method on our self-built dataset including 40 daily vocabularies. The experimental results show that the recognition method with CNN-LSTM can achieve a high recognition rate with small training sets, which will meet the needs of real-time SLR system.
© (2017) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Su Yang, Qing Zhu, "Continuous Chinese sign language recognition with CNN-LSTM", Proc. SPIE 10420, Ninth International Conference on Digital Image Processing (ICDIP 2017), 104200F (21 July 2017); doi: 10.1117/12.2281671; https://doi.org/10.1117/12.2281671
PROCEEDINGS
7 PAGES


SHARE
Back to Top