Translator Disclaimer
24 June 2020 A framework for multimodal sign language recognition under small sample based on key-frame sampling
Author Affiliations +
Proceedings Volume 11526, Fifth International Workshop on Pattern Recognition; 115260A (2020) https://doi.org/10.1117/12.2574424
Event: Fifth International Workshop on Pattern Recognition, 2020, Chengdu, China
Abstract
Sign language recognition is challenging, due to the scarcity of available annotated corpora and the difficulty of large vocabulary. In this paper, we study the task based on a Chinese SL database-DEVISIGN, but it only has a few samples to train the deep network on the scratch. First, we segment the hand to eliminate the disturbance of irrelevant factors. By analyzing the special movement tendency of sign words, we propose two novel Key-frame selection schemes. Since no other datasets can have similar data distribution with our preprocessed data, we invent a novel cross-sampling approach, which successfully prevent the overfitting under small sample. To enhance the diversity of data, we take several samplingbased videos as input, and learn spatiotemporal features based on R(2+1)D-18 layers, which is successful in action recognition tasks. Finally, it is shown that our solution can obtain the state-of-the-art performance.
© (2020) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Jianyu Wang, Jianxin Chen, and Yihao Cai "A framework for multimodal sign language recognition under small sample based on key-frame sampling", Proc. SPIE 11526, Fifth International Workshop on Pattern Recognition, 115260A (24 June 2020); https://doi.org/10.1117/12.2574424
PROCEEDINGS
7 PAGES


SHARE
Advertisement
Advertisement
RELATED CONTENT


Back to Top