Advances to the development of a basic Mexican sign-to-speech and text language translator

G. Garcia-Bautista; F. Trujillo-Romero; G. Diaz-Gonzalez

doi:10.1117/12.2238281

28 September 2016 Advances to the development of a basic Mexican sign-to-speech and text language translator

G. Garcia-Bautista, F. Trujillo-Romero, G. Diaz-Gonzalez

Proceedings Volume 9971, Applications of Digital Image Processing XXXIX; 99713E (2016) https://doi.org/10.1117/12.2238281
Event: SPIE Optical Engineering + Applications, 2016, San Diego, California, United States

Abstract

Sign Language (SL) is the basic alternative communication method between deaf people. However, most of the hearing people have trouble understanding the SL, making communication with deaf people almost impossible and taking them apart from daily activities. In this work we present an automatic basic real-time sign language translator capable of recognize a basic list of Mexican Sign Language (MSL) signs of 10 meaningful words, letters (A-Z) and numbers (1-10) and translate them into speech and text. The signs were collected from a group of 35 MSL signers executed in front of a Microsoft Kinect™ Sensor. The hand gesture recognition system use the RGB-D camera to build and storage data point clouds, color and skeleton tracking information. In this work we propose a method to obtain the representative hand trajectory pattern information. We use Euclidean Segmentation method to obtain the hand shape and Hierarchical Centroid as feature extraction method for images of numbers and letters. A pattern recognition method based on a Back Propagation Artificial Neural Network (ANN) is used to interpret the hand gestures. Finally, we use K-Fold Cross Validation method for training and testing stages. Our results achieve an accuracy of 95.71% on words, 98.57% on numbers and 79.71% on letters. In addition, an interactive user interface was designed to present the results in voice and text format.

Citation Download Citation

G. Garcia-Bautista, F. Trujillo-Romero, and G. Diaz-Gonzalez "Advances to the development of a basic Mexican sign-to-speech and text language translator", Proc. SPIE 9971, Applications of Digital Image Processing XXXIX, 99713E (28 September 2016); https://doi.org/10.1117/12.2238281

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available