CONFERENCE PROCEEDINGS
Advanced Search
Home > Proceedings > Volume 12529 > Article
Poster + Paper
13 June 2023 Leveraging body pose estimation for gesture recognition in human-robot interaction using synthetic data
Xiaoyu Zhu, Celso M. de Melo, Alexander Hauptmann
Author Affiliations +
Proceedings Volume 12529, Synthetic Data for Artificial Intelligence and Machine Learning: Tools, Techniques, and Applications; 125290Z (2023) https://doi.org/10.1117/12.2664030
Event: SPIE Defense + Commercial Sensing, 2023, Orlando, Florida, United States
Conference Poster
Abstract
Effectively recognizing human gestures from variant viewpoints plays a fundamental role in the successful collaboration between humans and robots. Deep learning approaches have achieved promising performance in gesture recognition. However, they are usually data-hungry and require large-scale labeled data, which are not usually accessible in a practical setting. Synthetic data, on the other hand, can be easily obtained from simulators with fine-grained annotations and variant modalities. Existing state-of-the-art approaches have shown promising results using synthetic data, but there is still a large performance gap between the models trained on synthetic data and real data. To learn domain-invariant feature representations, we propose a novel approach which jointly takes RGB videos and 3D meshes as inputs to perform robust action recognition. We empirically validate our model on the RoCoG-v2 dataset, which consists of a variety of real and synthetic videos of gestures from the ground and air perspectives. We show that our model trained on synthetic data can outperform state-of-the-art models under the same training setting and models trained on real data.
© (2023) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Citation Download Citation
Xiaoyu Zhu, Celso M. de Melo, and Alexander Hauptmann "Leveraging body pose estimation for gesture recognition in human-robot interaction using synthetic data", Proc. SPIE 12529, Synthetic Data for Artificial Intelligence and Machine Learning: Tools, Techniques, and Applications, 125290Z (13 June 2023); https://doi.org/10.1117/12.2664030
ACCESS THE FULL ARTICLE
ORGANIZATIONAL
Sign in with credentials provided by your organization.
INSTITUTIONAL
Select your institution to access the SPIE Digital Library.
SELECT YOUR INSTITUTION
PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.
PERSONAL SIGN IN
No SPIE Account? Create one
PURCHASE THIS CONTENT
SUBSCRIBE TO DIGITAL LIBRARY
50 downloads per 1-year subscription
Members: $195
Non-members: $335 ADD TO CART
25 downloads per 1 - year subscription
Members: $145
Non-members: $250 ADD TO CART
PURCHASE SINGLE ARTICLE
Includes PDF, HTML & Video, when available
Members: $17.00
Non-members: $21.00 ADD TO CART
PROCEEDINGS
 9 PAGES + POSTER
DOWNLOAD PAPER SAVE TO MY LIBRARY
DOWNLOAD
POSTER
GET CITATION
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Data modeling
3D modeling
Action recognition
Pose estimation
Gesture recognition
RELATED CONTENT
Subscribe to Digital Library
Receive Erratum Email Alert
Back to Top