Translator Disclaimer
31 October 2016 An interactive VR system based on full-body tracking and gesture recognition
Author Affiliations +
Most current virtual reality (VR) interactions are realized with the hand-held input device which leads to a low degree of presence. There is other solutions using sensors like Leap Motion to recognize the gestures of users in order to interact in a more natural way, but the navigation in these systems is still a problem, because they fail to map the actual walking to virtual walking only with a partial body of the user represented in the synthetic environment. Therefore, we propose a system in which users can walk around in the virtual environment as a humanoid model, selecting menu items and manipulating with the virtual objects using natural hand gestures. With a Kinect depth camera, the system tracks the joints of the user, mapping them to a full virtual body which follows the move of the tracked user. The movements of the feet can be detected to determine whether the user is in walking state, so that the walking of model in the virtual world can be activated and stopped by means of animation control in Unity engine. This method frees the hands of users comparing to traditional navigation way using hand-held device. We use the point cloud data getting from Kinect depth camera to recognize the gestures of users, such as swiping, pressing and manipulating virtual objects. Combining the full body tracking and gestures recognition using Kinect, we achieve our interactive VR system in Unity engine with a high degree of presence.
© (2016) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Xia Zeng, Xinzhu Sang, Duo Chen, Peng Wang, Nan Guo, Binbin Yan, and Kuiru Wang "An interactive VR system based on full-body tracking and gesture recognition", Proc. SPIE 10021, Optical Design and Testing VII, 1002129 (31 October 2016);


Gestural interfaces for immersive environments
Proceedings of SPIE (February 28 2014)
Experiments in mixed reality
Proceedings of SPIE (January 29 2010)
Gestural interaction in a virtual environment
Proceedings of SPIE (April 15 1994)

Back to Top