8 February 2012 An innovative multimodal virtual platform for communication with devices in a natural way
Author Affiliations +
Abstract
As technology grows people are diverted and are more interested in communicating with machine or computer naturally. This will make machine more compact and portable by avoiding remote, keyboard etc. also it will help them to live in an environment free from electromagnetic waves. This thought has made 'recognition of natural modality in human computer interaction' a most appealing and promising research field. Simultaneously it has been observed that using single mode of interaction limit the complete utilization of commands as well as data flow. In this paper a multimodal platform, where out of many natural modalities like eye gaze, speech, voice, face etc. human gestures are combined with human voice is proposed which will minimize the mean square error. This will loosen the strict environment needed for accurate and robust interaction while using single mode. Gesture complement Speech, gestures are ideal for direct object manipulation and natural language is used for descriptive tasks. Human computer interaction basically requires two broad sections recognition and interpretation. Recognition and interpretation of natural modality in complex binary instruction is a tough task as it integrate real world to virtual environment. The main idea of the paper is to develop a efficient model for data fusion coming from heterogeneous sensors, camera and microphone. Through this paper we have analyzed that the efficiency is increased if heterogeneous data (image & voice) is combined at feature level using artificial intelligence. The long term goal of this paper is to design a robust system for physically not able or having less technical knowledge.
© (2012) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Chhayarani R. Kinkar, Chhayarani R. Kinkar, Richa Golash, Richa Golash, Akhilesh R. Upadhyay, Akhilesh R. Upadhyay, } "An innovative multimodal virtual platform for communication with devices in a natural way", Proc. SPIE 8289, The Engineering Reality of Virtual Reality 2012, 82890O (8 February 2012); doi: 10.1117/12.907305; https://doi.org/10.1117/12.907305
PROCEEDINGS
7 PAGES


SHARE
Back to Top