27 January 2010 Camera assisted multimodal user interaction
Author Affiliations +
Abstract
Since more processing power, new sensing and display technologies are already available in mobile devices, there has been increased interest in building systems to communicate via different modalities such as speech, gesture, expression, and touch. In context identification based user interfaces, these independent modalities are combined to create new ways how the users interact with hand-helds. While these are unlikely to completely replace traditional interfaces, they will considerably enrich and improve the user experience and task performance. We demonstrate a set of novel user interface concepts that rely on built-in multiple sensors of modern mobile devices for recognizing the context and sequences of actions. In particular, we use the camera to detect whether the user is watching the device, for instance, to make the decision to turn on the display backlight. In our approach the motion sensors are first employed for detecting the handling of the device. Then, based on ambient illumination information provided by a light sensor, the cameras are turned on. The frontal camera is used for face detection, while the back camera provides for supplemental contextual information. The subsequent applications triggered by the context can be, for example, image capturing, or bar code reading.
© (2010) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Jari Hannuksela, Olli Silvén, Sami Ronkainen, Sakari Alenius, Markku Vehviläinen, "Camera assisted multimodal user interaction", Proc. SPIE 7542, Multimedia on Mobile Devices 2010, 754203 (27 January 2010); doi: 10.1117/12.839449; https://doi.org/10.1117/12.839449
PROCEEDINGS
9 PAGES


SHARE
RELATED CONTENT

Multiresolution example-based depth image restoration
Proceedings of SPIE (September 02 2009)
FCam for multiple cameras
Proceedings of SPIE (February 09 2012)
Vehicle real-time attitude-estimation system (VRAES)
Proceedings of SPIE (June 07 1996)
New video applications on mobile communication devices
Proceedings of SPIE (February 27 2008)

Back to Top