Translator Disclaimer
27 January 2010 Camera assisted multimodal user interaction
Author Affiliations +
Since more processing power, new sensing and display technologies are already available in mobile devices, there has been increased interest in building systems to communicate via different modalities such as speech, gesture, expression, and touch. In context identification based user interfaces, these independent modalities are combined to create new ways how the users interact with hand-helds. While these are unlikely to completely replace traditional interfaces, they will considerably enrich and improve the user experience and task performance. We demonstrate a set of novel user interface concepts that rely on built-in multiple sensors of modern mobile devices for recognizing the context and sequences of actions. In particular, we use the camera to detect whether the user is watching the device, for instance, to make the decision to turn on the display backlight. In our approach the motion sensors are first employed for detecting the handling of the device. Then, based on ambient illumination information provided by a light sensor, the cameras are turned on. The frontal camera is used for face detection, while the back camera provides for supplemental contextual information. The subsequent applications triggered by the context can be, for example, image capturing, or bar code reading.
© (2010) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Jari Hannuksela, Olli Silvén, Sami Ronkainen, Sakari Alenius, and Markku Vehviläinen "Camera assisted multimodal user interaction", Proc. SPIE 7542, Multimedia on Mobile Devices 2010, 754203 (27 January 2010);


Accelerating defocus blur magnification
Proceedings of SPIE (March 06 2013)
Image quality metrics for degraded visual environments
Proceedings of SPIE (May 04 2017)
FCam for multiple cameras
Proceedings of SPIE (February 09 2012)
The DIMM station at Subaru Telescope
Proceedings of SPIE (June 22 2006)
New video applications on mobile communication devices
Proceedings of SPIE (February 26 2008)

Back to Top