28 January 2010 Camera assisted multimodal user interaction
Author Affiliations +
Proceedings Volume 7542, Multimedia on Mobile Devices 2010; 754203 (2010); doi: 10.1117/12.839449
Event: IS&T/SPIE Electronic Imaging, 2010, San Jose, California, United States
Since more processing power, new sensing and display technologies are already available in mobile devices, there has been increased interest in building systems to communicate via different modalities such as speech, gesture, expression, and touch. In context identification based user interfaces, these independent modalities are combined to create new ways how the users interact with hand-helds. While these are unlikely to completely replace traditional interfaces, they will considerably enrich and improve the user experience and task performance. We demonstrate a set of novel user interface concepts that rely on built-in multiple sensors of modern mobile devices for recognizing the context and sequences of actions. In particular, we use the camera to detect whether the user is watching the device, for instance, to make the decision to turn on the display backlight. In our approach the motion sensors are first employed for detecting the handling of the device. Then, based on ambient illumination information provided by a light sensor, the cameras are turned on. The frontal camera is used for face detection, while the back camera provides for supplemental contextual information. The subsequent applications triggered by the context can be, for example, image capturing, or bar code reading.
© (2010) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Jari Hannuksela, Olli Silvén, Sami Ronkainen, Sakari Alenius, Markku Vehviläinen, "Camera assisted multimodal user interaction", Proc. SPIE 7542, Multimedia on Mobile Devices 2010, 754203 (28 January 2010); doi: 10.1117/12.839449; https://doi.org/10.1117/12.839449



Human-machine interfaces

Mobile devices

Facial recognition systems

Image processing

Image quality


Accelerating defocus blur magnification
Proceedings of SPIE (March 07 2013)
Nokia PureView oversampling technology
Proceedings of SPIE (March 07 2013)
Very low rate video processing
Proceedings of SPIE (December 27 2000)
FCam for multiple cameras
Proceedings of SPIE (February 09 2012)
Rethinking camera user interfaces
Proceedings of SPIE (January 24 2012)
Real-time implementation of image alignment and fusion
Proceedings of SPIE (December 06 2004)
New video applications on mobile communication devices
Proceedings of SPIE (February 27 2008)

Back to Top