13 April 2009 Biologically-inspired robust and adaptive multi-sensor fusion and active control
Author Affiliations +
Abstract
In this paper, we describe a method and system for robust and efficient goal-oriented active control of a machine (e.g., robot) based on processing, hierarchical spatial understanding, representation and memory of multimodal sensory inputs. This work assumes that a high-level plan or goal is known a priori or is provided by an operator interface, which translates into an overall perceptual processing strategy for the machine. Its analogy to the human brain is the download of plans and decisions from the pre-frontal cortex into various perceptual working memories as a perceptual plan that then guides the sensory data collection and processing. For example, a goal might be to look for specific colored objects in a scene while also looking for specific sound sources. This paper combines three key ideas and methods into a single closed-loop active control system. (1) Use high-level plan or goal to determine and prioritize spatial locations or waypoints (targets) in multimodal sensory space; (2) collect/store information about these spatial locations at the appropriate hierarchy and representation in a spatial working memory. This includes invariant learning of these spatial representations and how to convert between them; and (3) execute actions based on ordered retrieval of these spatial locations from hierarchical spatial working memory and using the "right" level of representation that can efficiently translate into motor actions. In its most specific form, the active control is described for a vision system (such as a pantilt- zoom camera system mounted on a robotic head and neck unit) which finds and then fixates on high saliency visual objects. We also describe the approach where the goal is to turn towards and sequentially foveate on salient multimodal cues that include both visual and auditory inputs.
© (2009) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Deepak Khosla, Paul A. Dow, David J. Huber, "Biologically-inspired robust and adaptive multi-sensor fusion and active control", Proc. SPIE 7345, Multisensor, Multisource Information Fusion: Architectures, Algorithms, and Applications 2009, 73450J (13 April 2009); doi: 10.1117/12.820503; https://doi.org/10.1117/12.820503
PROCEEDINGS
9 PAGES


SHARE
RELATED CONTENT

Cerebral palsy characterization by estimating ocular motion
Proceedings of SPIE (November 17 2017)
Babybot: a biologically inspired developing robotic agent
Proceedings of SPIE (October 16 2000)
Sign language for telemanipulation
Proceedings of SPIE (December 21 1995)
Laser eye: a new 3D sensor for active vision
Proceedings of SPIE (August 20 1993)
Gaze control for an active camera system by modeling human...
Proceedings of SPIE (November 01 1992)

Back to Top