13 April 2009 Biologically-inspired robust and adaptive multi-sensor fusion and active control
Author Affiliations +
In this paper, we describe a method and system for robust and efficient goal-oriented active control of a machine (e.g., robot) based on processing, hierarchical spatial understanding, representation and memory of multimodal sensory inputs. This work assumes that a high-level plan or goal is known a priori or is provided by an operator interface, which translates into an overall perceptual processing strategy for the machine. Its analogy to the human brain is the download of plans and decisions from the pre-frontal cortex into various perceptual working memories as a perceptual plan that then guides the sensory data collection and processing. For example, a goal might be to look for specific colored objects in a scene while also looking for specific sound sources. This paper combines three key ideas and methods into a single closed-loop active control system. (1) Use high-level plan or goal to determine and prioritize spatial locations or waypoints (targets) in multimodal sensory space; (2) collect/store information about these spatial locations at the appropriate hierarchy and representation in a spatial working memory. This includes invariant learning of these spatial representations and how to convert between them; and (3) execute actions based on ordered retrieval of these spatial locations from hierarchical spatial working memory and using the "right" level of representation that can efficiently translate into motor actions. In its most specific form, the active control is described for a vision system (such as a pantilt- zoom camera system mounted on a robotic head and neck unit) which finds and then fixates on high saliency visual objects. We also describe the approach where the goal is to turn towards and sequentially foveate on salient multimodal cues that include both visual and auditory inputs.
© (2009) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Deepak Khosla, Deepak Khosla, Paul A. Dow, Paul A. Dow, David J. Huber, David J. Huber, } "Biologically-inspired robust and adaptive multi-sensor fusion and active control", Proc. SPIE 7345, Multisensor, Multisource Information Fusion: Architectures, Algorithms, and Applications 2009, 73450J (13 April 2009); doi: 10.1117/12.820503; https://doi.org/10.1117/12.820503


Babybot: a biologically inspired developing robotic agent
Proceedings of SPIE (October 15 2000)
Virtual window telepresence system for telerobotic inspection
Proceedings of SPIE (December 20 1995)
Sign language for telemanipulation
Proceedings of SPIE (December 20 1995)
Harvard binocular head
Proceedings of SPIE (February 29 1992)

Back to Top