Biologically-inspired robust and adaptive multi-sensor fusion and active control

Deepak Khosla; Paul A. Dow; David J. Huber

doi:10.1117/12.820503

Abstract

In this paper, we describe a method and system for robust and efficient goal-oriented active control of a machine (e.g., robot) based on processing, hierarchical spatial understanding, representation and memory of multimodal sensory inputs. This work assumes that a high-level plan or goal is known a priori or is provided by an operator interface, which translates into an overall perceptual processing strategy for the machine. Its analogy to the human brain is the download of plans and decisions from the pre-frontal cortex into various perceptual working memories as a perceptual plan that then guides the sensory data collection and processing. For example, a goal might be to look for specific colored objects in a scene while also looking for specific sound sources. This paper combines three key ideas and methods into a single closed-loop active control system. (1) Use high-level plan or goal to determine and prioritize spatial locations or waypoints (targets) in multimodal sensory space; (2) collect/store information about these spatial locations at the appropriate hierarchy and representation in a spatial working memory. This includes invariant learning of these spatial representations and how to convert between them; and (3) execute actions based on ordered retrieval of these spatial locations from hierarchical spatial working memory and using the "right" level of representation that can efficiently translate into motor actions. In its most specific form, the active control is described for a vision system (such as a pantilt- zoom camera system mounted on a robotic head and neck unit) which finds and then fixates on high saliency visual objects. We also describe the approach where the goal is to turn towards and sequentially foveate on salient multimodal cues that include both visual and auditory inputs.

Citation Download Citation

Deepak Khosla, Deepak Khosla, Paul A. Dow, Paul A. Dow, David J. Huber, David J. Huber, } "Biologically-inspired robust and adaptive multi-sensor fusion and active control", Proc. SPIE 7345, Multisensor, Multisource Information Fusion: Architectures, Algorithms, and Applications 2009, 73450J (13 April 2009); doi: 10.1117/12.820503; https://doi.org/10.1117/12.820503

ACCESS THE FULL ARTICLE

PERSONAL SIGN IN
Full access may be available with your subscription

Email or Username Forgot your username?

Password Forgot your password?

Show

Remember Email/Username on this computer

Remember Password

No SPIE account? Create an account
or Institutional Sign In via Shibboleth

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $14.40

Non-members: $18.00 ADD TO CART

Show All Keywords

Keywords/Phrases

Search In:

Publication Years