SAN-RL: combining spreading activation networks and reinforcement learning to learn configurable behaviors

Daniel M. Gaines; Don Mitchell Wilkes; Kanok Kusumalnukool; Siripun Thongchai; Kazuhiko Kawamura; John H. White

doi:10.1117/12.457458

18 February 2002 SAN-RL: combining spreading activation networks and reinforcement learning to learn configurable behaviors

Daniel M. Gaines, Don Mitchell Wilkes, Kanok Kusumalnukool, Siripun Thongchai, Kazuhiko Kawamura, John H. White

Author Affiliations +

Proceedings Volume 4573, Mobile Robots XVI; (2002) https://doi.org/10.1117/12.457458
Event: Intelligent Systems and Advanced Manufacturing, 2001, Boston, MA, United States

Abstract

Reinforcement learning techniques have been successful in allowing an agent to learn a policy for achieving tasks. The overall behavior of the agent can be controlled with an appropriate reward function. However, the policy that is learned will be fixed to this reward function. If the user wishes to change his or her preference about how the task is achieved the agent must be retrained with this new reward function. We address this challenge by combining Spreading Activation Networks and Reinforcement Learning in an approach we call SAN-RL. This approach provides the agent with a causal structure, the spreading activation network, relating goals to the actions that can achieve those goals. This enables the agent to select actions relative to the goal priorities. We combine this with reinforcement learning to enable the agent to learn a policy. Together, these approaches enable the learning of a configurable behaviors, a policy that can be adapted to meet the current preferences. We compare the approach with Q-learning on a robot navigation task. We demonstrate that SAN-RL exhibits goal-directed behavior before learning, exploits the causal structure of the network to focus its search during learning and results in configurable behaviors after learning.

Citation Download Citation

Daniel M. Gaines, Don Mitchell Wilkes, Kanok Kusumalnukool, Siripun Thongchai, Kazuhiko Kawamura, and John H. White "SAN-RL: combining spreading activation networks and reinforcement learning to learn configurable behaviors", Proc. SPIE 4573, Mobile Robots XVI, (18 February 2002); https://doi.org/10.1117/12.457458

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available