Hierarchical structure stochastic automata can increase the efficiency of the back propagation method with momentum

Norio Baba; Hisashi Handa

doi:10.1117/12.235957

22 March 1996 Hierarchical structure stochastic automata can increase the efficiency of the back propagation method with momentum

Norio Baba, Hisashi Handa

Author Affiliations +

Proceedings Volume 2760, Applications and Science of Artificial Neural Networks II; (1996) https://doi.org/10.1117/12.235957
Event: Aerospace/Defense Sensing and Controls, 1996, Orlando, FL, United States

Abstract

One of the most remarkable breakthroughs in the field of neural networks may be the invention of the back propagation (BP) method by Rumelhart et al. The back propagation method has so far contributed a lot in constructing various useful intelligent systems. However, the original BP method involves several limitations. One of the most important is that it sometimes falls into a local minimum without being able to find a global minimum of the total error function. In order to overcome limitations of the original BP method, various modified BP methods have been proposed so far. The BP method with momentum may be one of the most popular modified algorithms. It has been reported that the BP method with momentum has been applied quite successfully to various actual problems. However, despite its effectiveness, this method involves the following serious problem: `Its learning performance depends heavily upon the selection of the value of the momentum factor'. Unfortunately, it seems that there has not so far been proposed an intelligent algorithm for determining an appropriate value of the momentum factor. In this paper, we suggest that hierarchical structure stochastic automata are quite helpful for finding an appropriate value of the momentum factor of the BP method with momentum. Several computer simulation results confirm our suggestion.

Citation Download Citation

Norio Baba and Hisashi Handa "Hierarchical structure stochastic automata can increase the efficiency of the back propagation method with momentum", Proc. SPIE 2760, Applications and Science of Artificial Neural Networks II, (22 March 1996); https://doi.org/10.1117/12.235957

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available