Multiple optimal learning factors for feed-forward networks

Sanjeev S. Malalur; Michael T. Manry

doi:10.1117/12.850873

12 April 2010 Multiple optimal learning factors for feed-forward networks

Sanjeev S. Malalur, Michael T. Manry

Proceedings Volume 7703, Independent Component Analyses, Wavelets, Neural Networks, Biosystems, and Nanoengineering VIII; 77030F (2010) https://doi.org/10.1117/12.850873
Event: SPIE Defense, Security, and Sensing, 2010, Orlando, Florida, United States

Abstract

A batch training algorithm for feed-forward networks is proposed which uses Newton's method to estimate a vector of optimal learning factors, one for each hidden unit. Backpropagation, using this learning factor vector, is used to modify the hidden unit's input weights. Linear equations are then solved for the network's output weights. Elements of the new method's Gauss-Newton Hessian matrix are shown to be weighted sums of elements from the total network's Hessian. In several examples, the new method performs better than backpropagation and conjugate gradient, with similar numbers of required multiplies. The method performs as well as or better than Levenberg-Marquardt, with several orders of magnitude fewer multiplies due to the small size of its Hessian.

Citation Download Citation

Sanjeev S. Malalur and Michael T. Manry "Multiple optimal learning factors for feed-forward networks", Proc. SPIE 7703, Independent Component Analyses, Wavelets, Neural Networks, Biosystems, and Nanoengineering VIII, 77030F (12 April 2010); https://doi.org/10.1117/12.850873

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available