Paper
1 July 1992 Inheritance of knowledge in neural networks via symbolic algebra
Author Affiliations +
Abstract
The theme is that of `inheritance' and the use of symbolic algebra to implement it. The notion of inheriting knowledge between networks is crucial since training a network can be exceedingly slow. Partial information acquired through long experience (learning epochs) should be `transferrable.' A technique using the notion of `gluing' networks has been pioneered by Alex Waibel of CMU. However, this technique cannot be considered true inheritance since the component networks are trained on similar but different subtasks that are later `concatenated.' The approach presented here is based on the observation that even when the problem is not separable, one can get a reasonable performance out of a 2 layer network. Its training is fast and its only minimum is often below that of many of the local minima of the corresponding multi-layer networks. After training such a net, if one can transfer the knowledge to a multi-layer net, not only would one have saved valuable training time, but one would have avoided many of the local minima associated with the multi-layer network. Training can then proceed with the task of the multi-layer net reduced to improving the performance of the 2-layer net, instead of having to start from scratch. Equations corresponding to this approach are derived. They can be written for specific topologies and solved exactly (toys) or approximately (larger problems) using symbolic algebra.
© (1992) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Samir I. Sayegh M.D. "Inheritance of knowledge in neural networks via symbolic algebra", Proc. SPIE 1710, Science of Artificial Neural Networks, (1 July 1992); https://doi.org/10.1117/12.140117
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Neural networks

Artificial neural networks

Computing systems

Distance measurement

Fourier transforms

Physics

Speech recognition

Back to Top