16 September 1992 Karhunen Loève feature extraction for neural handwritten character recognition
Author Affiliations +
The optimality of the Karhunen Loeve (KL) transform is well known. Since its basis is the eigenvector set of the covariance matrix, a statistical, not functional, representation of the variance in pattern ensembles is generated. By using the KL transform coefficients as a natural feature representation of a character image, the eigenvector set can be regarded as an unsupervised biological feature extractor for a (neural) classifier. The covariance matrix and its eigenvectors are obtained from 76,753 handwritten digits. This operation is a unique expense; once the basis set is calculated it forms a linear first layer of a three weight layer feed forward network. The subsequent nonlinear perceptron layers are trained using a scaled conjugate gradient algorithm that typically affords an order of magnitude reduction in computation over the ubiquitous back-propagation algorithm. In conjunction with a massively parallel computer, training is expedited such that tens of initially different random weight sets are trained and evaluated. Increase in training set size (up to 76,755 patterns) gives less accurate learning but improved generalization on the fixed disjoint test set. A neural classifier is realized that recognizes 96.1% of 15,000 handwritten digits from 944 different writers. This recognition is attributed to the energy compaction optimality of the KL transform.
© (1992) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Patrick J. Grother, Patrick J. Grother, } "Karhunen Loève feature extraction for neural handwritten character recognition", Proc. SPIE 1709, Applications of Artificial Neural Networks III, (16 September 1992); doi: 10.1117/12.139992; https://doi.org/10.1117/12.139992


Back to Top