We have examined the performance of a one-layer Perceptron for the detection and classification of small (resolution-limited) targets from their images, which are stochastic realizations of random processes. The processes are governed by non-Gaussian, non-white distributions. Our results show the potential of the Perceptron classifier as an Ideal Observer and suggest image detection and classification problems for which neural networks may be more reliable than human observers.

The Lister Hill National Center for Biomedical Communications is a Research and Development Division of the National Library of Medicine. One of the Center's current research projects involves the conversion of entire journals to bitmapped binary page images. In an effort to reduce operator errors that sometimes occur during document capture, three back error propagation networks were designed to automatically identify journal title based on features in the binary image of the journal's front cover page. For all three network designs, twenty five journal titles were randomly selected from the stored database of image files. Seven cover page images from each title were selected as the training set. For each title, three other cover page images were selected as the test set. Each bitmapped image was initially processed by counting the total number of black pixels in 32-pixel wide rows and columns of the page image. For the first network, these counts were scaled to create 122-element count vectors as the input vectors to a back error propagation network. The network had one output node for each journal classification. Although the network was successful in correctly classifying the 25 journals, the large input vector resulted in a large network and, consequently, a long training period. In an alternative approach, the first thirty-five coefficients of the Fast Fourier Transform of the count vector were used as the input vector to a second network. A third approach was to train a separate network for each journal using the original count vectors as input and with only one output node. The output of the network could be 'yes' (it is this journal) or 'no' (it is not this journal). This final design promises to be most efficient for a system in which journal titles are added or removed as it does not require retraining a large network for each change.

For several years the Naval Research Laboratory has worked toward the development of automated techniques for the analysis and interpretation of satellite oceanographic imagery. These techniques are combined to form the Semi-Automated Mesoscale Analysis System (SAMAS), which produces mesoscale charts of the Gulf Stream region. A key requirement of SAMAS is the ability to define location and size of mesoscale features known as eddies. A new method consists of a data reduction step using the Fourier power spectrum and a classification step using a neural network to define the presence or absence of eddies in satellite imagery. The original imagery is divided into chips, each of which overlaps the next by half the chip size. For each chip, a magnitude and direction of the maximum image 'energy' are computed from the local power spectrum. These magnitudes and directions are then used as the inputs into the neural network. The neural network has been successfully trained to distinguish 'warm eddy' and 'no-warm eddy' areas in the imagery. Accuracy of the method is shown to be high enough to produce useful results.

A technique for performing surface-based segmentation of range images using neural nets is introduced. In this approach, multilayered neural nets are used to classify local image patches according to the type of surface they belong to, based on features extracted from range and surface normal information. Central component to the efficiency and robustness is a near orientational invariant local data organization which takes place before features are extracted. This data organization reduces internal complexity by shifting the orientation invariance burden from the dimensionality of the feature spaces and/or from the internal architecture of the networks, to a much simpler sequencing of local data. The result is a well segmented image in which surfaces are properly labeled and delimited, without over segmentation. The approach shows to be robust in front of noise.

Investigation of two neural architectures is performed in two dimensions using both synthetic and real imagery. Our model follows the work performed by H. Ogmen and S. Gagne in 1990 on the fly's visual system. We extended their model to a two-dimensional architecture and also developed a new model by adding long-term memory at the input--adaptive model. Our investigation compares the response of the adaptive model against the original Ogmen and Gagne's cell-activity model. The output of both models were further processed using casual and noncausal moving average filters to help remove tonic image elements and identify direction of motion. Our simulations show that the adaptive model can be used to segment motion from sequences of imagery.

The prediction of the trajectory of mobile objects is important in many robotics applications like robot motion planning and collision avoidance. In most cases, the measurements, on which predictions are based, are subject to noise and errors. This paper presents a neural network based approach for the prediction of short-term motions of mobile objects. We studied the effect of white additive noise and gaussian noise on the prediction accuracy. An adaptive continued learning strategy is used to reduce the prediction error and accurately track the mobile objects. An empirical study was conducted to determine the architectural features of the network (number of layers and number of neurons in each layer) and the learning parameters (learning rate, momentum factor and convergence criterion) that minimize the mean squared prediction error giving an acceptable time response. The mean squared error, and the average time performance of the network (number of learning steps before convergence) are used as performance criteria. The network results are compared with those obtained from a linear regression algorithm. The neural network outperformed the linear regression in accurately predicting swiftly changing motion patterns.

Recent developments in passive millimeter-wave imaging technology are remarkable. Images of objects obtained through clouds and fog are almost indistinguishable from similar scenes taken under clear conditions. Of particular interest is the ability to image metal targets beneath camouflage, tents, polymers, wooden shelters, and certain levels of ceramic materials. A brief description of this emerging technology will be followed by several convincing examples of images to support the claims made above. Once image formation is complete, the technique of identifying objects in the image using neural networks is similar to the schemes utilized in previous Wright Laboratory Armament directorate implementations of Automatic Target Identification work for electro-optical and infrared images.

Simulation of infrared, radar, and other imaging sensors plays an important role in the planning and rehearsal of military missions and in the training of mission personnel. The challenge is to develop technology that will support the rapid use of reconnaissance imagery to generate cockpit sensor displays that accurately represent the real world while insuring correlation among the out-the-window scenes and sensor displays. This paper describes a novel, neural-network-based technique for infrared and radar image simulation directly from multi-spectral imagery (MSI). Source imagery, its processing using neural networks, and infrared and radar image simulation results are presented. Issues related to MSI database generation are also described.

In this paper we investigate the applicability of the feature extraction mechanisms found in the neurophysiology of mammals to the problem of object recognition in synthetic aperture radar imagery. Our approach presents multiple views of target objects to a two-stage-organizing neural network architecture. The first stage, a Neocognitron, performs two layers of feature extraction. The resulting feature vectors are presented to the second stage, an ART-2A classifier self-organizing neural network which clusters the features into multiple object categories. In our first experiments reported in a previous paper, the Neocognitron was trained on raw SAR imagery. The architecture was able to recognize a simulated vehicle at arbitrary azimuthal orientations at a single depression angle while rejecting clutter as well as other vehicles. Feature extraction on raw imagery yielded features that were robust but difficult to interpret. We have performed new experiments in which the self-organization process is used to discover features separately in shadow and bright returns from objects to be recognized. feature extraction on shadow returns yields oriented contrast edge operators suggestive of bipartite simple cells observed in the striate cortex of mammals. Feature extraction on the specularity patterns in bright returns yield a mixture of orientation-independent operators similar to those found in the retina, and a collection of symmetric oriented contrast edge operators. These operators are formed at multiple positions within the receptive fields during the self-organization process and collectively resemble a two-dimensional Haar basis set. we merge the feature operators discovered separately in shadow and bright returns into a combined feature extractor front end. This front end is designed to extract the desired features from raw imagery. We compare the performance of the earlier two-stage neural network with a modified network using the new feature set.

In the past fractal dimension has often been computed using a stochastic approach based on a random walk process, which has been found to be very time consuming. More recently, mathematical morphology has been used to compute the fractal dimension in a more timely fashion. This paper describes how the fractal dimension computed using mathematical morphology can be used in the texture analysis of ultrasonic imagery. The discriminatory ability of the fractal dimension as a pattern recognition feature is evaluated and compared to more traditional parameters. This analysis includes comparisons with statistical features in which each parameter is treated as an independent variable and in which interactions between those variables are evaluated. Pattern recognition techniques include Stepwise Discriminant Analysis, Linear Discriminant Analysis, and Nearest Neighbor Analysis in addition to Backpropagation Neural Network Classifiers. Our results identify the fractal dimension as one of the most important parameters for distinguishing between normal and abnormal livers. In this study, consisting of 186 images, a significant statistical difference was found for both the mean and standard deviation of the fractal dimension between the normal and abnormal groups using parametric and nonparametric statistical techniques.

A neural network was trained to classify ultrasound image samples of normal, adenomatous (benign tumor) and carcinomatous (malignant tumor) thyroid gland tissue. The samples themselves, as well as their Fourier spectrum, miscellaneous cooccurrence matrices and 'generalized' cooccurrence matrices, were successively submitted to the network, to determine if it could be trained to identify discriminating features of the texture of the image, and if not, which feature extractor would give the best results. Results indicate that the network could indeed extract some distinctive features from the textures, since it could accomplish a partial classification when trained with the samples themselves. But a significant improvement both in learning speed and performance was observed when it was trained with the generalized cooccurrence matrices of the samples.

Near-simultaneous, multispectral, coregistered imagery of ground target and background signatures were collected over a full diurnal cycle in the MWIR, LWIR, near-infrared, blue, green, and red wavebands using Battelle's portable sensor suite. The imagery data were processed with classical statistical algorithms and artificial neural networks to discriminate target signatures from background clutter and investigate automatic target detection and recognition schemes.

We present a biologically-inspired early vision network that is well-suited to highly active and responsive vision platforms. The network exploits normally undesirable camera motion as a necessary step in detecting image contrast. It also detects visual motion, producing distinctive signals from which useful image motion parameters are extracted. The network remains sensitive over a very wide dynamic range of inputs, and has self-calibrating properties that make it amenable to analog VLSI implementation. The results also support the hypothesis that vertebrate cones function primarily as detectors of contrast and motion, rather than intensity. Experiments verify that naturally occurring jitter in a motor-mounted camera, instead of being avoided, can be exploited in early visual processing.

Neural networks are general nonlinear systems that map from one vector space into another. These trainable mappers have been applied to numerous problems in geosciences and scientific computing. This study investigates the use of neural networks to map the present positions of mesoscale features in the ocean into future positions, thus performing neural-based 'forecasting' of mesoscale dynamics. Specifically, a neural network has been trained to predict the position of the North Wall of the Gulf Stream based on a recent history of Gulf Stream positions. An archive of Gulf Stream positional data covering several years has been assembled. Eigenvector analysis of the archive showed that any given realization of Gulf Stream shape can be reasonable parameterized as a set of eigenvector, or normal mode, coefficients. Gulf Stream dynamics can, therefore, be conceptualized as a time series of these coefficients. A neural network has been trained to take advantage of any existing time coherency in Gulf Stream normal mode coefficients to produce forecast coefficients that describe Gulf Stream shape and position in the future. The neural network performance is compared with persistence and with other forecasting systems. Forecast skill for the neural network is found to be generally superior to other methods considered, but computational requirements are only a fraction of those required by alternate methodologies.

Hybrid knowledge bases (HKBs), proposed by Nerode and Subrahmanian, provide a uniform theoretical framework for dealing with the mixed data types and multiple reasoning modes required for solving logical deployment problems. Algorithms based on mixed integer linear programming techniques have been developed for the syntactic subset of HKBs corresponding to function-free Prolog-like logic programs. In this study, we examine the ability of neural networks to solve a more comprehensive set of problems expressed within the hybrid knowledge base framework. The objective of this research is to design and implement a nonlinear optimization procedure for solving extended logic programs with neural networks. We focus upon two types of extensions which are typically required in the formulation of logical deployment problems. The first type of extension, which we shall refer to as a Type I extension, consists of embedding numerical and geometric constraints into logic programs. The second type of extension, which we shall call a Type II extension, consists of incorporating optimization problems into logic clauses.

Neural processing of microwave sea echo is proposed for the suppression of strong reflections from scatterers on the ocean surface, commonly referred as sea clutter. A radial basis function (RBF) neural network is shown to be effective for this purpose based on real experimental data. In addition, using the RBF neural network as a model for sea clutter, a novel adaptive detection technique is introduced and applied to the problem of detection of growlers (small fragments of icebergs) in sea clutter. The performance of this new detection method is shown to be superior to that of a conventional detector for the real data sets used in this paper.

With respect to Rosenblatt linear perceptron, a classical limitation theorem demonstrated by M. Minsky and S. Papert is discussed. This theorem, '$PSI_One-in-a-box', ultimately concern the intrinsic limitations of parallel calculations in pattern calculations in pattern recognition problems. We demonstrate a possible solution of this limitation problem by substituting the static definition of characteristic functions and of their domains in the 'geometrical' perceptron, with their dynamic definition. This dynamics consists in the mutual redefinition of the characteristic function and of its domain depending on the matching with the input. We show an application of this 'dynamic' perceptron scheme in particle tracks recognition in high energy physics. Actually, this algorithm is being used for real time automatic triggering of ADONE e⁺e^- storage ring (Frascati, Rome) to evaluate the neutron time-like electromagnetic form factor in the context of 'Fenice' collaboration by Italian Institute of Nuclear Physics (INFN).

This paper proposes a neural network recognition system for hand keying Radio Morse codes. The system has been trained and tested on real world data recorded from amateur radio Morse codes. The overall recognizing process can be partitioned into 3 major parts, the preprocessing, the feature extracting, and the character decoding. The whole operation is able to be performed in real-time on a PC/486 system. Self-Organizing Maps are used intensively in the recognition system to adaptively learn the variation of the Morse code signal. The average performance of the recognition system has been achieved about 96.4% with a rejection rate of 6.5%. It is hoped that many of the techniques would be applicable to a wide range of DSP and recognition tasks.

A benchmark study of two self-organizing artificial neural network models, ART2 and DIGNET, is conducted. The architecture differences and learning procedures between these two models are compared. The performance of ART2 and DIGNET on data clustering and pattern recognition problems with noise or interference is investigated by computer simulations. It is shown that DIGNET generally has faster learning and better clustering performance on the statistical pattern recognition problems. DIGNET has a simpler architecture, and the system parameters can be analytically determined from the self- organizing process. The threshold value used in DIGNET can be specifically determined from a given lower bound on the desirable signal-to-noise ratio (SNR). A modified model based on the features of ART2 and DIGNET is also derived and investigated. The simpler architecture combines the ART2 structure with the advantages of DIGNET model. The concepts of well depth and stage age originally introduced in DIGNET are applied in the modified model. The modified model preserves the features of noise suppression, contrast enhancement and self- organizing stable pattern recognition of ART2, yet provides a specific method to adjust parameters in the network. The network performs a variant of K-means learning, but without the knowledge of a priori information on the actual number of clusters. The networks discussed in this paper are applied and benchmarked against clustering and pattern recognition problems. Comparative simulation results of the networks are also presented.

Here we present a hybrid method of generating a hierarchical recognition system based on example learning. The method is 'hybrid' in that it uses both conventional Artificial Intelligence and Artificial Neural Network techniques. The integrated hierarchical recognition system, called IHKB (integrated hierarchical knowledge base), has a tree structure consisting of nodes and leaves. Each node is indexed by an attribute set and contains a small Kohonen network (KN). Each leaf represents a recognition class. The system uses a conceptual function to instruct the process of attribute choosing. Whenever a suitable attribute set is obtained for a certain group of training examples, a small Kohonen net is built and trained with those examples. This allows the machine to focus on special features of these training examples and thus to better describe the special characteristics of these patterns. Typically, there are many KNs in a IHKB, the number depending on the number of attribute sets. The position of each KN in the tree is fixed automatically. When the construction is complete, the training examples are classified by Kohonen nets, and recognition is achieved by a path from the root of the tree to a leaf. The method has been tested on individual handwritten character recognition, showing that high recognition rates can be achieved given enough training examples.

In this paper we present and compare three different massively parallel implementations of multilayer feedforward neural networks on a MasPar MP-1216, a parallel SIMD computer with 16,384 processors. For multilayer feedforward networks we have obtained sustained rates of up to 348 MCPS and 129 MCUPS with backpropagation, a high mark for general purpose SIMD computers. After a brief introduction to SNNS, the paper first focuses on the problems of mapping neural networks to parallel hardware. Different aspects of parallelism are presented. Two combinations of unit and training pattern parallelism were implemented as well as link and training pattern parallelism. We describe the implementation problems in obtaining high propagation rates on a SIMD machine and problems with the resulting learning algorithms in general.

How to determine the element numbers in hidden Layers is a key problem in architecture design of a multi-layer ANN (Artificial Neural Network). In order to solve this problem we propose a growth model of elements in hidden layer. In a multi-layer ANN, (for example three layers) the element numbers in input and output layer are obviously determined by the requirements of a given problem, but the problem of how to determine the element numbers in hidden layer is left with a certain randomness. It is sure that the more complex the problem, the higher the degree of nonlinearity, the higher the accuracy is required, and the larger amounts of hidden layer elements are needed. The approach adopted to this problem is briefly described as follows. we first use a fewer hidden layer elements, and check whether this amounts meet the requirements of the given problem complexity, if this fails to meet it, a new element can be grown out. Working in this way until given requirements can be fmnaly satisfied, we get an ANN architecture with a properly determined hidden layer element numbers. The neccessary steps are as follows. (1). Determine the element numbers both in input and output layer according to the given requirements, select an initial amount of the hidden layer elements and form an initial ANN. (2). Set a given training accuracy E , and set a maximun value K of the grown elements in hidden layer. (3). Train the initial ANN by a faster BP algorithm, and get a fmnal error e. (4). If Ie < E , stop. (5). Record the Value L, which denotes the times that Ie < , if L <K, then return to step (3) to get off the occasional errors. (6). If L <K, add one or more new elements to the initial hidden layer, based on these all trained parameters, go back to step (3). Finnaly, we get a grown stable ANN architecture. Computer simulation results are shown in Tab.1

Modular neural networks integrate several neural networks and possibly standard processing methods. Tackling such models is a challenge, since various modules have to be combined, either sequentially or in parallel, and the simulations are time critical in many cases. For this, specific tools are prerequisite that are both flexible and efficient. We have developed the MONNET software system that supports the investigation of complex modular models. The design of MONNET is based on the object oriented paradigm, the environment is C++/UNIX. The basic concepts are dynamic modularity, object passing, scalability, reusability, and extensibility. MONNET features flexible and compact definition of complex simulations, and minimal overhead in order to run computationally demanding simulations efficiently.

This paper presents an analog circuit implementation for an adaptive resonance theory neural network architecture, called the augmented ART-1 neural network (AART1-NN). The AART1-NN is a modification of the popular ART1-NN, developed by Carpenter and Grossberg, and it exhibits the same behavior as the ART1-NN. The AART1-NN is a real-time model, and has the ability to classify an arbitrary set of binary input patterns into different clusters. The design of the AART1-NN model. The circuit is implemented by utilizing analog electronic components, such as, operational amplifiers, transistors, capacitors, and resistors. The implemented circuit is verified using the PSpice circuit simulator, running on Sun workstations. Results obtained from the PSpice circuit simulation compare favorably with simulation results produced by solving the differential equations numerically. The prototype system developed here can be used as a building block for larger AART1-NN architectures, as well as for other types of ART architectures that involve the AART1-NN model.

A real-time recurrent learning algorithm was applied to a five class radar target identification problem. Wideband radar signatures were generated for five aircraft classes. Since an aircraft in flight is constantly in motion, a radar can measure sequences of radar signatures as the aspect angle changes. A radar can also generate aspect angle estimates by using kinematic information from aircraft position and velocity measurements. A recurrent neural network computer program (implementing a real time recurrent learning algorithm) was trained to recognize these sequences of radar signatures. Each radar signature was described by 6 external input features: the estimated target azimuth, the estimated target width, and 4 noisy amplitude values from 2 peak range bins. Nine consecutive radar signatures were sufficient to achieve a test set accuracy of 96%.

We have developed a novel neural network based automatic target recognition (ATR) indexing system. This system utilizes regularization edge detection, adaptive vector quantization (AVQ) clustering, model driven feedback, and backpropagation trained networks. It can be designed to be invariant to either translation, or translation and rotation. The system incorporates both top-down and bottom-up processing to suppress background clutter.

The n-mode fiber optic sensor built has four linearly polarized (LP) modes propagating simultaneously in the fiber, producing a two-dimensional, spatially distributed output intensity pattern. When the fiber is strained, there is a change in fiber parameters. Oscillating and rotating of the pattern caused by coupling between degenerate modes is observed. Thus the processing of this type of output signal becomes one of a two-dimensional image processor. A neural network signal processor employing a back propagation algorithm was used in conjunction with the few mode fiber optic sensor to categorize the spatial output patterns from the sensor, thus converting the optical pattern to its corresponding strain value. The testing results show that the neural network processor is capable of recognizing this kind of image with good accuracy, resulting in strain accuracies within 0.7 percent.

This paper considers the application of neural networks for jet engine diagnostics. Aircraft engine trending and diagnostics provide engine managers and fleet managers with critical information on the health of their engines and assist in identifying potential failures before they occur. The key to a trending system is its ability to model critical engine parameters accurately and then using the difference between the actual and modeled parameters to predict engine malfunction. A backpropagation neural network provides a powerful tool for modeling these parameters. Flight performance data from the F-16 F-100 engine was gathered over a four month period from 90 engines. Five separate, but identical in architecture, networks were implemented in software to model five key parameters of the engine using data from engines known to be good. The trained network then was tested against engine data unseen during training by the network and known to have corrected component failures during the period covered by the data. Comparing the difference between the network modeled parameter and the actual parameter, a measure of engine health was determined. In one case, for example, this difference averaged 26.8% (of the total range covered by the data) for the eight flights prior to the component replacement. After the component was replaced, the difference averaged 5.4% over the fourteen subsequent flights. This result suggests that neural networks may provide a basis for predictive assessment of engine performance. Extensions of this initial study will involve expanding the training data set, determining more precisely the cause and relationships between performance and repair actions, and exploring alternative architectures.

Optical Character Recognition (OCR) has been considered to be a major breakthrough in man- machine communication. The function of OCR is to recognize previously scanned images that may contain typed, printed, and/or handwritten characters and to output the appropriate text document. A preprocessing stage (segmentation) is first performed on the scanned text to isolate lines from documents, words from lines, and finally characters from words. Immediately following the segmentation stage is the recognition stage in which the isolated characters are first processed for feature extraction and then fed to the classification process which tries to recognize the upcoming character based on the extracted features. In this paper, a recognition stage which consists of a three-layer neural network trained by the back- propagation algorithm is considered in the recognition of different Arabic fonts. Our approach is built around three interacting processes, one procedure for feature extraction of the upcoming character element, one declarative for heuristic clustering, and one exemplar to identify the target element based on some previously learned examples.

A biologically inspired tactile sensor, the AFIT Neural Tactile Sensor (NTS), is proposed and experimentally evaluated. Incorporating ANN pattern recognition techniques, the NTS offers previously unattainable benefits in sensor size, simplicity, flexibility, sensitivity, and utility.

In this paper, we present the performance analysis of three different neural network paradigms, ART-1, ARTMAP inspired ART-1 and Neocognitron, for part recognition in an autonomous assembly system. This intelligent manufacturing system integrates machine vision, neural networks and robotics in order to identify, locate and assemble randomly places components on printed circuit boards requiring precision assembly. The system uses an IBM 7547 robot controlled by an IBM PS/2 computer, a CCD camera and an image capture card. The electronic components are identified and located by using artificial neural networks. The system's component location and identification accuracy are tested on all test components. The results show that the neocognitron-based system performed better than the other two systems.

This paper examines two issues upon any industrial inspection system using a neural network: the feature set which the sensory system must provide and the accuracy of neural based- inspection. The context is web-process inspection which requires rapid examination of vast amounts of data for on-line detection of faults in the sheet material. Feature vectors with nine or 17 dimensions, created by a simulated segmented photodetector using measurement of the angular distribution over a 25 degree(s) cone angle of the scattering were evaluated for inspection of CrO₂ coated sheet steel samples. The scattered coherent light from the surface of the material being processed could be directly conditioned by a photodetector so as to produce this small set of features which are then examined by a neural network trained to find and categorize unsatisfactory surface conditions. details are presented to show how a modified feature set was developed and tested after an examination of feature space. This new, smaller set proved to be more accurate than the larger set. Classification by fault or no fault categorized 133 samples correctly out of 135, while there were seven errors in one attempt at classification into the various common surface faults out of the same number of test samples and nine in another. It is shown that a bit of insight in feature selection can improve the capability of the network to recognize faults.

In this paper, a neural network with the reward/punish learning scheme is used to control manipulator arms. At each discrete point of the work space, one neuron for each joint is assigned to control the movement of the arm. The inputs to the neuron are the position error and the velocity of the joints. The net-input of the neuron, which is the linear combination of the input and its weight is passed through a Sigmoid function to generate the final output. The output of the neuron is the torque required to control the arm to its desired position. The reward/punish learning mechanism is implemented to adaptively modify the weights. The weights are punished if the previous move was in the wrong direction. Otherwise, the weights are rewarded. By doing this iteratively, the network learns the inverse dynamics of the manipulator without knowing its model or forward dynamics. The neurons can finally output appropriate torques to maintain the manipulator arm at a proper location. Due to the simple learning algorithm, the network learns the inverse dynamics quickly. Therefore, it can be used in real-time applications. A two-link planar manipulator is demonstrated in this paper. The position error and the torque generated for each joint are shown graphically. These figures also show that, after the inverse dynamics of the manipulator is learned, the network moves the arm to its desired position quickly after step disturbances of +/- 2.5 degrees were injected into the system. Although only a 2-DOF is illustrated, the concept can be extended to a 6-DOF system.

This paper presents a reliable neural network based system for compact disc serial number inspection. Four major steps are performed in this system, namely polar image conversion, segmentation, recognition, and verification. Polar image conversion straightens the circular stripe images containing the serial numbers to horizontally linear images, segmentation divides the images into separate character segments, recognition classifies the separate character segments by a trained neural network, and verification double checks the characters with their verification neural networks if the serial number is known. Special back-propagation training processes for recognition and verification neural networks are used to increase the system's performance. Excellent recognition results for this system implemented in the Sony SUPER vision system have been obtained. Moreover, this system can process more than 3 compact discs in one second.

Semiconductor wafer fabrication processing is becoming extremely complex as we strive for the continuous reduction of the minimum feature size of devices and for controlling the variability at each of the ever-increasing number of steps. Equipment/process models based on physical and chemical laws are difficult to build due to the process complexities. The non- linearity exhibited by many processes may restrict application of linear or quadratic statistical models to a very small operating range. The same remarks apply for feed-back control of a highly non-linear process. In this work we have generated predictor models using neural network and statistical response surface techniques for the Chemical Vapor Deposition (CVD) silicon epitaxy process. The prediction (generalization) performance of the neural network is appreciable better than both the linear and the quadratic response surface models. The comparative performance of the neural network model is expected to improve even further in representing more complex input-output relationships. We have also determined an inverse process model using neural network. An inverse process model is expected to be useful for determining the process control parameters when a specific output (scaler or vector) is required. It has also helped us identify the critical control parameters for the CVD process.

Rotocraft safety, survivability, and mission effectiveness depend on the structural integrity of dynamic components. The need exists to develop an on-board, continuous vibration diagnostic system to detect and to prognosticate faults in these components prior to failure. This paper overviews a generic fault detection, isolation, and estimation (FDIE) architecture for condition-based machinery maintenance applications. Neural network-based fault pattern recognition is used to analyze normal and defect vibration signatures in helicopter transmissions. Data from nine seeded-fault test-rig experiments, each corresponding to one of six different fault/no fault conditions, were used to train and evaluate polynomial neural networks at pattern classification tasks. Features were generated using the amplitude spectra of the time-series vibration signatures. The Algorithm for Synthesis of Polynomial Networks for Classification (CLASS), a neural network software package that utilizes a constrained, minimum-logistic-loss criterion for multiclass problem, was used to perform the pattern recognition tasks. By employing a multiple-look post-processing strategy, perfect vibration signature classification was achieved.

The present work describes a new technique for the modeling of unsteady aerodynamics using neural networks. Surface pressure readings obtained from an airfoil pitched at constant rate between 0 and 60 degrees were evaluated for 6 different pitch rates and 9 different span locations. Using 5 of 54 records as a training set both a nonlinear and a linear neural network were trained on the time-varying pressure gradients. Thus, post-training, given the pressure distribution at any time (t) the models should predict the pressure distribution at time (t+(Delta) t). In addition, following training a linear equation system was calculated from the weight matrices of the linear neural network. The performance of both the linear equation system and the nonlinear network were evaluated using both sum-squared error and waveform correlations of the predicted and measured data. The results indicated that both models accurately predicted the unsteady flow fields to within 5% of the experimental data. Sum- squared errors were less than 0.01 and correlations were highly significant r > 0.09, (p < 0.01), for all 15 predicted pressure traces in each data set. Further, both models accurately extrapolated to any of the 49 records not used during training. Again, sum-squared errors were less than 0.01 and correlations were highly significant r > 0.90, (p < 0.01), in all cases. Overall, the results clearly indicated that it was possible to predict a wide range of unsteady flow field conditions including novel pitch rates and novel span locations. Further, the results clearly showed that these techniques facilitated the mathematical quantification of these unsteady flow fields. A linear equation system was readily calculated from the linear neural network. The capability to predict this phenomenon across a wide range of flight envelopes in turn provides a critical step towards the development of control systems targeted at exploiting unsteady aerodynamics for aircraft maneuverability enhancement.

This paper presents a research for Artifical Neural Network (ANN) in the process control. It studies the following parts: (1) how the ANN is constructed (improved-BLTA will be proved and explained in detail), (2) the making of a knowledge model, and (3) the integral control system.

The Traveling Salesman Problem (TSP) is a well known problem which can be solved using Hopfield Networks. The TSP solution with Hopfield Networks is based on the uniqueness constraint. This is, each city must be visited once and only once while trying to minimize the traveling distance. But in the real world applications, usually there are other equally important constraints needed to be considered. For example, an ordering constraint in how cities are to be visited. This paper describes a Hopfield neural network that can solve a new class of optimization problems, called 'The Picking Stone Problem (PSP)'. The PSP requires not only the uniqueness but also the ordering constraints. The neural network implementation to solve PSP tends to turn on neurons which satisfy the ordering constraint and this constraint is essential in solving stereo correspondence problem in binocular vision. In this paper we define the PSP, formulate its computational complexity, propose the ordering-oriented neural network architecture, discuss the performance of the proposed network by both the traditional way and a new initialization method, then finally apply the network to incorporate with all the major stereo vision constraints to solve the stereo correspondence problem. The implementation and the performance of the ordering-oriented neural networks are investigated in detail and experimental results applying this technique to solve the stereo correspondence problem on real images are presented.

In this paper we deal with the problem of edge extraction for the purpose of matching to a known model or set of models. We describe an approach to using geometric model based information within a feedback system, without the requirement for prior pose estimation by a matching process. We call this process model driven feedback (MDF). The feedback system uses a chord based transform of the image edges that is invariant either to translation or both translation and rotation, depending on its form. By representing both the data and model information using a geometrically invariant transform, and iteratively minimizing a function of the differences between the model and data transforms, the system is able to eliminate background edges while retaining object edges that are similar in shape to the model.

Recognition of general unconstrained cursive handwriting remains largely unsolved. We present a system for recognizing off-line cursive English text guided in part by global characteristics of the handwriting. A new method for finding the letter boundaries based on minimizing a heuristic cost function is introduced. The function is evaluated at each point along the baseline of the word to find the best possible segmentation points. The algorithm tries to find all the actual letter boundaries and as few additional ones as possible. After a normalization step that removes much of the style variation, the normalized segments are classified by a one hidden layer feedforward neural network. The word recognition algorithms find the segmentation points that are likely to be extraneous and generates all possible final segmentations of the word by either keeping or removing them. Interpreting the output of the neural network as posterior probabilities of letters, it then finds the word that maximizes the probability of having produced the image, over a set of 30,000 words and over all the possible final segmentations. We compared two hypotheses for finding the likelihood of words that are in the lexicon and found that using a Hidden Markov Model of English is significantly less successful than assuming independence among the letters of a word. In our initial test involving multiple writers, 68% of the words were in the top three choices.

A neural network is used to extract relevant features of sign language from video images of a person communicating in American Sign Language or Signed English. The key features are hand motion, hand location with respect to the body, and handshape. A modular hybrid design is under way to apply various techniques, including neural networks, in the development of a translation system that will facilitate communication between deaf and hearing people. One of the neural networks described here is used to classify video images of handshapes into their linguistic counterpart in American Sign Language. The video image is preprocessed to yield Fourier descriptors that encode the shape of the hand silhouette. These descriptors are then used as inputs to a neural network that classifies their shapes. The network is trained with various examples from different signers and is tested with new images from new signers. The results have shown that for coarse handshape classes, the network is invariant to the type of camera used to film the various signers and to the segmentation technique.

Image compression can be achieved by using stochastic artificial neural networks (SANN). The idea is to store an image in stable distribution of a stochastic neural network. Given an input image f (epsilon) F, one can find a SANN t (epsilon) T such that the equilibrium distribution this SANN is the given image f. Therefore, the input image, f, is encoded into a specification of a SANN, t. This mapping from F (image space) to T (parameter space of SANN) defines SANN transformation. To complete a SANN transformation, an SANN equation has to be solved. In this paper, we will first introduce two types of SANN equations. Then, we will develop an algorithm to solve SANN equation.

Single Photon Emission Tomography (SPET) brain images are used to invesügate medical conditions such as Alzheimers disease, carotid artery occlusion, transient ischaemic attack and Basso-spasm. This work concentrates on the results of training a series of Back Propagation Neural Networks to recognise the presence or absence of a Basso-spasm. Network topology variations, in particular tesselated networks are discussed as well as the effects of using Regions of Interest and cascaded networks. Images with simulated abnormalities, additional noise and slight rotational variations have been added to the training set in an attempt to improve generalization. Raw pixel data has been used as network input, results with Principal Component Analysis will be discussed in a future publication. We conclude that a neural network based system could be employed as a diagnostic support tool in the diagnosis of Basso-spasm from SPET images.

Statistical models of neural networks predict that the difference in training and testing error will be linear in network complexity and quadratic in the feature noise of the training set. Models of this kind have been applied to the Boltzmann pruning of a large MLP (3786 weights) trained on 10,000 and tested on 10,000 Karhunen-Loeve (K-L) features sets derived from images of handprinted characters and to a fingerprint classification problem which has 17,157 weights and is trained and tested on 2,000 K-L feature sets. Using the information content to optimize network size, the pruned networks have achieved high rates of recognition and at the same time been reduced in size by up to 90%. In the pruning process the product of the network capacity and the recognition error can be used effectively to select an optimum pruned network. If, in addition to conventional Boltzmann weight reduction, a weight reduction method which takes the variance content of the K-L by weighting the features using the K-L eigenvalues is used, networks with optimal size and information content can be constructed.

It is frequently asked to individuals to prove their identity when writing official documents. This is done to avoid the use of someone else's signature and also to avoid that someone disowns a document that he has previously acknowledged. Texts are often typed, so it is not possible to authenticate these documents from handwriting. However, it is customary to append a mark authenticating the author of the document, thus showing that he agrees with the text of the document. Nowadays this mark is generally a handwritten signature, so it is interesting to devise an automatic and reliable system for the authentication of handwritten signatures appended on the numerous documents which are produced daily. The difficulty of the signature authentication problem is linked to the high number of writers, to the diversity of signatures to store, and also to the important variations between signatures from the same writer [Sabourin 90]. The authentication problem is different from the identification problem because the latter consists in determining the writer from his signature. In the authentication case, we know the writer who is supposed to have signed, as his name is written on the document, for example a check. So it is possible to access in a database to the signatures given by the writer to be used as reference signatures. Then, the authentication process consists in comparing the signature to the reference ones in order to judge if the supposed writer is really the author of the tested signature. The signature authentication can be used in several applications ; let us now focus on the verification of checks from the French Post Office. Our goal is to detect rough forgeries, which are signatures written by someone who is not imitating a genuine signature. Those rough forgeries are the most commonly found forgeries. Systems based on dynamic information (duration, speed of the signing, ...) are able to detect good imitations. In our application however, this dynamic information is lost because the image of the check contains only static information. Without major modifications, the authentication module of our system can be used by authentication systems based on other types of data such as digital fingerprints or dynamic information about the signatures.

We present an application of Radial Basis Function Networks to handwritten digits recognition. Inspired from regularization theory and Parzen windows non parametric estimator. Radial Basis Function networks are tested for a classification task. Reduction of the number of hidden nodes which is an important and necessary step to obtain a computationally tractable network is made using an original technique. A comparison is made with the k- nearest neighbor and Parzen windows methods. Results appear better for the network at a much lower computational cost.

This paper presents results from experiments on determining the sigmoidal transfer function in Intel's Electronically Trainable Analog Neural Network (ETANN), the 80170NX chip. Accurate simulator training off-chip is needed in order to reduce training time and to minimize chip-in-loop training (on-chip), which if done in excess, can decrease the chip's useful life. For this reason accurate characterization of the ETANN chip is of significant importance to application designers for off-chip simulation. A series of tests were performed to collect data from eight ETANN chips for analysis. After computing an average response value from eight chips and performing a minimum mean-square-error search for a gain coefficient (also called a hardness parameter in the sigmoidal function), a transfer function and gain coefficient were found. Using this transfer function and gain coefficient in a custom neural network simulator, called Neural Graphics, a performance evaluation was accomplished. A test shows a direct correlation between the Neural Graphics simulator output and the ETANN chip's output using the same synaptic weights and the test data. Moreover, for this test we have found that Neural Graphics, while using the characterized transfer function from this research, performed in a superior manner to that of iNNTS simulators. For researchers who desire to interface their own custom simulator with the ETANN hardware, a similar procedure as developed in this paper should be followed.

A LCD implementation of a truly analog multilayer perceptron that operates without external control is discussed. The inherent sigmoidal transmissivity transfer function of the LCD is utilized as the neural activation function. The use of photoconductors to directly control the transmissivity of the LCD cells are described, and an accommodation to intensity based negative weights are also noted. A computer simulation program using a curve fitted LCD transfer function (activation function) yielded excellent results in classifying sounds from several endangered and non-endangered species of fish from sonograms of underwater hydrophone recordings. The progress of the hardware implementation is reported.

The non-ideal behavior of analog integrated circuits make it necessary that Artificial Neural Network (ANN) systems be evaluated for the effect of error due to the non-idealities on its performance, before they are implemented in analog hardware. In this paper we describe a procedure for automatically evaluating a given ANN system, described in the form of a Data Flow Graph (DFG). The equations required for the quantitative evaluation are extracted from the DFG description using symbolic computation techniques. Optimization methods are applied for generating bounds on the maximum values of error that can be associated with each circuit block. The generated bounds are put back to behavioral models of individual circuits blocks in the design library, to help screening viable alternatives and to generate circuit level specifications. The methodology forms part of a design automation environment that helps to map ANN systems to hardware interconnection descriptions.

This paper describes a neural network for assigning weapons to targets and compares its execution time on four distinct machines. The network employs more than 46,000 neural elements and more than 49 million connections. It has produced excellent results for a realistic test scenario. Not only has the neural network produced high quality assignments for a realistic test scenario, the neural approach can potentially deliver results in real-time. The machines employed to evaluate the execution speed of the neural algorithm for assigning weapons to targets were: a DEC VAX 8810, a Neural Emulation Tool (NET) neural network accelerator from Loral Corporation, an Intel iPSC/2 Hypercube and a Cray Y-MP4/464.

The main objective of this study is to examine the feasibility of applying feed-forward neural networks to estimate training site vegetation coverage probability based on past disturbance pattern and vegetation coverage history.

A pilot project of the Center involves automatic document delivery in response to computerized Interlibrary loan requests. Each document request includes an unstructured comment field that patrons occasionally use to indicate whether or not they want the National Library of Medicine to fill that request. These comments vary widely in content, but were found to always contain the test 'NLM.' This paper describes a technique to automatically reduce the amount of operator intervention to resolve ambiguities in the intent of the patron as to whether the request should be filled or not.

The graded-response Hopfield neural network model has been used to solve the traveling salesman optimization problem. However, the mapping of an optical design optimization problem onto a neural net is more difficult. This paper describes how it can be done for the case of minimizing the chromatic aberration in a complicated twenty-element zoom-lens system by the selection of glass types. The problem is combinatorial in nature. It is suited to neural networks, and its solution is non-trivial by other means. Thus the use of neural networks to solve optical optimization problems is demonstrated.

The combination of hyperspectral imaging systems and neural networks are changing the approach to the challenging problem of automatic target recognition (ATR). This paper summarizes a research effort to demonstrate the utility of neural networks in processing hyperspectral imagery for target detection and segmentation. Pixel registered imagery containing 32 spectral bands in the 2.0 to 2.5 micrometers range was used to train and test a backpropagation neural network for detection of camouflaged relocatable targets. Initially, neural networks trained and tested using all 32 spectral bands. Because of the high degree of correlation between features (i.e. spectral bands), the dimensionality of the feature set was reduced to 11 spectral bands using both traditional (Karhunen-Loeve) and recently introduced neural network analysis techniques (Ruck's saliency). The neural network was reconfigured and retrained resulting in a probability of correct classification (Pcc) of 99.8%. The neural networks were implemented in hardware on the Intel ETANN chip, a special purpose analog neural network chip. Pixel level classification allows detection and segmentation of targets in parallel. Integrated detection and segmentation (IDS) offers a powerful, alternative approach in an ATR scenario.

The project investigates the character (digit) recognizing ability of the discrete Hopfield neural network model used as a pattern classifier. Through different experiments, we are looking for quantitative factors that influence the recognizing ability of the neural network.

User recognition is examined using neural and conventional techniques for processing speech and face images. This article for the first time attempts to overcome this significant problem of distortions inherently captured over multiple sessions (days). Speaker recognition uses both Linear Predictive Coding (LPC) cepstral and auditory neural model representations with speaker dependent codebook designs. For facial imagery, recognition is developed on a neural network that consists of a single hidden layer multilayer perceptron backpropagation network using either the raw data as inputs or principal components of the raw data computed using the Karhunen-Loeve Transform as inputs. The data consists of 10 subjects; each subject recorded utterances and had images collected for 10 days. The utterances collected were 400 rich phonetic sentences (4 sec), 200 subject name recordings (3 sec), and 100 imposter name recordings (3 sec). Face data consists of over 2000, 32 X 32 pixel, 8 bit gray scale images of the 10 subjects. Each subsystem attains over 90% verification accuracy individually using test data gathered on days following the training data.

Linear and nonlinear adaptive filtering algorithms are described, along with applications to signal processing and control problems such as prediction, modeling, inverse modeling, equalization, echo cancelling, noise cancelling, and inverse control.

In the last decade, much effort has been directed towards understanding the role of chaos in the brain. Work with rabbits reveals that in the resting state the electrical activity on the surface of the olfactory bulb is chaotic. But, when the animal is involved in a recognition task, the activity shifts to a specific pattern corresponding to the odor that is being recognized. Unstable, quasiperiodic behavior can be found in a class of conservative, deterministic physical systems called the Hamiltonian systems. In this paper, we formulate a complex version of Hopfield's network of real parameters and show that a variation on this model is a conservative system. Conditions under which the complex network can be used as a Content Addressable memory are studied. We also examine the effect of singularities of the complex sigmoid function on the network dynamics. The network exhibits unpredictable behavior at the singularities due to the failure of a uniqueness condition for the solution of the dynamic equations. On incorporating a weight adaptation rule, the structure of the resulting complex network equations is shown to have an interesting similarity with Kosko's Adaptive Bidirectional Associative Memory.

In this paper the application and performance of Artificial Neural Networks (ANN) to the problem of sensor data fusion is reported for an experimental system, Tracker. The task of sensor data fusion involves integrating numerous data streams, originating from disparate sensors, into a consistent model that represents the pertinent higher level features of the environment as well as presenting an assessment of their significance. In the case of the modern naval environment, the problem central to many tactical data fusion systems is the need for rapid acquisition and interpretation of the information. In a potentially hostile situation the time taken to perform such an assessment is severely limited and a rapid and accurate response is vital. This paper describes the application of ANN to tactical sensor data fusion and the automated processing of the radar behaviors for various vehicle types. In particular the tasks of target and behavioral identification for both automated surveillance and support tasks are highlighted as important in the modern naval environment. The experimental research program divided the analysis of the radar tracks into three distinct categories. These were (1) target identification, (2) behavioral analysis (target task identification) and (3) threat assessment. A Knowledge Based System (KBS), previously developed by the Defense Research Agency, was used as a comparison. In addition, support functions in the conventional KBS, such as clutter identification, were also evaluated using ANN based technology. The results of this research program are reported in this paper.

In this paper we discuss the means by which recurrent connections are used in neural control system architectures. We first consider the state feedback approach to control and the role of recurrent neural networks for plant modeling and control. In this content, we provide an explicit formation for the computation of dynamic derivatives in recurrent neural network architectures as required for training by the dynamic gradient method. For illustration, we apply dynamic gradient methods to train recurrent neural network controllers for a series of cart-pole problems with the simultaneous objectives of pole balancing and cart centering.

We describe a new theory of differential learning by which a broad family of pattern classifiers (including many well-known neural network paradigms) can learn stochastic concepts efficiently. We describe the relationship between a classifier's ability to generate well to unseen test examples and the efficiency of the strategy by which it learns. We list a series of proofs that differential learning is efficient in its information and computational resource requirements, whereas traditional probabilistic learning strategies are not. The proofs are illustrated by a simple example that lends itself to closed-form analysis. We conclude with an optical character recognition task for which three different types of differentially generated classifiers generalize significantly better than their probabilistically generated counterparts.

Two types of artificial neural networks are introduced for the robust classification of spatio- temporal sequences. The first network is the Adaptive Spatio-Temporal Recognizer (ASTER), which adaptively estimates the confidence that a (variable length) signal of a known class is present by continuously monitoring a sequence of feature vectors. If the confidence for any class exceeds a threshold value at some moment, the signal is considered to be detected and classified. The nonlinear behavior of ASTER provides more robust performance than the related dynamic time warping algorithm. ASTER is compared with a more common approach wherein a self-organizing feature map is first used to map a sequence of extracted feature vectors onto a lower dimensional trajectory, which is then identified using a variant of the feedforward time delay neural network. The performance of these two networks is compared using artificial sonograms as well as feature vectors strings obtained from short-duration oceanic signals.

We investigate the use of a Differential Vector Quantizer (DVQ) architecture for the coding of digital images. An Artificial Neural Network (ANN) is used to develop entropy-based codebooks which yield substantial data compression while retaining insensitivity to transmission channel errors. Two methods are presented for variable bit-rate coding using the described DVQ algorithm. In the first method, both the encoder and the decoder have multiple codebooks of different sizes. In the second, variable bit-rates are achieved by encoding using subsets of one fixed codebook. We compare the performance of these approaches under conditions of error-free and error-prone channels.