In this paper we describe how the processing speed of a radial basis neural network can be performed by the use of field programmable gate arrays (FPGA). The calculation of the very time-consuming exponential function is taken by an optimized CORDIC-processor. We determine the number of the necessary FPGAs and do a processing speed comparison between FPGA and DSP referring to an application in speech recognition.

To fully exploit the real-time computational capabilities of neural networks (NN) -- as applied to image processing applications -- a high performance VMEbus based analog neurocomputing architecture (VMENA) is developed. The inherent parallelism of an analog VLSI NN embodiment enables a fully parallel and hence high speed and high-throughput hardware implementation of NN architectures. The VMEbus interface is specifically chosen to overcome the limited bandwidth of the PC host computer industrial standard architecture (ISA) bus. The NN board is built around cascadable VLSI NN chips (32 X 32 synapse chips and 32 X 32 neuron/synapse composite chips) for a total of 64 neurons and over 8 K synapses. Under software control, the system architecture could be flexibly reconfigured from feedback to feedforward and vice versa, and once selected, the NN topology (i.e. the number of neurons per input, hidden, and output layer and the number of layers) could be carved out from the set of neuron and synapse resources. An efficient hardware-in-the-loop cascade backpropagation (CBP) learning algorithm is implemented on the hardware. This supervised learning algorithm allows the network architecture to dynamically evolve by adding hidden neurons while modulating their synaptic weights using standard gradient-descent backpropagation. As a demonstration, the NN hardware system is applied to a computationally intensive map-data classification problem. Training sets ranging in size from 50 to 2500 pixels are utilized to train the network, and the best result for the hardware-in-the-loop learning is found to be comparable to the best result of the software NN simulation. Once trained, the VMENA subsystem is capable of processing at approximately 75,000 feedforward passes/second, resulting in over twofold computational throughput improvement relative to the ISAbus based neural network architecture.

In this paper, an analog fuzzifier is designed and verified by SPICE simulation. Building blocks of winner-take-all circuits were developed to build the fuzzifier. The fuzzifier generates the membership functions of a fuzzy system by converting the input analog signals to their corresponding predetermined grade of memberships. It takes voltage-mode inputs and produces current-mode outputs for easy manipulations of the output fuzzy signals by the arithematic operations that follow. Furthermore, the fuzzifier is so flexible such that it can be readily adapted into most fuzzy systems, including neural networks, by changing the reference voltages and the number of building blocks used. The design of each building block is relatively simple as it only consists of four PMOS, four resistors and three current sources. The high speed of analog implementation also makes this fuzzifier more attractive than software implementations of the fuzzification process.

Compact, portable systems capable of quickly identifying contaminants in the field are of great importance when monitoring the environment. In this paper, we examine the effectiveness of using artificial neural networks for real-time data analysis of a sensor array. Analyzing the sensor data in parallel may allow for rapid identification of contaminants in the field without requiring highly selective individual sensors. We use a prototype sensor array which consists of nine tin-oxide Taguchi-type sensors, a temperature sensor, and a humidity sensor. We illustrate that by using neural network based analysis of the sensor data, the selectivity of the sensor array may be significantly improved, especially when some (or all) of the sensors are not highly selective.

The signal processing and neural network simulator (SPANNS) is a digital signal processing simulator with the capability to invoke neural networks into signal processing chains. This is a generic tool which will greatly facilitate the design and simulation of systems with embedded neural networks. The SPANNS is based on the Signal Processing WorkSystem^TM (SPW^TM), a commercial-off-the-shelf signal processing simulator. SPW provides a block diagram approach to constructing signal processing simulations. Neural network paradigms implemented in the SPANNS include Backpropagation, Kohonen Feature Map, Outstar, Fully Recurrent, Adaptive Resonance Theory 1, 2, & 3, and Brain State in a Box. The SPANNS was developed by integrating SAIC's Industrial Strength Neural Networks (ISNN) Software into SPW.

In this paper, we propose an artificial neural network (ANN) based buffer allocation scheme for ATM networks. The proposed scheme assumes limited buffer capacity and nonuniform incoming traffic patterns. The effective utilization of limited buffer capacity under nonuniform traffic conditions is a major concern in maintaining acceptable level of quality of service in these high speed communication networks. The proposed ANN based approach takes advantage of the favorable control characteristics of neural networks such as high adaptability and high speed collective computing power for effective buffer utilization. The proposed model uses complete sharing buffer allocation strategy and enhances its performance for high traffic loads and highly asymmetric traffic patterns by regulating the buffer allocation process dynamically via a neural network based controller. In this study, we considered the buffer allocation problem in the context of routing optimization in ATM networks.

The optimal location of wireless transceivers or communicating sensor devices in an urban area and within large human-made structures is considered. The purpose of the positioning of the devices is formation of a distributed network, either in a mesh or hub-spoke topology, that achieves robust connectivity of the nodes. Real-world examples include wireless local area networks (LANs) within buildings and radio beacons in an outdoor mobile radio environment. Operating environments contain both fixed and moving interferers that correspond to both stationary and time-varying spatial distributions of path distortion of stationary and transient fading and multipath delays that impede connectivity. The positioning of the autonomous wireless devices in an area with an unknown spatial pattern of interferers would normally be a slow incremental process. The proposed objective is determination of the spatial distribution of the devices to achieve the maximum radio connectivity in a minimal number of iterative steps. Impeding the optimal distribution of wireless nodes is the corresponding distribution of environmental interferers in the area or volume of network operation. The problem of network formation is posed as an adaptive learning problem, in particular, a self-organizing map of locally competitive wireless units that recursively update their positions and individual operating configurations at each iterative step of the neural algorithm. The scheme allows the wireless units to adaptively learn the pattern distribution of interferers in their operating environment based on the level of radio interference measured at each node by an equivalent received signal strength from wireless units within the node's hearing distance. Two cases are considered. The first is an indoor human-made environment where the interference pattern is largely deterministic and stationary and the units are positioned to form a wireless LAN. The second situation applies to an outdoor urban environment, where a fixed number of units on mobile platforms operating in a random spatial distribution of interferers.

This paper presents a mixed analog/digital circuit design for an adaptive resonance theory (ART) architecture, called the augmented adaptive resonance theory-I neural network (AART1-NN). The circuit is implemented based on the transconductance-mode approach and mixed analog/digital components, in which analog circuits are used to fully incorporate the parallel mechanism of the neural network, whereas digital circuits are used to provide a reduced circuit size as well as more precise multiplication operation. It is shown that the Pspice simulation results of the implemented circuit are in good agreement with the results calculated numerically from the coupled nonlinear differential equations governing the AART1-NN.

In a previous paper, a neural network with a reward/punish learning scheme controller for a manipulator arm was described. The inputs to the torque-generating neuron are the position error and the velocity of the joints. The output of the neuron is the torque required to control the arm to its desired position. The reward/punish learning mechanism is implemented to adaptively modify the weights. The neural network controller does not need a dynamic model of the arm. The dynamics are learned through training. In this paper we describe the hardware/software implementation of the neural network to control the shoulder joint of a Mitsubishi RM501 arm. Once the system was checked for correct operation the following tests were performed: (1) training the arm to hold is position at different angles (10, 40, 70, 100 and 120 degrees). The angle was to hold with very small error, even in the presence of significant disturbances, after a training period that varied from 3 to 12 seconds. (2) Training the arm at 50 degrees and then commanding it to follow a cosine trajectory from 50 to 70 degrees. The maximum error in this test was less than 1% of the desired value.

The optoelectronic devices are the most effective for realization in the form of the vector- matrix multiplier. Proposed optoelectronic neurocomputers (OENC) consist of optical vector- matrix multiplier (OVMM), random access memory (RAM) and electronic control system. There are two variants of realization. The first neurocomputer scheme includes OVMM based on MAOM -- multichannel multifrequency acousto-optic modulator (Bragg cell). MAOM is the fastest up-to-date spatial light modulator. The second neurocomputer is constructed on the basis of planar OVMM (POVMM). Vector-matrix multiplication in POVMM is executed in a very small volume. The POVMM consists of matrix of light emitting diodes and array of linear photodetectors. A special computer program `NEUROEMULATOR' was designed to learn and to test performance of neural networks. Neural networks were trained with gradient and stochastic algorithms. The paper presents results of computer simulation and hardware implementation of neural networks.

We present a control strategy that is able to both slow down and direct the orientation of a satellite subjected to chaotic feedback forces. The experiments show that the neural-based control system is able to do this in real-time even on very inexpensive hardware such as a PC. Although no prior knowledge of the moments of inertia of the satellite are assumed, implicit knowledge of the nature of the problem is exploited in the form of a heuristic control strategy employed in the training process. The controller is adaptive to possibly changing satellite dynamics (e.g. modifications and additions to the satellite such as the one done to the Hubble telescope). The control torques produced by the controller are smooth and the system demonstrates the ability to bring the satellite into a desired orientation in the presence of large chaotic external forces in addition to noise in the sensor system. We contrast our approach to that of conventional heuristic, time-optimal and fuel-optimal control methods as well as earlier similar work using a more general neuro-genetic architecture developed by Dracopoulus and Jones.

Helicopters are highly non-linear systems that have dynamics that change significantly with respect to environmental conditions. The system parameters also vary heavily with respect to velocity. These nonlinearities limit the use of traditional fixed controllers, since they can make the aircraft unstable. The purpose of this paper is to make contributions to the development of an `intelligent' control system that can be applied to complex problems such as this in real- time. Using a slowly changing model and a simplified nonlinear model as examples, a neural network based controller is shown to have the ability to learn from these example plants and to generalize this knowledge for previously unseen plants. The adaptability comes from a neural network that adjusts coefficients of the controller in real-time while running on the accurate automation neural network processor.

Boltzmann machines have been used to solve a variety of combinatorial optimization problems. The use of simulated annealing allows the network to evolve into a state of maximum consensus (or minimum energy) corresponding to an optimal solution of the given problem. In this paper, Boltzmann machine neural networks (without learning) are used to generate initial configurations of assets for a generic game (e.g. chess). The desired distribution of playing pieces is subject to restrictions on the number of pieces (of several different types) that are present, as well as some preferences for the relative positions of the pieces. The rules implemented in the network allow for flexibility in assigning locations for available resources while the probabilistic nature of the network introduces a degree of variability in the solutions generated. The architecture of the network and a method for assigning the weights are described. Results are given for several different examples.

In this paper the noise removal capabilities of a cellular automaton are applied in two different fields. The first application is performed on 4 Gev pion and electron experimental events taken at Cern PS with a silicon tungsten tracking calorimeter. Particle interaction with the material of the calorimeter can produce different interactions resulting in energy releases and topology patterns dependent on the primary particle nature. The evolution rules devised for the CA have therefore to reckon with these different topologies in order to remove noise and restore interrupted tracks. The distributions of some discriminating parameters are compared with Monte Carlo data before and after filtering by the automaton and the agreement is shown to improve if pions are considered. To successfully take into account electromagnetic showers, more than one different evolutionary rule has to be considered. A neural network accordingly trained selects each step of the evolutions closer to the training classes. Upon convergence of these two different `paths,' obtained with dynamic update rules, the image with the highest output results is filtered and classified. The second use of cellular automata is in DNA sequence autoradiograph films. These images may be filtered by a CA which improves nucleotide readability and speeds up sequencing process.

Numerous vision applications rely upon efficient techniques for detecting edges in an image. Edge detection is especially difficult in low-contrast images which are characterized by the general lack of sharp variations in the gray-scale intensity values between objects of interest and their backgrounds. In low-contrast images, the application of commonly employed edge detection algorithms may result in excessive noise. This paper presents a neural network model which enhances edges and reduces noise in low-contrast gray-scale images. A neural element is associated with each pixel in an image. Each neuron receives weighted gray-scale inputs from its immediate neighbors. The weights associated with the gray-scale inputs are determined through a fuzzy compatibility function that grades the degree of similarity between the gray-scale intensity values of neighboring pixels. The neural element sums its weighted inputs and subjects the weighted sum to a sigmoid function that produces gray-scale outputs ranging between 0 and 255. The slope of the sigmoid function is chosen to force resulting pixel values away from mid-range values and closer to either 0 or 255. The resulting image is then subjected to the Sobel edge detection algorithm. The technique is illustrated by applying it to several low-contrast infrared images containing military vehicles. The results show significant noise reduction and edge enhancement.

Broccoli is grown extensively in Scotland, and has become one of the main vegetables cropped, due to its high yields and profits. Broccoli, weed and soil samples from 6 different farms were collected and their spectra obtained and analyzed using discriminant analysis. High crop/weed/soil discrimination success rates were encountered in each farm, but the selected wavelengths varied in each farm due to differences in broccoli variety, weed species incidence and soil type. In order to use only three wavelengths, neural networks were introduced and high crop/weed/soil discrimination accuracies for each farm were achieved.

This paper presents a novel application of neural networks to the problem of classifying software modules into different risk classes based upon source code measures. Neural network models that classify program modules as either high-risk or low-risk are developed. Inputs to these networks include a selection of source code measure data and fault data that were collected from two large commercial systems. The criterion variable for class determination was a quality measure of program faults or changes. Discriminant models using the same data sets provide for a comparative analysis. The neural network technique displayed better classification error rates on both data sets. These successes demonstrate the utility of neural networks in isolating high-risk modules.

It is proposed by Eccles that (1) The mental world consists of mental units called psychons; (2) The brain world consists of nerve units called dendrons; (3) In mind-brain interaction, psychons and dendrons are linked through quantum physics. In this note, we formalize the above ideas. Emotions are represented by emotion state vectors. The motion of emotion (emotion state vector) depends on the previous psychon state vector, the previous emotion and brain state. The brain state is in turn stimulated by outside stimuli and influenced by psychon state vectors. In this paper, we specify the mind-brain interaction by several operators. A complete theory is presented in this paper.

We describe three experiments attempting to model differences in cortical EEG following stimulation with different odors. The data used in these experiments was obtained in previous studies, described briefly here. Subjects sit in an environmentally stabilized low odor cubicle. Twenty-eight electrodes are placed on the scalp and connect the subject to a neurosciences brain imager, which digitizes cortical EEG response. In a given trial, a specific odor is introduced, and the response recorded. In the first experiment, alpha wave data from a subset of ten electrodes and a single subject was used. In the original experiment, the subject was presented with a number of odors and the resulting brain electrical activity was resolved into 16 time slices (5 preceding presentation, 4 during presentation and 7 following presentation). Only data from frames 6, 7 and 8 (during presentation) was used here. A model was constructed to discriminate morning from afternoon responses. The network used measurements from 10 electrodes as input, and backpropagation was used for training. During training, the network was presented with responses to just one odor. Generalization was demonstrated for five other odors. The weights in the network have been analyzed and indicate a role for a specific group of electrode sites in this discrimination. The second experiment involved constructing a network to discriminate cortical EEG responses to two odors. In the original experiment from which we drew our data, fourteen subjects were presented with each odor once. Data from only the frame at first presentation of the odor were used here. Data from three subjects (chosen pseudo-randomly) was selected for use in the generalization phase and dropped from the training set. Output targets were constructed that took account of subjective ratings of `pleasantness.' A feed-forward network with twenty-eight input units was trained using data from the eleven remaining subjects, using conjugate gradient descent.

This article presents a practical application of artificial multi-layer perceptron (MLP) neural networks in neurosciences. The data that are processed are labeled data from the visual analysis of electrical signals of human sleep. The objective of this work is to automatically classify into sleep stages the electrophysiological signals recorded from electrodes placed on a sleeping patient. Two large data bases were designed by experts in order to realize this study. One data base was used to train the network and the other to test its generalization capacity. The classification results obtained with the MLP network were compared to a type K nearest neighbor Knn non-parametric classification method. The MLP network gave a better result in terms of classification than the Knn method. Both classification techniques were implemented on a transputer system. With both networks in their final configuration, the MLP network was 160 times faster than the Knn model in classifying a sleep period.

Behavioral data show that echolocating bats can discriminate between different speeds of a moving object. A neural network system presented with echoes of simulated bat sounds recorded from targets moving at different speeds exhibits similar performance. In both cases, discriminations were successfully made between targets moving at 30 Hz and a slower variable speed (5, 10, and 20 Hz). However, in both cases, it is not clear what underlying concept was learned and used to perform the task. In other words, did the bat and the neural network perform the tasks based on a relative concept of faster (and thus learned to recognize `this is the faster speed') or did they perform the task based on an absolute concept of 30 Hz speed (and thus learned to recognize `this is the 30 Hz speed')? Both concepts would produce the observed performance of the bat and the neural networks. In this paper, we developed an approach for using neural networks to explore which concept was used to perform the task, and what factors may influence which type of concept is developed. First, we observed the behavior of the neural network when presented with a novel speed of 40 Hz. Its behavior on the novel 40 Hz speed suggested what underlying concept was implemented in the system (if it classified the novel 40 Hz as `30 Hz,' then it used a relative concept of faster; if it classified the novel 40 Hz speed as `not-30 Hz,' then it used an absolute concept, specifically quantifying the 30 Hz speed). Second, we examined what factors influenced the formation of the concept. Our computer simulations of neural networks that discriminated between 30 Hz and the slower variable speeds of 5, 10, and 20 Hz showed that the formation of a concept depends on the magnitude of the difference between the speeds used in training, i.e., the difficulty of the task. Specifically, when the difference between the speeds in the training set was large, the system formulated a relative concept of fast and slow. However, when the difference was small, the system formulated an absolute concept. Our neural network simulation of a system that learns to discriminate between a variety of speeds (i.e., 5, 10, and 20 Hz vs. 30 Hz speeds) demonstrated that the network developed the simplest concept that allowed it to perform the task correctly. Taken together, our results show that the development of concepts is influenced by the underlying computational demands of the task, and that neural networks can be used to explore the processes involved in concept formation.

This paper presents a new recursive neural network to solve optimization problems. It is made of binary neutrons with feedbacks. From a random initial state, the dynamics alternate successively several pulsation and constraint satisfaction phases. Applying the previous neural network has the following advantages: (1) There is no need to precisely adjust some parameters in the motion equations to obtain good feasible solutions. (2) During each constraint satisfaction phase, the network converges to a feasible solution. (3) The convergence time in the constraint satisfaction phases is very fast: only a few updates of each neuron are necessary. (4) The end user can limit the global response time of the network which regularly provides feasible solutions. This paper describes such a neural network to solve a complex real time resource allocation problem and compare the performances to a simulated annealing algorithm.

This paper presents a general methodology for the development of fuzzy algorithms for learning vector quantization (FALVQ). These algorithms can be used to train feature maps to perform pattern clustering through an unsupervised learning process. The development of FALVQ algorithms is based on the minimization of a fuzzy objective function, formed as the weighted sum of the squared Euclidean distances between an input vector, which represents a feature vector, and the weight vectors of the map, which represent the prototypes. This formulation leads to the development of genuinely competitive algorithms, which allow all prototypes to compete for matching each input. The FALVQ 1, FALVQ 2, and FALVQ 3 families of algorithms are developed by selecting admissible generalized membership functions with different properties. The efficiency of the proposed algorithms is illustrated by their use in codebook design required for image compression based on vector quantization.

There is currently much interest in the application of artificial neural network (ANN) technology to the field of on-line machine condition monitoring (CM) for complex electro- mechanical systems. In this paper the authors discuss, with the help of an industrial case study, a few of the difficulties inherent in the application of neural network based condition monitoring. A method of overcoming these difficulties by utilizing a combination of human knowledge, encoded using techniques borrowed from fuzzy logic, Kohonen neural networks, and statistical K-means clustering has been constructed. The methodology is discussed in the paper by means of a direct comparison between this new approach and a purely neural approach. An analysis of other situations where this approach would be applicable is also presented and the paper discusses other current research work in the area of hybrid AI technologies which should assist further with the alleviation of the problems under consideration.

An adaptive system is described which generates and trains neural networks using genetic algorithms. A genetic algorithm optimizes the network architecture trying to use as few connections as possible. The neurons of the networks generated by this algorithm are not necessarily organized in layers (except input and output). Because of this, classical algorithms for training neural networks can not be used. Therefore a second genetic algorithm is used to optimize the weights for each generated architecture. During simulation it is possible to change the parameters for the genetic algorithms like the mutation probability or the population size, the size of the networks generated as well as the desired size of the input and output layer and even the data used for training the networks. Therefore the system is able to adapt to a changing environment. The system generates C/C++ code for a `recall only' version of the best network found.

A method of translating a two-way table of qualified symptom/cause relationships into a four layer expert network for diagnosis of machine or sample preparation failure for gas chromatography is presented. This method has proven to successfully capture an expert's ability to predict causes of failure in a gas chromatograph based on a small set of symptoms, derived from a chromatogram, in spite of poorly defined category delineations and definitions. In addition, the resulting network possesses the advantages inherent in most neural networks: the ability to function correctly in the presence of missing or uncertain inputs and the ability to improve performance through data-based training procedures. Acquisition of knowledge from the domain experts produced a group of imprecise cause-to-symptom relationships. These are reproduced as parallel pathways composed of symptom-filter-combination-cause node chains in the network representation. Each symptom signal is passed through a filter node to determine if the signal should be interpreted as positive or negative evidence and then modified according to the relationship established by the domain experts. The signals from several processed symptoms are then combined in the combination node(s) for a given cause. The resulting value is passed to the cause node and the highest valued cause node is then selected as the most probable cause of failure.

In this paper we present a neural generator method that uses a neural network to generate initial search points for a discrete heuristic. We demonstrate the method for the subset-sum problem (SSP), and consider the SSP to be typical of the sub-problems that a scheduling algorithm must solve while on route to solving an entire scheduling problem. The neural generator method hinges on using the continuous valued activations of the neural system to select a corner of the n-cube that can be used to initialize a discrete search. This can be done at each neural iteration, resulting in many discrete searches over the source of a single neural run. Without the discrete heuristic, the selected corners can be interpreted as instantaneous neural solutions and the best-so-far tabulated as the neural system runs. This allows the neural system to be terminated without losing the full effort of the run, and should the network be run until convergence, the best-so-far result is at least as good as the convergent corner, and usually better. With the discrete heuristic, a search is launched from the instantaneous neural solutions, greatly improving the overall results (again keeping the best-so-far). The results are presented for an n equals 25 SSP.

In this study an attempt is being made to encode the architecture of a neural network in a chromosome string for evolving robust, fast-learning, minimal neural network architectures through genetic algorithms. Various attributes affecting the learning of the network are represented as genes. The performance of the networks is used as the fitness value. Neural network architecture design concepts are initially demonstrated using a backpropagation architecture with the standard data set of Rosenberg and Sejnowski for text to speech conversion on Adaptive Solutions Inc.'s CNAPS Neuro-Computer. The architectures obtained are compared with the one reported in the literature for the standard data set used. The study concludes by providing some insights regarding the architecture encoding for other artificial neural network paradigms.

Many applications reported in artificial neural networks are associated with military problems. This paper reviews concepts associated with the processing of military data to find and recognize targets -- automatic target recognition (ATR). A general-purpose automatic target recognition system does not exist. The work presented here is demonstrated on military data, but it can only be considered proof of principle until systems are fielded and proven `under- fire.' ATR data can be in the form of nonimaging one-dimensional sensor returns, such as ultra-high range-resolution radar returns (UHRR) for air-to-air automatic target recognition and vibration signatures from a laser radar for recognition of ground targets. The ATR data can be two-dimensional images. The most common ATR images are infrared, but current systems must also deal with synthetic aperture radar (SAR) images. Finally, the data can be three-dimensional, such as sequences of multiple exposures taken over time from a nonstationary world. Targets move, as do sensors, and that movement can be exploited by the ATR. Hyperspectral data, which are views of the same piece of the world looking at different spectral bands, is another example of multiple image data; the third dimension is now wavelength and not time. ATR system design usually consists of four stages. The first stage is to select the sensor or sensors to produce the target measurements. The next stage is the preprocessing of the data and the location of regions of interest within the data (segmentation). The human retina is a ruthless preprocessor. Physiologically motivated preprocessing and segmentation is demonstrated along with supervised and unsupervised artificial neural segmentation techniques. The third design step is feature extraction and selection: the extraction of a set of numbers which characterize regions of the data. The last step is the processing of the features for decision making (classification). The area of classification is where most ATR related neural network research has been accomplished. The relation of neural classifiers to Bayesian techniques is emphasized along with the more recent use of feature sequences to enhance classification.

A new classifier neural network is described for distortion-invariant multi-class pattern recognition. The input analog neurons are a feature space. All distorted aspect views of one object are described by a trajectory in feature space. Classification of test data involves calculation of the closest feature space trajectory. Pose estimation is achieved by determining the closest line segment on the closest trajectory. Rejection of false class clutter is demonstrated. Comparisons are made to other neural network classifiers, including a radial basis function and a new standard backpropagation neural net. The shapes of the different decision surfaces produced by our feature space trajectory classifier are analyzed.

The U.S. Navy has been interested in applying neural network processing architectures to automatically determine the naval class of ships from an inverse synthetic aperture radar (ISAR) on-board an airborne surveillance platform. Currently an operator identifies the target based on an ISAR display. The emergence of the littoral warfare scenario, coupled with the addition of multiple sensors on the platform, threatens to impair the ability of the operator to identify and track targets in a timely manner. Thus, on-board automation is quickly becoming a necessity. Over the past four years the Opto-Radar System Group at MIT Lincoln Laboratory has developed and fielded a neural network based automatic ship classification (ASC) system for ISAR imagery. This system utilizes imagery from the APS-137 ISAR. Previous related work with ASC systems processed either simulated or real ISAR imagery under highly controlled conditions. The focus of this work was to develop a ship classification system capability of providing real-time identification from imagery acquired during an actual mission. The ship classification system described in this report uses both neural network and conventional processing techniques to determine the naval class of a ship from a range- Doppler (ISAR) image. The `learning' capability of the neural network classifier allows a single naval class to be distributed across many categories such that a degree of invariance to ship motion is developed. The ASC system was evaluated on 30 ship class database that had also been used for an operational readiness evaluation of ISAR crews. The results of the evaluation indicate that the ASC system has a performance level comparable to ISAR operators and typically provides a significant improvement in throughput.

In signal processing, artificial neural networks (ANN) have been found to be very useful in solving pattern recognition and classification problems. In this application, the performance of ANNs depends, to a large extent, on the quality of features extracted from the given signal. The features, most often, are extracted using conventional signal processing techniques. In this paper, the feature extraction of radar returns is carried out through the use of neural networks and the final recognition of radar targets is carried out by a second stage neural network. Thus feature extraction and classification of experimental radar targets using feed forward and self organizing neural networks are demonstrated.

Simple recurrent networks have been widely used in temporal processing applications. In this study we investigate temporal generalization of simple recurrent networks, drawing comparisons between network capabilities and human characteristics. Elman networks were trained to regenerate temporal trajectories sampled at different rates, and then tested with trajectories at both the trained sampling rates and at other sampling rates. The networks were also tested with trajectories representing mixtures of different sampling rates. It was found that for simple trajectories, the networks show interval invariance, but not rate invariance. However, for complex trajectories which contain greater contextual information, these networks do not seem to show any temporal generalization. Similar results were also obtained employing measured speech data. Thus, these results suggest that this class of networks exhibits severe limitations in temporal generalization.

A novel neural network is proposed for the dynamic classification of spatio-temporal signals. The network is designed to classify signals of different durations, taking into account correlations among different signal segments. Such a network is applicable to SONAR and speech signal classification problems, among others. Network parameters are adapted based on the biologically observed habituation mechanism. This allows the storage of contextual information, without a substantial increase in network complexity. Experiments on classification of high dimensional feature vectors obtained from Banzhaf sonograms, demonstrate that the proposed network performs better than time delay neural networks while using a less complex structure. A mathematical justification of the capabilities of the habituation based mechanism is also provided.

In this paper we investigate the learning properties of ART1, Fuzzy Art, and ARTMAP architectures. These architectures were introduced by Carpenter and Grossberg over the last eight years. some of the learning properties discussed in this paper involve characteristics of the clusters formed in these architectures while other learning properties concentrate on how fast it will take these architectures to converge to a solution for the type of problems that are capable of solving. This latter issue is very important in the neural network literature, and there are very few instances where it has been answered satisfactorily.

This paper presents an alternating minimization (AM) algorithm used in training radial basis function (RBF) networks. The AM algorithm can also be viewed as a competitive learning paradigm. Its use is illustrated by optimizing a colony-style control system. The application arises in the context of hybrid control systems. The algorithm is a modification of a small-step interior point method used in solving primal linear programs. The algorithm has a convergence rate of 0((root)nL) iterations where n is a measure of the network size and L is a measure of the resulting solution's accuracy.

The idea of storing memory states or patterns to be recognized or recalled as static attractors of the neural network dynamics implies that initial configurations of neurons in some neighborhood of a memory stated will be attracted to it. In applications, these attractors could represent memories, pattern classes, or stable control actions. A novel property of autonomous spatio-temporal pattern recognition and production with the Adaptive Time-delay Neural Network (ATNN) has been explored. The ATNN, a paradigm for training a nonlinear neural network with adaptive time-delays and weights, has a rich repertoire of capabilities that are used to perform signal production, and to learn repetitive spatial motions. ATNN has the property that initial segments on signals that contain large amounts of noise can be "cleaned up" to result in trained trajectory motions. The results on noise removal suggest that the trajectories trained into the ATNN networks are in fact attractors. We conducted experiments on the basins of attraction for the circle and figure eight attractors in these networks. The circular and figure eight trajectories can be considered limit cycle attractors or multistate oscillations because of the repetition of points along the closed figures. In spite of the fact that different initial arcs were used, including very noisy arcs, when starting the network trajectory, the nets always arrived at the trajectory for which they were trained. Thus the initial arcs were within the basin of attraction for the trained figures (attractors) both for the circle and figure eight. The impact of this result is that an autonomous or controlled system, such as a robotic arm, or other moving object, could be trained to generate repetitive desired motions, and could attain this repetitive motion from arbitrary starting trajectories.

Complex pattern recognition problems are not usually solved by a single classifier. Multiple classifiers are used instead. Typically, the classifiers are arranged hierarchically, such that the low level classifier (LLC) produces not only a single decision of its best guess of the class of the input pattern but a list of choices ranked according to their likelihood. The high level classifier (HLC) then chooses from this set of classes using additional information that is not usually available to or well represented in a single LLC, such as knowledge of the context or the model. Training neural networks (NNs) as low level classifiers has been traditionally performed independent of what the HLC may do. The traditional performance measure for evaluating classifiers is the classification counting function which counts the number of correct classifications performed by that classifier. It is of course desired that the LLC produces the correct classification (by ranking the correct class as the top choice). Moreover, it is preferred that the LLC ranks the correct class as the second choice if it is not able to correctly classify it (as the first choice). A new cost function (which accounts for the correctness of class rankings) is presented. When this cost function is optimized, it will achieve this desired ranking performance. The parameters of this new cost function will be linked to statistical parameters of our proposed hierarchical model. Unfortunately, this cost function cannot be used to train neural networks because it is not differentiable. Thus we investigate differentiable approximations that are well suited to training NNs using the backpropagation algorithm. Initial simulation results show that superiority of this new error measure over the traditional mean square error measure both in terms of classification and ranking performance.

Recently, neural network technology has been applied to various actual problems. It has succeeded in producing a large number of intelligent systems. In this article, we suggest that it could be applied to the field of gaming. In particular, we suggest that the neural network model could be used to mimic players' characters. Several computer simulation results using a computer gaming system which is a modified version of the COMMONS GAME confirm our idea.

A distributive neural control system is advocated for flexible multibody structures. The proposed neural controller is designed to achieve trajectory slewing of a structural member as well as vibration suppression for precision pointing capability. The motivation to support such an innovation is to pursue a real-time implementation of a robust and fault tolerant structural controller. The proposed control architecture which takes advantage of the geometric distribution of piezoceramic sensors and actuators has provided a tremendous freedom from computational complexity. In the spirit of model reference adaptive control, we utilize adaptive time-delay radial basis function networks as a building block to allow the neural network to function as an indirect closed-loop controller. The horizon-of-one predictive controllers cooperatively regulates the dynamics of the nonlinear structure to follow the prespecified reference models asymptotically. The proposed control strategy is validated in the experimental facility, called the Planar Articulating Controls Experiment which consists of a two-link flexible planar structure constrained to move over a granite table. This paper addresses the theoretical foundation of the architecture and demonstrates its applicability via a realistic structural test bed.

Most of the pattern recognition applications of multilayer neural networks have been concerned with classification and not rejection of a given pattern. For example, in character recognition all alphabetical characters must be recognized as one of the 26 characters, as there is nothing to reject. However, in many situations, there is no guarantee that all the patterns that will be presented to the network would actually belong to one of the classes on which the network has been trained. In such cases, a useful network must be capable of rejection as well as classification. In this paper we propose a scheme to develop multilayer networks with rejection capabilities. The discriminating power of the proposed technique appears to be comparable to that of the human eye.

The learning method of layered neural networks can be supervised or unsupervised. Back propagation learning algorithm is a common method of supervised learning that can learn automatically from teaching patterns. However, accurate teaching patterns are not always available for robotic applications and it is necessary to devise a method of producing them. In this paper, two applications of neural network for self-supervised learning are described. One is a system for which a mobile robot learns its behavior by automatically generating and self- evaluating teaching data through a random walk. The other is a control method of an inverted pendulum using a knowledge-based neural network. The system collects the state data of the inverted pendulum such as angles and angular velocities by trial and error. After that, the system generates teaching data by comparing the collected data with stored knowledge. This knowledge expresses the ideal status of the inverted pendulum when it inverts. The system learns from the generated teaching data and the pendulum inverts stably after some trial and error. In both systems, the neural network learns the teaching data that is generated by the system itself.

In this paper, recurrent neural networks are analyzed from the point of view of sequential machines. Their dynamical behavior is described by a system of coupled Boolean equations, and stability results are presented. The typical stability analysis for recurrent Hopfield-type neural nets is to define an energy function and demonstrate that it is a Liaponov function for the system. This analysis works well for single layer networks, but has not been successfully applied to multilayer networks; although, in theory, an energy function for multi-layered nets may be possible to derive. Alteratively, the stability results presented in this paper are applicable to single layer as well as multilayer recurrent networks. Furthermore, our approach is potentially more systematic and easier to apply than the ad-hoc energy function synthesis methods. As an application of this approach, we show how to design recurrent neural nets to design high performance associative neural memories.

We describe a system based on neural networks that is designed to recognize speech transmitted through the telephone network. Context-dependent phonetic modeling is studied as a method of improving recognition accuracy, and a special training algorithm is introduced to make the training of these nets more manageable. Our system is designed for real-world applications, and we have therefore specialized our implementation for this goal; a pipelined DSP structure and a compact search algorithm are described as examples of this specialization. Preliminary results from a realistic test of the system (a field trial for the U.S. Census Bureau) are reported.

First, we identify an algorithmic defect of the generalized learning vector quantization (GLVQ) scheme that causes it to behave erratically for a certain scaling of the input data. We demonstrate the problem using the IRIS data. Then, we show that GLVQ can behave incorrectly because its learning rates are reciprocally dependent on the sum of squares of distances from an input vector to the node weight vectors. Finally, we propose a new family of models -- the GLVQ-F family -- that remedies the problem. We derive algorithms for competitive learning using the GLVQ-F model, and prove that they are invariant to all positive scalings of the data. The learning rule for GLVQ-F updates all nodes using a learning rate function which is inversely proportional to their distance from the input data point. We illustrate the failure of GLVQ and success of GLVQ-F with the ubiquitous IRIS data.

We introduce a supervised learning method for feed-forward networks that solves the credit assignment problem for error in concert with solving the error reduction problem normally associated with methods such as backpropagation. The method reverberates between forward and reverse activations of the network. Forward activation using an exemplar computes output for each node in the network using the connection weights as usual. Reverse activation using output error as input computes local error at each node using reverse weights, or responsibilities, on the reverse connections. Reverse-reverse activation (the same as forward activation with linear output functions) using reverse output error as input computes local reverse error at each node. Once local error and local reverse error have been assigned to each node, weights and responsibilities are modified using the standard delta rule and local error and local reverse error, respectively. The method relies on convergence toward an optimal set of responsibilities for reverse error distribution in concert with convergence toward an optimal set of weights, and thus avoids calculation of nonlinear terms in the usual error backpropagation method. Thus the method is free of derivative evaluations, and by allowing credit assignment to optimize simultaneously with error reduction, it promotes clustering of responsibility among the nodes.

Real-world applications of neural networks often involve simulation and clustering. Reduction of subjective decisions and increasing the potential for real-time automation of cluster evaluation is a target of the cluster check (CC) method suggested here. CC quantitatively assess the variation within a cluster, produces a `central' pattern for a cluster which is robust in the presence of wide variation and skewed data, and suggests a measure for the distance between clusters. Such a measure of the effectiveness of a segmentation scheme is helpful in many applications, including traditional analysis, neural systems, fuzzy systems and evolutionary systems. This work reports successful use of the CC and companion analytic procedures to measure the consistency of movement of neuroanatomical tracer down neural pathways associated with injection sites (tract tracing). Opposite injection sites produce distinctive L shaped accumulations of tracer in different locations. Consistency of pathways for particular injection sites varies from 0.10 to 0.20 out of a possible 0.80. The pathway rejected by the nonparametric statistics and subdivided by Kohonen's self organizing map (SOM) measures 0.20. These quantitative results are consistent with the expert qualitative inspection traditionally accepted in the study of neuronanatomy of the rat olfactory bulb and tubercle. This work suggests further application of the CC and companion techniques to fault detection, identification and recovery of systems for control of diabetes and systems for control of missiles. Use of managerial decisions in the supervisory portions of these systems may also be facilitated by the consistency measure and distance metric allowing reinforcement of consistent decisions and focus on areas needing reconsideration. Automation of such procedures may facilitate real-time, robust and fault-tolerant control by adding a capability for evaluation needed for automated reinforcement and/or selection in neural, fuzzy and evolutionary systems.

Several researchers have experimentally shown that substantial improvements can be obtained in difficult pattern recognition problems by combining or integrating the outputs of multiple classifiers. This paper provides an analytical framework that quantifies the improvements in classification results due to linear combining. We show that combining networks in output space reduces the variance of the actual decision region boundaries around the optimum boundary. In the absence of network bias, the added classification error is directly proportional to the boundary variance. Moreover, if the network errors are independent, then the reduction in variance boundary location is by a factor of N, the number of classifiers that are combined. In the presence of network bias, the reductions are less than or equal to N, depending on the interaction between network biases. We discuss how the individual networks can be selected to achieve significant gains through combining, and support them with experimental results on 25-dimensional sonar data. The analysis presented here facilitates the understanding of the relationships among error rates, classifier boundary distributions, and combining in output space.

Since the widespread recognition of the capacity for neural networks to perform general function approximation, a variety of such mapping functions have been used to address difficult problems in pattern recognition, time series forecasting, automatic control, image compression, and other engineering applications. Although these efforts have met with considerable success, the design and training of neural networks have remained much of an art, relying on human expertise, trial, and error. More recently, methods in evolutionary computation, including genetic algorithms, evolution strategies, and evolutionary programming, have been used to assist in and automate the design and training of neural networks. This presentation offers a review of these efforts and discusses the potential benefits and limitations of such combinations.

It is well-known that neural network systems have many typical properties compared with other information processing methods. Some of them are learning ability, generalization, and nonlinear characteristics. We propose a neuro-control method to construct adaptive control systems. Based on three kinds of neuro-control methods, we construct a corresponding neuro- controller to each method. Then we apply the method to a temperature control problem for a water bath and a heating furnace. Experimental results show that the present method provides excellent control results.

This paper describes how the similarities and differences among similar objects can be discovered during learning to facilitate recognition. The application domain is single views of flying model aircraft captured in silhouette by a CCD camera. The approach was motivated by human psychovisual and monkey neurophysiological data. The implementation uses neural net processing mechanisms to build a hierarchy that relates similar objects to superordinate classes, while simultaneously discovering the salient differences between objects within a class. Learning and recognition experiments both with and without the class similarity and difference learning show the effectiveness of the approach on this visual data. To test the approach, the hierarchical approach was compared to a non-hierarchical approach, and was found to improve the average percentage of correctly classified views from 77% to 84%.

This research explores the use of Neural Networks (NNs) to implement Automatic Fingerprint Matching (AFM) system. A new three stage architecture, consisting of a preprocessor, feature finder, and matcher stage, is shown to be successful for fingerprint matching. The NN-based AFM system used 20 fingerprints as its training set and 80 fingerprints as its test set. By dividing a fingerprint into 256 16 x 16 pixel blocks, the achieved success matching rates are 95% on the training set and 93.75% on the test set. The Feature Finder based on the Counter Propagation NN realized a high dimension reduction ratio of 256: 1. It finds a feature vector of 256 bytes from a digitized fingerprint of 512 x 512 pixels with 8-bit grayscale. This system also achieved 91.67% matching success m the cross-iferenced fingerprints. Keywords: neural networks, fingerprints, automatic fingerprint matching systems

From the visual stimuli of disparity given for occluded objects only partially along the contour of an occluding object which is not physically depicted, the human visual system can perceive an entire 3-D illusory occluding object. There are several new findings concerning this illusion: an interaction between several illusory objects; an edge line perceived as an intersection of two illusory surfaces; an occlusion between illusory objects; and a transparency which is unique for binocular fusion. Two types of occlusion cues for 3-D perception are postulated and investigated. In further investigations of these illusions with motion, the author found new types of visual effects, which are named as `dynamic illusion' and could not be expected from the stational case. These newly found illusory phenomena have close relations with the visual functions of the 3-D space perception and can propose new subjects not only in the field of psychophysics and neuro-physiology but also artificial neural networks and computer vision.

The demand for intensive computational power for real time information processing applications is rapidly increasing. These information systems are anticipated to support tera operations-per-second in 1996 and beyond. Artificial neural networks represent one of the approaches to enhance computational capabilities in real-time information processing. Analog or mixed digital/analog VLSI neural network processors have been widely developed and are preferred over digital solutions particularly for dedicated applications. We, therefore, see a strong connection between advances in analog VLSI design and the rate of progress of application driven neural network hardware. This paper is devoted primarily to advances and trends in analog VLSI, and discusses the challenges as well as the tremendous potential of analog VLSI neural nets to address real world problems. We also present an interdisciplinary view on VLSI in general and on analog VLSI in particular.