Traditional imaging systems focus on converting light's intensity and color property into suitable electronic signals. An important property of light, polarization is ignored with these traditional imaging systems. Polarization vision contains information about the imaged environment, such as surface shapes, curvature and material properties. Real time extraction of polarization properties would further allow synergy with traditional adaptive spatiotemporal image processing techniques for synthetic imaging. Therefore, we have developed an image sensor with real-time polarimetric extraction capability at the focal plane using low power analog circuits. This novel imaging system is the first of its kind to compute Stokes parameters at the focal plane in real-time. In order to fully describe the polarization state of light in nature, three linear polarized projections or two linear polarized projections in combination with the total intensity are required. We have fabricated a two layer micro polarizer array with total thickness of around 20μm. The micro polarizer array is mounted on top of the imaging sensor. The image sensor is composed of a 256 by 256 photo pixel array, noise suppression circuitry and analog processing circuitry for polarimetric computation. The image sensor was fabricated in 0.18μ process with 10μm pixel pitch and 75% fill factor. Block-parallel pixel read out is employed in order to compute Stokes parameters on a neighborhood of 2 by 2 pixels. The Stokes parameters are presented together with the noise suppressed intensity image. Experimental data from the polarimetric imaging system is also presented.
Two systems for velocity-based visual target tracking are presented. The first two computational layers of both implementations are composed of VLSI photoreceptors (logarithmic compression) and edge detection (difference-of-Gaussians) arrays that mimic the outer-plexiform layer of mammalian retinas. The subsequent processing layers for measuring the target velocity and to realize smooth pursuit tracking are implemented in software and at the focal plane in the two versions, respectively. One implentation uses a hybrid of a PC and a silicon retina (39 X 38 pixels) operating at 333 frames/second. The software implementation of a real-time optical flow measurement algorithm is used to determine the target velocity, and a closed-loop control system zeroes the relative velocity of the target and retina. The second implementation is a single VLSI chip, which contains a linear array of photoreceptors, edge detectors and motion detectors at the focal plane. The closed-loop control system is also included on chip. This chip realizes all the computational properties of the hybrid system. The effects of background motion, target occlusion, and disappearance are studied as a function of retinal size and spatial distribution of the measured motion vectors (i.e. foveal/peripheral and diverging/converging measurement schemes). The hybrid system, which tested successfully, tracks targets moving as fast as 3 m/s at 1.3 meters from the camera and it can compensate for external arbitrary movements in its mounting platform. The single chip version, whose circuits tested successfully, can handle targets moving at 10 m/s.
A method for optical pattern recognition which is based on the human visual system and is suitable for hardware implementation is presented. The system is composed of two stages. The first stage detects local features such as line orientation, linestops, corners, and intersections to create a feature map, which represents the number of these features and hence is invariant to position, size, and slight deformation of an input pattern. The next stage is a multilayered neural network that classifies an input pattern to one of predefined categories using the feature map. We have found a method of detecting these features in analog hardware which would considerably speed up the process of pattern recognition. The decomposition of an input pattern into lines with different orientations is done by an array of two-dimensional orientation sensors. We have built an orientation sensor which is invariant to the position, size, and contrast of an input pattern. The generation of the feature map is currently being done in software which receives its inputs from the line orientation sensor. Linestops, corners and intersections are detected after a series of convolution and thresholding operations for each orientation. The convolution operation can be mapped into hardware using a resistive grid technique. The simulation with an example of character recognition showed that the proper selection of convolution kernels and thresholds can detect local features described above and demonstrated the feasibility of a full hardware implementation of a feature detector.
A prototype programmable analog neural computer has been assembled from over 100 custom VLSI modules containing neurons, synapses, routing switches, and programmable synaptic time constants. The modules are directly interconnected and arbitrary network configurations can be programmed. Connection symmetry and modular construction allow expansion of the network to any size. The network runs in real time analog mode, but connection architecture as well as neuron and synapse parameters are controlled by a digital host. Network performance is monitored by the host through an A/D interface and used in the implementation of learning algorithms. The machine is intended for real time, real world computations. In its current configuration maximal speed is equivalent to that of a digital machine capable of 1011 FLOPS. The programmable synaptic time constants permit the real time computation of temporal patterns as they occur in speech and other acoustic signals. Several applications involving the dynamic decomposition and recognition of acoustical patterns including speech signals (phonemes) are described. The decomposition network is loosely based on the primary auditory system of higher vertebrates. It extracts and represents by the activity in different neuron arrays the following pattern primitives: frequency, bandwidth, amplitude, amplitude modulation, amplitude modulation frequency, frequency modulation, frequency modulation frequency, duration, sequence. The frequency tuned units are the first stage and form the input space for subsequent stages that extract the other primitives, e.g., bandwidth, amplitude modulation, etc., for different frequency bands. Acoustic input generates highly specific, relatively sparse distributed activity in this feature space, which is decoded and recognized by units trained by specific input patterns such as phonemes or diphones or active sonar patterns. Through simple feedback connections in conjunction with synaptic time constants the neurons can be transformed into spiking units resembling biological neurons and networks of such units can be used in simulations of small biological neural assemblies. A larger machine, with much higher component count, speed and density as well as higher resolution of synaptic weights and time constants is currently under development. Some specific design issues for the construction of larger machines including selection of optimal component parameters, high density interconnect methods, and control software are discussed.
Arranging the photosensitive elements of an imaging sensor in a log-polar grid automatically samples an image in a logpolar space. The Retina project is a chip with such a spatially varying layout that can produce the advantages of image processing in the new space at real-time speeds. The actual chip is a small part of a complete imaging system. The system is part of a class of imagers called foveal sensors and these sensors have distinct and significant computational savings over conventional imagers as many as 3-10 orders of magnitude improvement in processing time and memory. The design maintains a large region of high-resolution data although it is still only a fraction of the total photosensitive area.
This paper gives an overview of the principles and hardware realizations of artificial neural networks. The first section describes the operation of neural networks, using simple examples to illustrate some of its key properties. Next the different architectures are described, including single and multiple perceptron networks, Hopfield and Kohonen nets. A brief discussion of the learning rules employed in feedforward and feedback networks follows. The final section discusses hardware implementations of neural systems with emphasis on analog VLSI. Different approaches for the realizations of neurons and synapses are described. A brief comparison between analog and digital techniques is given.
The retina is a smart sensor, but in the sense of intelligent design and not on-chip computing power. It uses a unique
layout and elementary charge computing elements to implement in hardware a polar-exponential transform on visual data.
The final chip includes a large section of photosites arranged in a circular pattern. Further, the pixels grow m size as radial
distance increases. The retina also has a fovea (a high resolution area at the chip's center) and the computational circuitry.
The sensor works and will serve as the key component of a real-time imaging system.