The human vision system (HVS) is remarkably robust against eye distortions. Through a combination of eye movements and visual feedback, the HVS can often appropriately interpret scene information acquired from flawed optics. Inspired by biological systems, we have built an electronically and mechanically reconfigurable "saccadic" camera system. The saccadic camera is designed to efficiently examine scenes through foveated imaging, where scrutiny is reserved for salient regions of interest. The system's "eye" is an electronic image sensor used in multiple modes of resolution. We use a subwindow set at high resolution as the system's fovea, and capture the remaining visual field at a lower resolution. The ability to program the subwindow's size and position provides an analog to biological eye movements. Similarly, we can program the system's mechanical components to provide the "neck's" locomotion for modified perspectives. In this work, we use the saccadic camera to develop a "work-around" routine in response to possible degradations in the camera's lens. This is particularly useful in situations where the camera's optics are exposed to harsh conditions, and cannot be easily repaired or replaced. By exploiting our knowledge of the image sensor's electronic coordinates relative to the camera's mechanical movement, the system is able to develop an empirical distortion model of the image formation process. This allows the saccadic camera to dynamically adapt to changes in its image quality.
A concept is described for the detection and location of transient objects, in which a "pixel-binary" CMOS imager is used to give a very high effective frame rate for the imager. The sensitivity to incoming photons is enhanced by the use of an image intensifier in front of the imager. For faint signals and a high enough frame rate, a single "image" typically contains only a few photon or noise events. Only the event locations need be stored, rather than the full image. The processing of many such "fast frames" allows a composite image to be created. In the composite image, isolated noise events can be removed, photon shot noise effects can be spatially smoothed and moving objects can be de-blurred and assigned a velocity vector. Expected objects can be masked or removed by differencing methods. In this work, the concept of a combined image intensifier/CMOS imager is modeled. Sensitivity, location precision and other performance factors are assessed. Benchmark measurements are used to validate aspects of the model. Options for a custom CMOS imager design concept are identified within the context of the benefits and drawbacks of commercially available night vision devices and CMOS imagers.
An optical beam combined with an array detector in a suitable geometrical arrangement is well-known to provide a range measurement based on the image position. Such a 'triangulation' rangefinder can measure range with short-term repeatability below the 10-5 level, with the aid of spatial and temporal image processing. This level of precision is achieved by a centroid measurement precision of ±0.02 pixel. In order to quantify its precision, accuracy and linearity, a prototype triangulation rangefinder was constructed and evaluated in the laboratory using a CMOS imager and a collimated optical source. Various instrument, target and environmental conditions were used. The range-determination performance of the prototype instrument is described, based on laboratory measurements and augmented by a comprehensive parametric model. Temperature drift was the dominant source of systematic error. The temperature and vibration environments and target orientation and motion were controlled to allow their contributions to be independently assessed. Laser, detector and other effects were determined both experimentally and through modeling. Implementation concepts are presented for a custom CMOS imager that can enhance the performance of the rangefinder, especially with regards to update rate.
Vanishing point and Z-tranform image center calibration techniques are reported for a prototype “compound-eye” camera system which can contain up to 25 “eyelets”. One application of this system is to track a fast-moving object, such as a tennis ball, over a wide field of view. Each eyelet comprises a coherent fiber bundle with a small imaging lens at one end. The other ends of the fiber bundles are aligned on a plane, which is re-imaged onto a commercial CMOS camera. The design and implementation of the Dragonfleye prototype is briefly described. Calibration of the image centers of the eyelet lenses is performed using a vanishing point technique, achieving an error of approximately ±0.2 pixels. An alternative technique, the Z-transform, is shown to be able to achieve similar results. By restricting the application to a two-dimensional surface, it is shown that similar accuracies can be achieved using a simple homography transformation without the need for calibrating individual eyelets. Preliminary results for object tracking between eyelets are presented, showing an error between actual and measured positions of around 3.5 mrad.
Zoom magnification is an essential element of video-based low vision enhancement systems. However, since optical
zoom systems are bulky and power intensive, digital zoom is an attractive alternative. This paper determines the visual
acuity of 15 subjects when a letter chart is viewed through a video system with various levels of digital zoom. A strategy
in which the 1:1 magnified image is obtained by combining optical magnification with digital minification gives the best
result, provided background scene information is know from the other cameras. A real-time FPGA based system for
simultaneous zoom and smoothing is also demonstrated for text reading and enhancement.
Compound eyes are a highly successful natural solution to the issue of wide field of view and high update rate for vision systems. Applications for an electronic implementation of a compound eye sensor include high-speed object tracking and depth perception. In this paper we demonstrate the construction and operation of a prototype compound eye sensor which currently consists of up to 20 eyelets, each of which forms an image of approximately 150 pixels in diameter on a single CMOS image sensor. Post-fabrication calibration of such a sensor is discussed in detail with reference to experimental measurements of accuracy and repeatability.