Proc. SPIE. 9464, Ground/Air Multisensor Interoperability, Integration, and Networking for Persistent ISR VI
KEYWORDS: Cameras, Video, Digital cameras, Zoom lenses, Digital recording, Wireless communications, Data communications, Commercial off the shelf technology, Global Positioning System, Near field optics
Current US Army issued binoculars lack the digital capabilities of today’s electro-optic devices. By linking traditional optical binoculars with a smartphone, users can take advantage of the smartphone’s digital camera. Live images viewed through the binocular can be captured as an image or recorded as video in real time. Additional capabilities of the Smartphone can be utilized such as, digital zoom on top of the binoculars optical magnification, GPS for geo-tagging information, wireless communication for transmission of recorded data, etc. The linking of Commercial Off-The-Shelf (COTS) smartphones with optical based binoculars has shown enormous potential including persistent ISR capability. The paper discusses the demonstration, results and lessons learned of B-LINK-S applications.
The development and demonstration of a new snapshot hyperspectral sensor is described. The system is a significant
extension of the four dimensional imaging spectrometer (4DIS) concept, which resolves all four dimensions of
hyperspectral imaging data (2D spatial, spectral, and temporal) in real-time. The new sensor, dubbed "4×4DIS" uses a
single fiber optic reformatter that feeds into four separate, miniature visible to near-infrared (VNIR) imaging
spectrometers, providing significantly better spatial resolution than previous systems. Full data cubes are captured in
each frame period without scanning, i.e., "HyperVideo". The current system operates up to 30 Hz (i.e., 30 cubes/s), has
300 spectral bands from 400 to 1100 nm (~2.4 nm resolution), and a spatial resolution of 44×40 pixels. An additional
1.4 Megapixel video camera provides scene context and effectively sharpens the spatial resolution of the hyperspectral
data. Essentially, the 4×4DIS provides a 2D spatially resolved grid of 44×40 = 1760 separate spectral measurements
every 33 ms, which is overlaid on the detailed spatial information provided by the context camera. The system can use a
wide range of off-the-shelf lenses and can either be operated so that the fields of view match, or in a "spectral fovea"
mode, in which the 4×4DIS system uses narrow field of view optics, and is cued by a wider field of view context
camera. Unlike other hyperspectral snapshot schemes, which require intensive computations to deconvolve the data
(e.g., Computed Tomographic Imaging Spectrometer), the 4×4DIS requires only a linear remapping, enabling real-time
display and analysis. The system concept has a range of applications including biomedical imaging, missile defense,
infrared counter measure (IRCM) threat characterization, and ground based remote sensing.
Coherent Logix has implemented a digital video stabilization algorithm for use in soldier systems and small unmanned
air / ground vehicles that focuses on significantly reducing the size, weight, and power as compared to current
implementations. The stabilization application was implemented on the HyperX architecture using a dataflow
programming methodology and the ANSI C programming language. The initial implementation is capable of stabilizing
an 800 x 600, 30 fps, full color video stream with a 53ms frame latency using a single 100 DSP core HyperX hx3100TM
processor running at less than 3 W power draw. By comparison an Intel Core2 Duo processor running the same base
algorithm on a 320x240, 15 fps stream consumes on the order of 18W. The HyperX implementation is an overall 100x
improvement in performance (processing bandwidth increase times power improvement) over the GPP based platform.
In addition the implementation only requires a minimal number of components to interface directly to the imaging sensor
and helmet mounted display or the same computing architecture can be used to generate software defined radio
waveforms for communications links. In this application, the global motion due to the camera is measured using a
feature based algorithm (11 x 11 Difference of Gaussian filter and Features from Accelerated Segment Test) and model
fitting (Random Sample Consensus). Features are matched in consecutive frames and a control system determines the
affine transform to apply to the captured frame that will remove or dampen the camera / platform motion on a frame-by-frame
This paper describes the microfabrication process and characterization of wavelength selective germanium dielectric
supported microbolometers, which should be compatible with standard microbolometer fabrication processes. Here we
have demonstrated a micro fabricated robust germanium dielectric structure layer that replaces the usual silicon nitride
structural layer in microbolometers. The fabricated microbolometers consist of a chromium resistive sheet as an absorber
layer above an air-gap/germanium dielectric structure.
The hypersensor camera operates with a unique multispectral imaging modality developed
recently at Surface Optics Corporation. The Hypersensor camera is small, low cost, rugged, and
solid state, using micro-optics and an array of spectral filters, which captures a complete
multispectral cube of spatial and spectral data with every focal plane exposure. The prototype
VNIR Hypersensor camera captures full cubes of 588x438 (spatial pixels) x 16 (spectral bands)
at frame rates up to 60 Hz. This paper discusses the optical design of the Hypersensor camera,
the measured performance, and the design and operation of a custom video-rate hyperspectral
processor developed for this system.
Hyperspectral ground to ground viewing perspective presents major challenges for autonomous window based detection.
One of these challenges has to do with object scales uncertainty that occur when using a window-based detection
approach. In a previous paper, we introduced a fully autonomous parallel approach to address the scale uncertainty
problem. The proposed approach featured a compact test statistic for anomaly detection, which is based on a principle of
indirect comparison; a random sampling stage, which does not require secondary information (range or size) about the
targets; a parallel process to mitigate the inclusion by chance of target samples into clutter background classes during
random sampling; and a fusion of results at the end. In this paper, we demonstrate the effectiveness and robustness of
this approach on different scenarios using hyperspectral imagery, where for most of these scenarios, the parameter
settings were fixed. We also investigated the performance of this suite over different times of the day, where the spectral
signatures of materials varied with relation to diurnal changes during the course of the day. Both visible to near infrared
and longwave imagery are used in this study.
The use of a patterned resistive sheet acting as an infrared frequency-selective absorber is discussed. These patterned resistive sheets are a modified form of classical Salisbury Screens that utilize a resistive absorber layer placed a quarter-wavelength in front of a mirror. In contrast with previously designed planar antenna-coupled microbolometers that consist of both resistive and highly conductive metal strips (acting as antennas), the absorption layer in these structures involves a single resistive layer with patterned holes.
Hyperspectral imaging in the 2-5 um band has held interest for applications in detection and discrimination of targets. Real time instrumentation is particularly powerful as a tool for characterization and field measurement. A compact, real-time, refractive MWIR hyperspectral imaging instrument has been designed and is undergoing integration and test. The system has been designed for cryogenic operation to improve signal to noise ratio, reduce background noise, and enable real-time hyperspectral video processing. Partial testing has been completed on cryogenic elements and “first light” 2-5 μm hyperspectral images have been collected at room temperature.
Hyperspectral imaging in the 2-5 μm band has held interest for applications in detection and discrimination of targets. Real time instrumentation is particularly powerful as a tool for characterization and field measurement. A compact, real-time, refractive MWIR hyperspectral imaging instrument has been designed, and is undergoing testing. Using a combination of dispersive and corrective elements, the system has been designed for integration and preliminary test at room temperature with passive focus correction for the cryogenic elements. The F/1.75 design supports near diffraction limited performance from 2.5 μm to 5.0 μm. This paper will review the challenges in manufacturing such a system as well as the alignment and performance data.
A linear CdZnTe pad detector array with approximately 1 mm2 pad area has been developed. The detector has a wide energy range from about 20 to 200 keV. To read out these detector arrays, a fast, low-noise monolithic mixed signal ASIC chip has been developed. A prototype x-ray imaging system consisting of the CdZnTe detector array and the monolithic ASIC chip has been fabricated and tested. In this system, the detectors are abutted against each other to form an approximately 1 m long linear array. The system has been used to take preliminary scanned images of complex objects at various energies. New results from this system will be presented.
Hyperspectral imaging is the latest advent in imagin technology, providing the potential to extract information about the objects in a scene that is unavailable to panchromatic imagers. This increased utility, however, comes at the cost of tremendously increased data. The ultimate utility of hyperspectral imagery is in the information that can be gleaned from the spectral dimensions, rather than in the hyperspectral imagery itself. To have the broadest range of applications, extraction of this information must occur in real-time. Attempting to produce and exploit compete cubes of hyperspectral imagery at video rates, however, presents unique problems for both the imager and the processor, since data rates are scaled by the number of spectral planes in the cube. MIDIS allows both real-time collection and processing of hyperspectral imagery over the range of 0.4 micrometers to 12 micrometers .
Hyperspectral imaging is the latest advent in imaging technology, providing the potential to extract information about the objects in a scene that is unavailable to panchromatic imagers. This increased utility, however, comes at the cost of tremendously increased data. The ultimate utility of hyperspectral imagery is in the information that can be gleaned from the spectral dimension, rather than in the hyperspectral imagery itself. To have the broadest range of applications, extraction of this information must occur in real-time. Attempting to produce and exploit complete cubes of hyperspectral imagery at video rates, however, presents unique problems for both the imager and the processor, since data rates are scaled by the number of spectral planes in the cube. MIDIS, the Multi-band Identification and Discrimination Imaging Spectroradiometer, allows both real-time collection and processing of hyperspectral imagery over the range of 0.4 micrometer to 12 micrometer. Presented here are the major design challenges and solutions associated with producing high-speed, high-sensitivity hyperspectral imagers operating in the Vis/NIR, SWIR/MWIR and LWIR, and of the electronics capable of handling data rates up to 160 mega-pixels per second, continuously. Beyond design and performance issues associated with producing and processing hyperspectral imagery at such high speeds, this paper also discusses applications of real-time hyperspectral imaging technology. Example imagery includes such problems as buried mine detection, inspecting surfaces, and countering CCD (camouflage, concealment, and deception).
Hyperspectral imaging is the latest advent in imaging technology, providing the potential to extract information about the objects in a scene that is unavailable to panchromatic imagers. This increased utility, however, comes at the cost of tremendously increased data. The ultimate utility of hyperspectral imagery is in the information that can be gleaned from the spectral dimension, rather than in the hyperspectral imagery itself. To have the broadest range of applications, extraction of this information must occur in real-time. Attempting to produce and exploit complete cubes of hyperspectral imagery at video rates, however, present unique problems for both the imager and the processor, since data rates are scaled by the number of spectral planes in the cube. MIDIS, the Multi-band Identification and Discrimination Imaging Spectroradiometer, allows both real-time here are the major design innovations associated with producing high-speed, high-sensitivity hyperspectral imagers operating in the SWIR and LWIR, and of the electronics capable of handling data rates up to 160 megapixels per second, continuously. Discussion of real-time algorithms capable of exploiting the spectral dimension of the imagery is also included. Beyond design and performance issues associated with producing and processing hyperspectral imagery at such high speeds, this paper also discusses applications of real-time hyperspectral imaging technology. Example imagery includes such problems as detecting counterfeit money, inspecting surfaces, and countering CCD.
In order to achieve their goal of surreptitious operation within a country, terrorist organizations attempt to hide themselves from public view. In many instances such masking takes the form of simply appearing like the surrounding populace. In others, such as training facilities, standard military camouflaging techniques are used to conceal the group's equipment and activities. To effectively monitor and suppress activities of terrorist organizations, defeating the groups' attempt to hide is essential. Although finding individuals hiding within a society is extremely problematic, discovering camouflaged equipment, facilities, and personnel is readily accomplished by proper exploitation of hyperspectral imagery. Camouflage techniques attempt to make an object appear similar to its background, thereby making it difficult to find. Although making an object have similar color to its background is fairly easy, making it have the same spectral appearance is nearly impossible, unless the object is covered in the same material as the background. Even attempting to hide an object by covering it in background material will not work against a spectral imager since the act of moving the background material, e.g., foliage cuttings, changes the material's spectral characteristics. Hence, by collecting and properly exploiting spectral imagery, camouflaged objects can be readily differentiated from their background. This paper presents development of this technique, and of the MIDIS (multi-band identification and discrimination imaging spectroradiometer) instrument capable of real-time discrimination of camouflaged objects throughout a scene. Spectral matched-filtering of hyperspectral imagery also has the potential to find vehicles or structures which may be laden with explosives. Many explosives contain volatile materials, the release of which can be imaged by viewing appropriate spectral regions. Volatiles from the fuel oil in readily-produced ANFO are an example. If such volatiles were seen emanating from a vehicle or structure where they would not normally be expected, closer inspection would be warranted. Additionally, packing a vehicle with explosives often leaves trace residues on the outside of the vehicle. Spectral imaging and matched filtering can be used to identify these residues. Incorporation of spectral imaging surveillance equipment at probable terrorist targets could avert disasters such as the tragic bombing of the Murrah Federal Building in Oklahoma City. Application of MIDIS technology to explosive identification is also detailed.
Many imaging applications require quantitative determination of a scene's spectral radiance. This paper describes a new system capable of real-time spectroradiometric imagery. Operating at a full-spectrum update rate of 30Hz, this imager is capable of collecting a 30 point spectrum from each of three imaging heads: the first operates from 400 nx m to 950 nm, with a 2% bandwidth; the second operates from 1.5 micrometers to 5.5 micrometers with a 1.5% bandwidth; the third operates from 5 micrometers to 12 micrometers , also at a 1.5% bandwidth. Standard image format is 256 X 256, with 512 X 512 possible in the VIS/NIR head. Spectra of up to 256 points are available at proportionately lower frame rates. In order to make such a tremendous amount of data more manageable, internal processing electronics perform four important operations on the spectral imagery data in real-time. First, all data in the spatial/spectral cube of data is spectro-radiometrically calibrated as it is collected. Second, to allow the imager to simulate sensors with arbitrary spectral response, any set of three spectral response functions may be loaded into the imager including delta functions to allow single wavelength viewing; the instrument then evaluates the integral of the product of the scene spectral radiances and the response function. Third, more powerful exploitation of the gathered spectral radiances can be effected by application of various spectral-matched filtering algorithms to identify pixels whose relative spectral radiance distribution matches a sought- after spectral radiance distribution, allowing materials-based identification and discrimination. Fourth, the instrument allows determination of spectral reflectance, surface temperature, and spectral emissivity, also in real-time. The spectral imaging technique used in the instrument allows tailoring of the frame rate and/or the spectral bandwidth to suit the scene radiance levels, i.e., frame rate can be reduced, or bandwidth increased to improve SNR when viewing low radiance scenes.