Advancements in machine learning (ML) and deep learning (DL) have enabled imaging systems to perform complex classification tasks, opening numerous problem domains to solutions driven by high quality imagers coupled with algorithmic elements. However, current ML and DL methods for target classification typically rely upon algorithms applied to data measured by traditional imagers. This design paradigm fails to enable the ML and DL algorithms to influence the sensing device itself, and treats the optimization of the sensor and algorithm as separate sequential elements. Additionally, this current paradigm narrowly investigates traditional images, and therefore traditional imaging hardware, as the primary means of data collection. We investigate alternative architectures for computational imaging systems optimized for specific classification tasks, such as digit classification. This involves a holistic approach to the design of the system from the imaging hardware to algorithms. Techniques to find optimal compressive representations of training data are discussed, and most-useful object-space information is evaluated. Methods to translate task-specific compressed data representations into non-traditional computational imaging hardware are described, followed by simulations of such imaging devices coupled with algorithmic classification using ML and DL techniques. Our approach allows for inexpensive, efficient sensing systems. Reduced storage and bandwidth are achievable as well since data representations are compressed measurements which is especially important for high data volume systems.
We report on the design, modeling, calibration, and experimental results of a LWIR, spectrally and temporally resolved broad band bi-directional reflectance distribution function measuring device. The system is built using a commercial Fourier transform infrared spectrometer, which presents challenges due to relatively low power output compared to laser based methods. The instrument is designed with a sample area that is oriented normal to gravity, making the device suitable for measuring loose powder materials, liquids, or other samples that can be difficult to measure in a vertical orientation. The team built a radiometric model designed to understand the trade space available for various design choices as well as to predict instrument success at measuring the target materials. The radiometric model was built by using the output of commercial non sequential raytracing tools combined with a scripted simulation of the interferometer. The trade space identified in this analysis will be presented.
The design was based on moving periscopes with custom off axis parabolas to focus the light onto the sample. The system assembly and alignment will be discussed. The calibration method used for the sensor will be detailed, and preliminary measurements from this research sensor will be presented.
We report on the design of a refracting prism array for use in a computational lensless imaging system. The technique discussed enables creation of a refracting element that maximizes signal on a detector region. Examples of pseudo-random prism arrays for the generation of images are provided. The pseudo-random prism array is compared to a randomly oriented prism array and the advantages of the optimal scattering element are highlighted.
Computational imagers fundamentally enable new optical hardware through the use of both physical and algorithmic elements. We report on the creation of a static lensless computational imaging system enabled by this paradigm.
The modeling and simulation of non-traditional imaging systems require holistic consideration of the end-to-end system. We demonstrate this approach through a tolerance analysis of a random scattering lensless imaging system.
Hyperspectral imaging polarimetry enables both the spectrum and its spectrally resolved state of polarization to be measured. This information is important for identifying material properties for various applications in remote sensing and agricultural monitoring. We describe the design and performance of a ruggedized, field deployable hyperspectral imaging polarimeter, designed for wavelengths spanning the visible to near-infrared (450 to 800 nm). An entrance slit was used to sample the scene in a pushbroom scanning mode across a 30 deg vertical by 110 deg horizontal field-of-view. Furthermore, athermalized achromatic retarders were implemented in a channel spectrum generator to measure the linear Stokes parameters. The mechanical and optical layout of the system and its peripherals, in addition to the results of the sensor’s spectral and polarimetric calibration, are provided. Finally, field measurements are also provided and an error analysis is conducted. With its present calibration, the sensor has an absolute polarimetric error of 2.5% RMS and a relative spectral error of 2.3% RMS.
Channeled linear imaging polarimeters measure the two-dimensional distribution of the linear Stokes parameters. A key aspect of this technique is to accurately reconstruct the Stokes parameters from a snapshot, modulated measurement of the channeled linear imaging polarimeter. The state-of-the-art reconstruction takes the Fourier transform of the measurement to separate the Stokes parameters into channels. While straightforward, this approach is sensitive to channel cross-talk and imposes bandwidth limitations that cut off high frequency details. To overcome these drawbacks, we present a reconstruction method called compressed channeled linear imaging polarimetry. In this framework, reconstruction in channeled linear imaging polarimetry is an underdetermined problem, where we measure N pixels and recover 3N Stokes parameters. We formulate an optimization problem by creating a mathematical model of the channeled linear imaging polarimeter with inspiration from compressed sensing. Through simulations, we show that our approach mitigates artifacts seen in Fourier reconstruction, including image blurring and degradation and ringing artifacts caused by windowing and channel cross-talk. By demonstrating more accurate reconstructions, we push performance to the native resolution of the sensor, allowing more information to be recovered from a single measurement of a channeled linear imaging polarimeter.
Lensless imaging systems have the potential to provide new capabilities for lower size and weight configuration than traditional imaging systems. Lensless imagers frequently utilize computational imaging techniques, which moves the complexity of the system away from optical subcomponents and into a calibration process whereby the measurement matrix is estimated. <p> </p>We report on the design, simulation, and prototyping of a lensless imaging system that utilizes a 3D printed optically transparent random scattering element. Development of end-to-end system simulations, which includes simulations of the calibration process, as well as the data processing algorithm used to generate an image from the raw data are presented. These simulations utilize GPU-based raytracing software, and parallelized minimization algorithms to bring complete system simulation times down to the order of seconds.<p> </p> Hardware prototype results are presented, and practical lessons such as the effect of sensor noise on reconstructed image quality are discussed. System performance metrics are proposed and evaluated to discuss image quality in a manner that is relatable to traditional image quality metrics. Various hardware instantiations are discussed.
This paper describes measurements being made on a series of material systems for the purpose of developing a radiative-transfer model that describes the reflectance of light by granular solids. It is well recognized that the reflectance spectra of granular materials depend on their intrinsic (n(λ) and k(λ)) and extrinsic (morphological) properties. There is, however, a lack of robust and proven models to relate spectra to these parameters. The described work is being conducted in parallel with a modeling effort<sup>1</sup> to address this need. Each follows a common developmental spiral in which material properties are varied and the ability of the model to calculate the effects of the changes are tested. The parameters being varied include particle size/shape, packing density, material birefringence, optical thickness, and spectral contribution of a substrate. It is expected that the outcome of this work will be useful in interpreting reflectance data for hyperspectral imaging (HSI), and for a variety of other areas that rely on it.
Channeled spectropolarimeters (CSP) measure the polarization state of light as a function of wavelength. Conventional Fourier reconstruction suffers from noise, assumes the channels are band-limited, and requires uniformly spaced samples. To address these problems, we propose an iterative reconstruction algorithm. We develop a mathematical model of CSP measurements and minimize a cost function based on this model. We simulate a measured spectrum using example Stokes parameters, from which we compare conventional Fourier reconstruction and iterative reconstruction. Importantly, our iterative approach can reconstruct signals that contain more bandwidth, an advancement over Fourier reconstruction. Our results also show that iterative reconstruction mitigates noise effects, processes non-uniformly spaced samples without interpolation, and more faithfully recovers the ground truth Stokes parameters. This work offers a significant improvement to Fourier reconstruction for channeled spectropolarimetry.
Compact snapshot imaging polarimeters have been demonstrated in literature to provide Stokes parameter estimations for spatially varying scenes using polarization gratings. However, the demonstrated system does not employ aggressive modulation frequencies to take full advantage of the bandwidth available to the focal plane array. A snapshot imaging Stokes polarimeter is described and demonstrated through results. The simulation studies the challenges of using a maximum bandwidth configuration for a snapshot polarization grating based polarimeter, such as the fringe contrast attenuation that results from higher modulation frequencies. Similar simulation results are generated and compared for a microgrid polarimeter. Microgrid polarimeters are instruments where pixelated polarizers are superimposed onto a focal plan array, and this is another type of spatially modulated polarimeter, and the most common design uses a 2x2 super pixel of polarizers which maximally uses the available bandwidth of the focal plane array.
We report on the development of a prototype polarization tag based system for detecting chemical vapors. The system primarily consists of two components, a chemically sensitive tag that experiences a change in its optical polarization properties when exposed to a specific chemical of interest, and an optical imaging polarimeter that is used to measure the polarization properties of the tags. Although the system concept could be extended to other chemicals, for the initial system prototype presented here the tags were developed to be sensitive to hydrogen fluoride (HF) vapors. HF is used in many industrial processes but is highly toxic and thus monitoring for its presence and concentration is often of interest for personnel and environmental safety. The tags are periodic multilayer structures that are produced using standard photolithographic processes. The polarimetric imager has been designed to measure the degree of linear polarization reflected from the tags in the short wave infrared. By monitoring the change in the reflected polarization signature from the tags, the polarimeter can be used to determine if the tag was exposed to HF gas. In this paper, a review of the system development effort and preliminary test results are presented and discussed, as well as our plan for future work.
Modulated imaging Stokes polarimeters require processing of acquired data to produce an estimate of the Stokes
parameters from the scene. The total polarimeter operator describes the estimation of the Stokes parameters from
the incident fields from the scene through reconstruction. In this discussion will shall consider the polarimeter
being applied to an application where the spectral density matrix of the scene Stokes parameters and detector
noise are known. The spectral density matrix of the estimated Stokes parameters is found using the known
spectral density matrix of the scene to find the response of the operator to signal fluctuations. This analysis
grants the ability to optimize the operator for a given application. We demonstrate an optimization of system
processing algorithm that takes inspiration from the classical Wiener filter.
Non-uniformity noise is common in infrared imagers, and is usually corrected through calibration, often by
momentarily blocking the optical system with a relatively uniform temperature plate. The non-uniformity
patterns also tend to drift and require periodic recalibration, necessitating occasional loss of video from the imager
during the recalibration process. Microgrid polarimeters are especially sensitive to fixed-pattern noise because
the polarization signal is acquired by differentiation of neighboring pixels. Scene-based algorithms attempt to
alleviate the need for recalibration of the imager through image processing techniques. We introduce a new
frequency-domain scene-based non-uniformity estimation and correction technique, and apply the technique to
infrared and microgrid polarimeter imagery. The technique demonstrates promising results for shutter-assisted
(recalibration) video, for microgrid polarization systems as well as most spatially modulated sensor systems.
Recently, a polarimetric data reduction technique has been developed that in the presence of a time varying
signals and noise free measurement process can achieve an error free reconstruction provided that the signal
was band limited. Error free reconstruction for such a signal is not possible using conventional data reduction
methods. The new approach provides insight for processing arbitrary modulation schemes in space, time, and
wavelength. Theory predicts that a polarimeter that employs a spatio-temporal modulation scheme may be able
to use the high temporal resolution of a spatially modulated device combined with the high spatial resolution
of a temporally modulated system to attain greater combined resolution capabilities than either modulation on
scheme can produce alone. A polarimeter that contains both spatial and temporal modulation can be constructed
(for example) by placing a rotating retarder in front of a micropolarizer array (microgrid). This study develops
theory and analysis for the rotating retarder microgrid polarimeter to show how the available bandwidth for
each channel is affected by additional dimensions of modulation and demonstrates a working polarimeter with
a simulation of Stokes parameters that are band limited in both space and time with a noisy measurement
For the past several years we have been working on strategies to mitigate the effects of IFOV errors on
LWIR microgrid polarimeters. In this paper we present a detailed, theoretical analysis of the source of
IFOV error in the frequency domain, and show a frequency domain strategy to mitigate those effects.
Microgrid polarimeters are a type of division of focal plane (DoFP) imaging polarimeter that contains a mosaic
of pixel-wise micropolarizing elements superimposed upon an FPA sensor. Such a device measures a slightly
different polarized state at each pixel. These measurements are combined to estimate the Stokes vector at each
pixel in the image. DoFP devices have the advantage that they can obtain Stokes vector image estimates for
an entire scene from a single frame capture. However, they suffer from the disadvantage that the neighboring
measurements that are used to estimate the Stokes vector images are acquired at differing instantaneous fields of
view (IFOV). This IFOV issue leads to false polarization signatures that significantly degrade the Stokes vector
images. Interpolation and other image processing strategies can be employed to reduce IFOV artifacts; however
these techniques have a limit to the amount of enhancement they can provide on a single microgrid image.
Here we investigate algorithms that use multiple microgrid images that contain frame-to-frame global motion
to further enhance the Stokes vector image estimates. Motion-based imagery provides additional redundancy
that can be exploited to recover information that is "missing" from a single microgrid frame capture. We have
found that IFOV and aliasing artifacts can be defeated entirely when these types of algorithms are applied to the
data prior to Stokes vector estimation. We demonstrate results on real LWIR microgrid data using a particular
resolution enhancement technique from the literature.