Nontraditional optical systems offer a larger, richer design space than traditional arrangements. Some are inspired by the optical imaging systems of invertebrate animals.12.–3 Compound apposition and superposition eyes are two broad categories. In this research, a lenslet array serves as the basis for a thin, compound imaging apposition optic that, owing to its multichannel nature, admits a diversity of spectral filters and linear polarizers. Circular polarizers could be included in future systems4 to open up the possibility of obtaining full Stokes data. The inclusion of multiple filters deals with the important problem of subimage crosstalk inherent in multichannel optics. This architecture improves upon the thin-observation module by bounded optics (TOMBO)5 which also employs a lenslet array. The TOMBO device includes three layers: lenslet array, separation layer, and photodetector array. The image formed on the photodetector comprises the multiple subimages that are formed by the lenslet array. This three layer architecture is shown in Fig. 1.
The images formed by the lenslet array have lower angular resolution individually, but through multiframe super-resolution these low resolution images may be combined into a single high resolution image.5 It is worth noting that Ref. 5 contains a caveat concerning the TOMBO system describing a signal separation scheme that employs two like polarizer arrays arranged at the aperture and focal planes, each array containing alternating horizontal and linear polarizers. This would prevent crosstalk between a subfield and four laterally adjacent neighbors, but not the diagonal neighbors or nonadjacent neighboring channels. In the course of this paper, it will become obvious that this is insufficient to effectively deal with crosstalk.
Cross-talk results from light rays entering one subaperture at a high field angle and falling within the projected image area of another subaperture. The TOMBO designers use a physical signal separation layer to eliminate the cross-talk by absorbing rays that originate from points outside the subimage field of view. The drawbacks of this strategy include reduction in the effective aperture (especially since circular lenslets are employed) and in the field of view of the system by the separator walls.
This investigation shows that it is possible to reduce or even eliminate cross-talk using a combination of polarizers and spectral filters. As a natural consequence of this simple strategy the imager attains the ability to provide spectro-polarimetric resolution, and a larger, richer design space is opened up to the optical designer. Others have demonstrated multispectral imaging6,7 via lenslet array architectures, but they still used a signal separator and had less than 100% fill factor.
The purpose of this paper is to illuminate this simple strategy for eliminating cross-talk while creating a small spectro-polarimetric imager, to describe the tools developed in Code V® for modeling the system, and to convey the results of this preliminary modeling. Section 2 will briefly introduce the geometry of the lenslet array as well as the effect of cross-talk. Section 3 discusses the bare lenslet array model, which was the starting point for all designs. In Sec. 4, we present the first, simpler design involving only RGB filters. Section 5 presents the design including more, narrower filters to suppress parasitic light over a larger FOV. In Sec. 6, we provide concluding remarks as well as possible directions for future work.
Subimage Cross-Talk in Lenslet Array Imagers
Figure 2 schematically illustrates three simple rays paths that constitute cross-talk in a system without subfield stops. Ray 1 traverses the lenslet through a single channel, but falls on the image plane within an adjacent channel. Ray 2 enters the lenslet array through one channel and leaves the rear of the array through an adjacent one. Ray 3 undergoes multiple partial reflections and contributes to scattered light. There are several other, higher order scenarios that can contribute to crosstalk and stray light.
All three of these scenarios have the net effect of multiplexing and corrupting the image data. In the presence of detector noise and limited dynamic range, the resulting image is difficult, if not impossible, to satisfactorily de-multiplex. The lenslet array can be seen as spreading the incoming light into multiple spatial modes, a process that, due to the second law of thermodynamics, is not completely reversible.8 Methods of cross-talk elimination based solely on post-processing are not, therefore, tenable. This particular issue also happens to be a focus of research in Shack-Hartmann sensor design.9
Optical Model of Bare Lenslet Array
In this investigation we used Code V® to perform nonsequential ray tracing of this multichannel optic. The model is based upon an in-house, 100% fill factor, square aperture, fused silica lenslet array graciously provided by Jenoptik Optical Systems in Huntsville, AL that had a 1 mm pitch, 1.5 mm thickness, 9 mm focal length, and equiconvex surfaces. The full, lateral field of view of a single channel is, therefore 6.36 deg.
A array structure was constructed based on the pitch, focal length, and thickness of the array. Figure 3 shows a Code V® layout of the nonsequential element.
The effective field of view (EFOV) for the lenslet array is defined by the rays of the highest field angle that make it to the focal plane of the bare lenslet array, i.e., in the absence of any subaperture filters. The model that includes steradian illumination is restricted to this EFOV to reduce computation time. The object plane was located a finite distance of 1500 mm from the array, because it is simpler than modeling collimated ray bundles from infinity in Code V® and the distance places the object well beyond the hyperfocal distance for the 7.5 μm pixels of the model detector plane. The diagonal maximum effective half-angle field of view (EFOV) is 49 deg. This is calculated as the inverse tangent of the diagonal length of the detector over the focal length of a single lenslet. At this angle, rays from a lenslet at the very corner of the array will make it to the opposite corner of the array. Higher angle rays do not reach the focal plane. Figure 4 illustrates the progression of the spot diagram for the central lenslet as the point source reaches the edge of the effective field. To illustrate the image multiplexing problem presented by a bare lenslet array with no signal separation layer or filters, a small, sparse, bulls-eye shaped, Lambertian object was traced, via Monte Carlo ray tracing, from the established object plane. Figure 4 shows the concentric, constant radiance rings.
The innermost ring is inscribed within the field of view (FOV) of a single 1 mm subaperture while the outer two rings extend only to the middle of an adjacent channel. The wavelengths traced extended from 500 to in 10 nm increments. The total number of rays traced was twenty-two million. The detector plane in the model was with pixels.The results of the ray trace are presented in Fig. 5. The expected cross-talk between adjacent subapertures is clearly evident. If the input field were larger and less sparse the image would be completely unintelligible.
Preliminary Model Including Subaperture Filter Arrays
A simple, preliminary model based on idealized RGB filters was constructed in order to develop our modeling strategy and algorithms used in the simulation, to tease out any simple but unforeseen problems, and to probe the ability of the normal illumination roll-off to prevent crosstalk between nonadjacent channels. By combining nonoverlapping bandpass filters with linear polarizers in one of two orthogonal orientations, one can create an ensemble of incommensurate filters so that light passing through any one filter would be blocked by an unlike filter.
By positioning like arrays of polarizers and filters at both the front of the lenslet array and at the focal plane one can mitigate the subimage crosstalk illustrated in Fig. 2. Note that this arrangement simultaneously affords the important benefit of spectro-polarimetric resolution along with the subimage cross-talk suppression.
Figure 6 above shows an exploded, three-dimensional view of a , 1-mm pitch lenslet array with filter arrays included at the entrance aperture and focal planes. Elements in the filter arrays consist of RGB, nonoverlapping bandpass filters each with a linear polarizer in one of two orthogonal orientations. This combination yields a total of six different filters. Mutually incommensurate filters can be arranged in the first row. If successive rows are formed by cyclically shifting the first row to the right by two places, one can insure that no subaperture filter is adjacent to any like filter. This should allow the suppression of cross-talk, at least for subfields corresponding to subaperture regions of the array. This is shown in Fig. 7, where 1 stands for Filter 1, 2 for Filter 2, 3 for Filter 3, H for horizontal polarizer, and V for vertical polarizer.
The polarizers used in the Code V® model were linear polarizers with a reasonable 0.99 attenuation factor. Figure 8 illustrates the extinction provided by crossed polarizers of 1.00 attenuation factor. The crossed polarizers have a 100% attenuation and fully block on-axis rays, but not rays at higher field angles.10 This demonstrates the inevitable leaking of light through crossed polarizers at high angles of incidence.
The three bandpass filters were modeled as idealized top hat filters with individual bandwidths of 30 nm spanning the wavelength range of 500 to 600 nm. The spectral response of the three idealized filters is shown in Fig. 9.
The three-filter, two-polarizer design described above, designated D1, was modeled using the same three-ring object used in the model of the bare array. The first iteration of this model included filters of thickness equal to 0.76 mm. This was based upon the actual thickness of some sheet polarizers and filters in the laboratory. Modeling showed that this finite thickness contributed to the parasitic component of the transmitted light, as it caused some rays near the edges of the subapertures to miss the filters as shown in Fig. 10(a). We reduced the thickness to 10 μm, which is more in line with thin film technology but still conservative, and performed a raytrace with a single channel of the front filter illuminated and all but the corresponding channel in the rear filter array open. With the realistically thinner filters, the leakage of rays is stopped, but the polarization leakage remains in the adjacent channels with like spectral filters as is evident in Fig. 10(b). It is evident that the thickness made an order of magnitude difference in the peak transmitted parasitic light for the channel. Design D1 was, therefore, modified to include the thinner, and more realistic, filters.
Figure 11 shows results for both the original, bare array and for the final D1 configuration. One can see that the subimage crosstalk has been largely eliminated.
Geometric projection errors between the crossed polarizers still result in some residual cross-talk between adjacent channels. A little crosstalk is also still evident at the outer edges of the focal plane, as the detector (modeled after a detector in our lab) is slightly larger than the lenslet array.
Addressing the Problem of Steradian Illumination
The preliminary model, involving only a few wideband filters and linear polarizers successfully demonstrated stray light rejection for the small field subtended by the sparse bulls-eye object. In many applications, however, the sensor may be exposed to steradian illumination. This might be the case, for example, if the sensor were installed as a conformal element in the airframe of a small unmanned aerial system.
To simulate full hemispherical illumination and investigate worst-case parasitic light rejection, the object needs to be uniform and effectively fill a steradian field. To minimize computing time, we used a rectangular, Lambertian object at 1500 mm distance that filled the effective field of view, as determined in Sec. 3. Higher angle rays do not contribute to the light received at the focal plane and need not be traced.
Under these conditions the stray light rejection capabilities of design D1 were examined again. In this analysis only a single filter channel at the lenslet array was illuminated and all but the corresponding filter channel were open at the detector. Such an arrangement would show only the parasitic transmittance of that single channel. Figure 12 shows the results and demonstrates that this simple design is insufficient to effectively suppress subimage crosstalk between nonadjacent channels. The illumination roll-off does not provide sufficient crosstalk suppression between nonadjacent channels. For a larger array, or one with a large EFOV, it should lead to better suppression for channels that are very far apart.
In Figure 12 the view is from behind the detector, as shown in Code V®, so that it is a mirror image of Fig. 7. The strongest parasitic signal, or cross-talk occurs, of course, in channels with filters that are the same as the filter in the illuminated channel. A small amount of cross-talk also occurs in channels with like spectral filters and orthogonal linear polarizers. This, again, is due to the presence of a residual orthogonal polarization component for higher angle rays passing through the front polarizers.
Note also that only the average value of the parasitic light is displayed in each channel, so that it is not spatially resolved within the channel. This is the because parasitic signal is very uniform over the channels exhibiting cross-talk and the numbers of rays reaching the focal plane in these channels under these conditions was necessarily quite small, though a large total number of rays was traced. The raw detector data, therefore, suffers from a kind of “shot noise” that obscures the true nature of the parasitic signal if it is not averaged out within each channel. The alternative is to trace an inordinately huge number of rays and take a vast amount of computing time to reach what would effectively be the same conclusion.
The previous results revealed that a greater diversity of incommensurate filters is necessary to suppress crosstalk for the flat lenslet array and focal plane for exposure to steradian illumination. To more precisely characterize the rejection of parasitic light and suppression of the corresponding subimage crosstalk, we define the imager’s signal-to-noise-ratio (SNR) as the total power in the desired, direct signal divided by the total power in the parasitic signal. In order to calculate the SNR, we devised a means to quantify both the direct and parasitic signal. This method relies on the notion that the detection process is linear in irradiance and not saturated or otherwise suffering from nonlinearities in its responsivity.
As mentioned before, a light ray that enters a channel and propagates through its corresponding channel at the image plane filter is considered part of the direct light, or the signal. The size of a direct subimage is the same size as the corresponding aperture to prevent overlapping of images. To simulate direct signal through any single channel only that channel in the model is illuminated at the lenslet array, and only its corresponding channel is open at the image plane, as illustrated in Fig. 13. In other words, it is like masks are applied at the aperture and focal planes.
To obtain the total direct signal for the entire array, under effective illumination, this raytrace procedure is repeated for each channel and the corresponding distributions of light at the detector plane for all such raytraces (in terms of number of ray intercepts in each pixel) are summed at the end.
The parasitic light for any one channel was determined by opening that channel in the filter at the lenslet array and leaving all channels in the focal play filter array open, except for the corresponding channel, so that the complementary mask is applied at the focal plane. Figure 14 illustrates this. Once again, the parasitic irradiance distribution for each individual channel is summed at the end to obtain the total distribution.
The SNR was defined as the sum of the energy in the direct pixels, obtained in the manner just described, divided by the sum of the energy in the parasitic pixels.
In Eq. (1), is the number of rows in the data, is the number of columns in the data, is the energy (total number of ray intercepts) in the direct case at pixel , and is the energy in the parasitic case at pixel .
Following this procedure, the SNR for design D1 under hemispherical illumination was determined to be 0.3327. This shows quantitatively that the design is unable to reject stray light and successfully image objects within the effective field of view. This is further illustrated by the full-field en-squared energy (FE) for the single channel incident light described in Fig. 15. For this channel, less than 30% of the energy propagating to the receiver is direct.
Using the method outlined above for quantifying direct and parasitic signal, the same aperture lenslet array was analyzed under effective steradian illumination, but this time with 15 different spectral filters to insure no two filters were alike once the linear polarizers were added. Both the direct and parasitic raytrace routines (60 in all) were captured in a Code V® macro so that the entire analysis could be accomplished in a single run without user intervention. The macro recorded focal plane data for each channel in a buffer and exports it to a. dat file. Each of the 60 raytraces was saved as a Matlab array, and all were summed to obtain the direct and parasitic energies.
In Eq. (2), is the center wavelength and is the full width half maximum (FWHM) of the response. The filters were modeled with a FWHM of 20 nm with the peak transmittances spaced evenly from 480–1000 nm. The value of was chosen so that the peak transmittance was 1.0. The spectral transmittances of the filters are superimposed in Fig. 16. The figure shows that the profiles of spectrally adjacent filters cross at a common 20% transmittance point. This overlap was included in the model simply to take into account the possible overlapping of real filters. The filters are labeled Filter 1 through Filter 15 starting with the 480 nm filter and ending with the 1000 nm filter.
Horizontal and linear polarizers are labeled as H and V, respectively. The next step in the process was to arrange the filters and polarizers in two like arrays, one at front of the lenslet array and one at the focal plane, as before. It is important to note that the design presented in this work does not necessarily reflect the optimum arrangement of filters and polarizers. A design algorithm to optimize the filter array, in terms of arrangement and spectral profiles, is a fascinating subject of work to come—work based partly on previous research done at the Center for Applied Optics on evolution strategies optimization of a multiple Fabry-Pérot tunable filter system.15 Figure 17 shows the design, called D2, chosen for simulation.
The raytrace included twenty-nine wavelengths, one corresponding to the peak of each filter and one to each of the crossover points in Fig. 15. The number of rays traced per wavelength was 500,000. With twenty-nine wavelengths, this results in a total of 14.5 million rays traced per channel for both the direct and parasitic raytraces. This results in a very lengthy computation time for this nonsequential raytracing task in Code V®’s Illumination option.
An image of the composite data for either the direct case or the parasitic case does not provide any insight into the design, since a uniform source was used as the standard to determine the system SNR and the resulting images also appear uniform and uninformative. Once again, although several million rays were traced, it was still not enough rays to overcome the simulation “shot noise” that results in significant pixel-to-pixel fluctuation in signal and renders a spatially resolved map of SNR rather useless. To better visualize the effects of cross-talk on the system in an intelligible way, the results from raytracing only the (3, 3) channel are once again presented. These results are shown on a log scale in Fig. 18. The SNR value of this design, calculated using the same method as the previous design is 15.1.
This design more effectively eliminates cross-talk through the suppression of parasitic light. The FE plot of Fig. 19 further illustrates that the FE for the (3, 3) element is now around 90%.
Conclusion and Future Considerations
An important goal of this investigation was to develop a modeling technique to accommodate multiaperture, nonsequential designs, using a reconfigurable array of subaperture filters. This goal was achieved through the development of Code V® macros that autonomously execute the computationally intensive raytrace, along with Matlab routines that take detector data from the Code V® buffer and process it into a merit function for the imaging system. A simple, standardized and practical merit function was devised along with a simple technique for calculating the figure of merit, which both significantly reduces the computation time and provides an intuitive understanding of the performance. The nonsequential model with the developed macros and routines is easily modified for future investigations and can be included in an optimization loop, likely captured as a Matlab routine that calls Code V® to provide raytrace data and uses the developed routines to update a merit function.
These results demonstrate not only the successful reduction of parasitic light and the resulting subimage cross-talk in a lenslet-array-based thin imager, but also the opportunity to provide spectral and polarimetric resolution, using cheap and easily aligned filter arrays. This design eliminates the physical signal separator of earlier designs, such as TOMBO, thus preserving a 100% fill factor at the lenslet array super-aperture, and it also provides spectro-polarimetric sensitivity. One should also be aware of the trade-offs inherent in such a scheme between spatial and spectral resolution. More spectral resolution implies more channels with more filters and, for the same detector size, lower spatial resolution per channel. To relieve this problem, the detector resolution must be increased, or several thin imagers would have to be used together.
Such an optical configuration can be integrated with the detector and electronics into a small, mass-producible package that could prove useful in areas such as micro-unmanned aerial and ground systems where weight, form factor, and simplicity become very important. Of course, proper post-processing of the resultant spectro-polarimetric data could be required to fuse the subimages into a useful whole.
Devices of this sort are potentially very useful as small form factor, specifically tailored, and conformal imaging (or even nonimaging) spectro-polarimetric sensors on very small platforms such as micro-UASs. A tightly integrated thin imager of this class could conform to the airframe, comprising in essence a very thin sensor system “painted” onto the aircraft. One could imagine many other applications, such as an ultra-lightweight solar vector magnetograph integrated as a thin sheet onto the outer surface of a satellite. Depending on the necessary polarimetric data, this could require the inclusion of circular polarizers in the filter array. This structure could also have an impact in optical testing, specifically in the Shack-Hartmann lenslet array sensor design.
This work was supported by the University of Alabama in Huntsville Electrical and Computer Engineering Dept.; by Jenoptik, Huntsville, AL; and by Code V® through its provision of free optical design software to students.
Josh Walters attended the University of Alabama in Huntsville, where he received his BS in 2009 and MS in 2012 in optical engineering and electrical engineering, respectively. In 2012 he won the annual Robert S. Hilbert Memorial Optical Design Competition for his entry concerning nonsequential modeling lenslet array system design, upon which his MS thesis was based. Prior to entering graduate school he worked for Trideum corporation developing models of bio-inspired optical systems. In the fall of 2012 he joined Torch Technologies, Inc. working in the area of system modeling and simulation supporting the US Army AMRDEC.
Brian Robinson received his PhD in optical science and engineering from the University of Alabama in Huntsville in 2004. His current activities include the provision of optical testing, alignment, design, and fabrication support for various normal and grazing incidence EUV and soft X-ray imaging and spectroscopy spaceflight systems at NASA MSFC; consulting in support of the Air Force’s ground-based Improved Solar Optical Observing Network through development of test and calibration procedures for Fabry-Pérot tunable imaging filters; analysis of the impact of apodization and cavity errors in tunable filter systems on solar astronomy data; and research into unconventional optical systems, including multiaperture and compressive imagers.
Patrick J. Reardon received his BS in physics from DePaul University in 1986, then his MS and PhD in physics from UAH in 1990 and 1993, respectively, performing research in optical design techniques for analogue optical computing. He was the Chief Optical Systems Designer at Teledyne Brown Engineering for three years where he worked on analogue optical computing, diffractive and micro-optics technology, IR seekers, IR zoom systems, and an optical system for a space borne protein crystal growth experiment. As a consultant, he designed novel intraocular lenses and developed metrology systems for testing them. He briefly joined Johnson & Johnson in Roanoke, VA, as the Manager of Optical Design in their Progressive Addition Spectacle Lens team. Then he joined the Center for Applied Optics where his work spans the fields of polarimeter calibration, space based LiDAR, large optics metrology, eye oximetry, and optical systems design.