1.

Introduction

Nontraditional optical systems offer a larger, richer design space than traditional arrangements. Some are inspired by the optical imaging systems of invertebrate animals.^1–3 Compound apposition and superposition eyes are two broad categories. In this research, a lenslet array serves as the basis for a thin, compound imaging apposition optic that, owing to its multichannel nature, admits a diversity of spectral filters and linear polarizers. Circular polarizers could be included in future systems⁴ to open up the possibility of obtaining full Stokes data. The inclusion of multiple filters deals with the important problem of subimage crosstalk inherent in multichannel optics. This architecture improves upon the thin-observation module by bounded optics (TOMBO)⁵ which also employs a lenslet array. The TOMBO device includes three layers: lenslet array, separation layer, and photodetector array. The image formed on the photodetector comprises the multiple subimages that are formed by the lenslet array. This three layer architecture is shown in Fig. 1.

Fig. 1

Architecture of TOMBO sensor from Ref. 5.

The images formed by the lenslet array have lower angular resolution individually, but through multiframe super-resolution these low resolution images may be combined into a single high resolution image.⁵ It is worth noting that Ref. 5 contains a caveat concerning the TOMBO system describing a signal separation scheme that employs two like polarizer arrays arranged at the aperture and focal planes, each array containing alternating horizontal and linear polarizers. This would prevent crosstalk between a subfield and four laterally adjacent neighbors, but not the diagonal neighbors or nonadjacent neighboring channels. In the course of this paper, it will become obvious that this is insufficient to effectively deal with crosstalk.

Cross-talk results from light rays entering one subaperture at a high field angle and falling within the projected image area of another subaperture. The TOMBO designers use a physical signal separation layer to eliminate the cross-talk by absorbing rays that originate from points outside the subimage field of view. The drawbacks of this strategy include reduction in the effective aperture (especially since circular lenslets are employed) and in the field of view of the system by the separator walls.

This investigation shows that it is possible to reduce or even eliminate cross-talk using a combination of polarizers and spectral filters. As a natural consequence of this simple strategy the imager attains the ability to provide spectro-polarimetric resolution, and a larger, richer design space is opened up to the optical designer. Others have demonstrated multispectral imaging^6,7 via lenslet array architectures, but they still used a signal separator and had less than 100% fill factor.

The purpose of this paper is to illuminate this simple strategy for eliminating cross-talk while creating a small spectro-polarimetric imager, to describe the tools developed in Code V® for modeling the system, and to convey the results of this preliminary modeling. Section 2 will briefly introduce the geometry of the lenslet array as well as the effect of cross-talk. Section 3 discusses the bare lenslet array model, which was the starting point for all designs. In Sec. 4, we present the first, simpler design involving only RGB filters. Section 5 presents the design including more, narrower filters to suppress parasitic light over a larger FOV. In Sec. 6, we provide concluding remarks as well as possible directions for future work.

2.

Subimage Cross-Talk in Lenslet Array Imagers

Figure 2 schematically illustrates three simple rays paths that constitute cross-talk in a system without subfield stops. Ray 1 traverses the lenslet through a single channel, but falls on the image plane within an adjacent channel. Ray 2 enters the lenslet array through one channel and leaves the rear of the array through an adjacent one. Ray 3 undergoes multiple partial reflections and contributes to scattered light. There are several other, higher order scenarios that can contribute to crosstalk and stray light.

Fig. 2

Cross-talk in a lenslet system.

All three of these scenarios have the net effect of multiplexing and corrupting the image data. In the presence of detector noise and limited dynamic range, the resulting image is difficult, if not impossible, to satisfactorily de-multiplex. The lenslet array can be seen as spreading the incoming light into multiple spatial modes, a process that, due to the second law of thermodynamics, is not completely reversible.⁸ Methods of cross-talk elimination based solely on post-processing are not, therefore, tenable. This particular issue also happens to be a focus of research in Shack-Hartmann sensor design.⁹

3.

Optical Model of Bare Lenslet Array

In this investigation we used Code V® to perform nonsequential ray tracing of this multichannel optic. The model is based upon an in-house, 100% fill factor, square aperture, fused silica lenslet array graciously provided by Jenoptik Optical Systems in Huntsville, AL that had a 1 mm pitch, 1.5 mm thickness, 9 mm focal length, and equiconvex surfaces. The full, lateral field of view of a single channel is, therefore 6.36 deg.

A $5\times 6$ array structure was constructed based on the pitch, focal length, and thickness of the array. Figure 3 shows a Code V® layout of the nonsequential element.

Fig. 3

3-D view of lenslet array model: (a) shows 45 deg view, (b) shows $xy$ planar view.

The effective field of view (EFOV) for the lenslet array is defined by the rays of the highest field angle that make it to the focal plane of the bare lenslet array, i.e., in the absence of any subaperture filters. The model that includes $2\pi$ steradian illumination is restricted to this EFOV to reduce computation time. The object plane was located a finite distance of 1500 mm from the array, because it is simpler than modeling collimated ray bundles from infinity in Code V® and the distance places the object well beyond the hyperfocal distance for the 7.5 μm pixels of the $640\times 480$ model detector plane. The diagonal maximum effective half-angle field of view (EFOV) is 49 deg. This is calculated as the inverse tangent of the diagonal length of the detector over the focal length of a single lenslet. At this angle, rays from a lenslet at the very corner of the array will make it to the opposite corner of the array. Higher angle rays do not reach the focal plane. Figure 4 illustrates the progression of the spot diagram for the central lenslet as the point source reaches the edge of the effective field. To illustrate the image multiplexing problem presented by a bare lenslet array with no signal separation layer or filters, a small, sparse, bulls-eye shaped, Lambertian object was traced, via Monte Carlo ray tracing, from the established object plane. Figure 4 shows the concentric, constant radiance rings.

Fig. 4

User-created object for image simulation.

The innermost ring is inscribed within the field of view (FOV) of a single 1 mm subaperture while the outer two rings extend only to the middle of an adjacent channel. The wavelengths traced extended from 500 to $600{\mathrm{nm}}^2$ in 10 nm increments. The total number of rays traced was twenty-two million. The detector plane in the model was $640\times 480$ with $7.5{\mu \mathrm{m}}^2$ pixels.The results of the ray trace are presented in Fig. 5. The expected cross-talk between adjacent subapertures is clearly evident. If the input field were larger and less sparse the image would be completely unintelligible.

Fig. 5

Irradiance output of bare lenslet array simulation.

4.

Preliminary Model Including Subaperture Filter Arrays

A simple, preliminary model based on idealized RGB filters was constructed in order to develop our modeling strategy and algorithms used in the simulation, to tease out any simple but unforeseen problems, and to probe the ability of the normal illumination roll-off to prevent crosstalk between nonadjacent channels. By combining nonoverlapping bandpass filters with linear polarizers in one of two orthogonal orientations, one can create an ensemble of incommensurate filters so that light passing through any one filter would be blocked by an unlike filter.

By positioning like arrays of polarizers and filters at both the front of the lenslet array and at the focal plane one can mitigate the subimage crosstalk illustrated in Fig. 2. Note that this arrangement simultaneously affords the important benefit of spectro-polarimetric resolution along with the subimage cross-talk suppression.

Figure 6 above shows an exploded, three-dimensional view of a $5\times 6$, 1-mm pitch lenslet array with filter arrays included at the entrance aperture and focal planes. Elements in the filter arrays consist of RGB, nonoverlapping bandpass filters each with a linear polarizer in one of two orthogonal orientations. This combination yields a total of six different filters. Mutually incommensurate filters can be arranged in the first row. If successive rows are formed by cyclically shifting the first row to the right by two places, one can insure that no subaperture filter is adjacent to any like filter. This should allow the suppression of cross-talk, at least for subfields corresponding to $3\times 3$ subaperture regions of the array. This is shown in Fig. 7, where 1 stands for Filter 1, 2 for Filter 2, 3 for Filter 3, H for horizontal polarizer, and V for vertical polarizer.

Fig. 6

3-D view of proposed polarizer/filter array structures.

Fig. 7

Six-Filter arrangement for design D1.

The polarizers used in the Code V® model were linear polarizers with a reasonable 0.99 attenuation factor. Figure 8 illustrates the extinction provided by crossed polarizers of 1.00 attenuation factor. The crossed polarizers have a 100% attenuation and fully block on-axis rays, but not rays at higher field angles.¹⁰ This demonstrates the inevitable leaking of light through crossed polarizers at high angles of incidence.

Fig. 8

Pupil maps showing polarization attenuation for 100% attenuation.

Polarizers of this size are eminently manufacturable. Micropolarizer fabrication techniques have been developed that have resulted in elements as small as 0.5 μm.^11,12

The three bandpass filters were modeled as idealized top hat filters with individual bandwidths of 30 nm spanning the wavelength range of 500 to 600 nm. The spectral response of the three idealized filters is shown in Fig. 9.

Fig. 9

Filter spectral response.

The three-filter, two-polarizer design described above, designated D1, was modeled using the same three-ring object used in the model of the bare array. The first iteration of this model included filters of thickness equal to 0.76 mm. This was based upon the actual thickness of some sheet polarizers and filters in the laboratory. Modeling showed that this finite thickness contributed to the parasitic component of the transmitted light, as it caused some rays near the edges of the subapertures to miss the filters as shown in Fig. 10(a). We reduced the thickness to 10 μm, which is more in line with thin film technology but still conservative, and performed a raytrace with a single channel of the front filter illuminated and all but the corresponding channel in the rear filter array open. With the realistically thinner filters, the leakage of rays is stopped, but the polarization leakage remains in the adjacent channels with like spectral filters as is evident in Fig. 10(b). It is evident that the thickness made an order of magnitude difference in the peak transmitted parasitic light for the channel. Design D1 was, therefore, modified to include the thinner, and more realistic, filters.

Fig. 10

Detector irradiance plots showing parasitic light for (a) 0.76 mm and (b) 10 μm filter thicknesses.

Figure 11 shows results for both the original, bare array and for the final D1 configuration. One can see that the subimage crosstalk has been largely eliminated.

Fig. 11

Simulation results for (a) original and (b) filter array design.

Geometric projection errors between the crossed polarizers still result in some residual cross-talk between adjacent channels. A little crosstalk is also still evident at the outer edges of the focal plane, as the detector (modeled after a detector in our lab) is slightly larger than the lenslet array.

5.

Addressing the Problem of $2\pi$ Steradian Illumination

The preliminary model, involving only a few wideband filters and linear polarizers successfully demonstrated stray light rejection for the small field subtended by the sparse bulls-eye object. In many applications, however, the sensor may be exposed to $2\pi$ steradian illumination. This might be the case, for example, if the sensor were installed as a conformal element in the airframe of a small unmanned aerial system.

To simulate full hemispherical illumination and investigate worst-case parasitic light rejection, the object needs to be uniform and effectively fill a $2\pi$ steradian field. To minimize computing time, we used a rectangular, Lambertian object at 1500 mm distance that filled the effective field of view, as determined in Sec. 3. Higher angle rays do not contribute to the light received at the focal plane and need not be traced.

Under these conditions the stray light rejection capabilities of design D1 were examined again. In this analysis only a single filter channel at the lenslet array was illuminated and all but the corresponding filter channel were open at the detector. Such an arrangement would show only the parasitic transmittance of that single channel. Figure 12 shows the results and demonstrates that this simple design is insufficient to effectively suppress subimage crosstalk between nonadjacent channels. The ${\cos }^4\theta$ illumination roll-off does not provide sufficient crosstalk suppression between nonadjacent channels. For a larger array, or one with a large EFOV, it should lead to better suppression for channels that are very far apart.

Fig. 12

Log plot of normalized irradiance receiver data.

In Figure 12 the view is from behind the detector, as shown in Code V®, so that it is a mirror image of Fig. 7. The strongest parasitic signal, or cross-talk occurs, of course, in channels with filters that are the same as the filter in the illuminated channel. A small amount of cross-talk also occurs in channels with like spectral filters and orthogonal linear polarizers. This, again, is due to the presence of a residual orthogonal polarization component for higher angle rays passing through the front polarizers.

Note also that only the average value of the parasitic light is displayed in each channel, so that it is not spatially resolved within the channel. This is the because parasitic signal is very uniform over the channels exhibiting cross-talk and the numbers of rays reaching the focal plane in these channels under these conditions was necessarily quite small, though a large total number of rays was traced. The raw detector data, therefore, suffers from a kind of “shot noise” that obscures the true nature of the parasitic signal if it is not averaged out within each channel. The alternative is to trace an inordinately huge number of rays and take a vast amount of computing time to reach what would effectively be the same conclusion.

The previous results revealed that a greater diversity of incommensurate filters is necessary to suppress crosstalk for the flat lenslet array and focal plane for exposure to $2\pi$ steradian illumination. To more precisely characterize the rejection of parasitic light and suppression of the corresponding subimage crosstalk, we define the imager’s signal-to-noise-ratio (SNR) as the total power in the desired, direct signal divided by the total power in the parasitic signal. In order to calculate the SNR, we devised a means to quantify both the direct and parasitic signal. This method relies on the notion that the detection process is linear in irradiance and not saturated or otherwise suffering from nonlinearities in its responsivity.

As mentioned before, a light ray that enters a channel and propagates through its corresponding channel at the image plane filter is considered part of the direct light, or the signal. The size of a direct subimage is the same size as the corresponding aperture to prevent overlapping of images. To simulate direct signal through any single channel only that channel in the model is illuminated at the lenslet array, and only its corresponding channel is open at the image plane, as illustrated in Fig. 13. In other words, it is like masks are applied at the aperture and focal planes.

Fig. 13

Illustration of direct signal from a single channel.

To obtain the total direct signal for the entire array, under effective $2\pi$ illumination, this raytrace procedure is repeated for each channel and the corresponding distributions of light at the detector plane for all such raytraces (in terms of number of ray intercepts in each pixel) are summed at the end.

The parasitic light for any one channel was determined by opening that channel in the filter at the lenslet array and leaving all channels in the focal play filter array open, except for the corresponding channel, so that the complementary mask is applied at the focal plane. Figure 14 illustrates this. Once again, the parasitic irradiance distribution for each individual channel is summed at the end to obtain the total distribution.

Fig. 14

Illustration of parasitic signal from a single channel.

The SNR was defined as the sum of the energy in the direct pixels, obtained in the manner just described, divided by the sum of the energy in the parasitic pixels.

In Eq. (1), $M$ is the number of rows in the data, $N$ is the number of columns in the data, $D(i,j)$ is the energy (total number of ray intercepts) in the direct case at pixel $(i,j)$, and $P(i,j)$ is the energy in the parasitic case at pixel $(i,j)$.

Following this procedure, the SNR for design D1 under hemispherical illumination was determined to be 0.3327. This shows quantitatively that the design is unable to reject stray light and successfully image objects within the effective field of view. This is further illustrated by the full-field en-squared energy (FE) for the single channel incident light described in Fig. 15. For this channel, less than 30% of the energy propagating to the receiver is direct.

Fig. 15

FE plot for incident light on channel (3, 3) of design D1.

Using the method outlined above for quantifying direct and parasitic signal, the same $5\times 6$ aperture lenslet array was analyzed under effective $2\pi$ steradian illumination, but this time with 15 different spectral filters to insure no two filters were alike once the linear polarizers were added. Both the direct and parasitic raytrace routines (60 in all) were captured in a Code V® macro so that the entire analysis could be accomplished in a single run without user intervention. The macro recorded focal plane data for each channel in a buffer and exports it to a. dat file. Each of the 60 raytraces was saved as a Matlab array, and all were summed to obtain the direct and parasitic energies.

The spectral filters in this model were based on a more realistic Lorentzian profile,^13,14 defined as follows:

In Eq. (2), $x_0$ is the center wavelength and $\mathrm{\Gamma }$ is the full width half maximum (FWHM) of the response. The filters were modeled with a FWHM of 20 nm with the peak transmittances spaced evenly from 480–1000 nm. The value of $P$ was chosen so that the peak transmittance was 1.0. The spectral transmittances of the filters are superimposed in Fig. 16. The figure shows that the profiles of spectrally adjacent filters cross at a common 20% transmittance point. This overlap was included in the model simply to take into account the possible overlapping of real filters. The filters are labeled Filter 1 through Filter 15 starting with the 480 nm filter and ending with the 1000 nm filter.

Fig. 16

Transmittance plot of modeled filters.

Horizontal and linear polarizers are labeled as H and V, respectively. The next step in the process was to arrange the filters and polarizers in two like arrays, one at front of the lenslet array and one at the focal plane, as before. It is important to note that the design presented in this work does not necessarily reflect the optimum arrangement of filters and polarizers. A design algorithm to optimize the filter array, in terms of arrangement and spectral profiles, is a fascinating subject of work to come—work based partly on previous research done at the Center for Applied Optics on evolution strategies optimization of a multiple Fabry-Pérot tunable filter system.¹⁵ Figure 17 shows the design, called D2, chosen for simulation.

Fig. 17

15-filter design D2.

The raytrace included twenty-nine wavelengths, one corresponding to the peak of each filter and one to each of the crossover points in Fig. 15. The number of rays traced per wavelength was 500,000. With twenty-nine wavelengths, this results in a total of 14.5 million rays traced per channel for both the direct and parasitic raytraces. This results in a very lengthy computation time for this nonsequential raytracing task in Code V®’s Illumination option.

An image of the composite data for either the direct case or the parasitic case does not provide any insight into the design, since a uniform source was used as the standard to determine the system SNR and the resulting images also appear uniform and uninformative. Once again, although several million rays were traced, it was still not enough rays to overcome the simulation “shot noise” that results in significant pixel-to-pixel fluctuation in signal and renders a spatially resolved map of SNR rather useless. To better visualize the effects of cross-talk on the system in an intelligible way, the results from raytracing only the (3, 3) channel are once again presented. These results are shown on a log scale in Fig. 18. The SNR value of this design, calculated using the same method as the previous design is 15.1.

Fig. 18

Log plot of normalized irradiance data for 15-filter design D2.

This design more effectively eliminates cross-talk through the suppression of parasitic light. The FE plot of Fig. 19 further illustrates that the FE for the (3, 3) element is now around 90%.

Fig. 19

FE plot for incident light on channel (3, 3) in 15-filter design D2.

6.

Conclusion and Future Considerations

An important goal of this investigation was to develop a modeling technique to accommodate multiaperture, nonsequential designs, using a reconfigurable array of subaperture filters. This goal was achieved through the development of Code V® macros that autonomously execute the computationally intensive raytrace, along with Matlab routines that take detector data from the Code V® buffer and process it into a merit function for the imaging system. A simple, standardized and practical merit function was devised along with a simple technique for calculating the figure of merit, which both significantly reduces the computation time and provides an intuitive understanding of the performance. The nonsequential model with the developed macros and routines is easily modified for future investigations and can be included in an optimization loop, likely captured as a Matlab routine that calls Code V® to provide raytrace data and uses the developed routines to update a merit function.

These results demonstrate not only the successful reduction of parasitic light and the resulting subimage cross-talk in a lenslet-array-based thin imager, but also the opportunity to provide spectral and polarimetric resolution, using cheap and easily aligned filter arrays. This design eliminates the physical signal separator of earlier designs, such as TOMBO, thus preserving a 100% fill factor at the lenslet array super-aperture, and it also provides spectro-polarimetric sensitivity. One should also be aware of the trade-offs inherent in such a scheme between spatial and spectral resolution. More spectral resolution implies more channels with more filters and, for the same detector size, lower spatial resolution per channel. To relieve this problem, the detector resolution must be increased, or several thin imagers would have to be used together.

Such an optical configuration can be integrated with the detector and electronics into a small, mass-producible package that could prove useful in areas such as micro-unmanned aerial and ground systems where weight, form factor, and simplicity become very important. Of course, proper post-processing of the resultant spectro-polarimetric data could be required to fuse the subimages into a useful whole.

Devices of this sort are potentially very useful as small form factor, specifically tailored, and conformal imaging (or even nonimaging) spectro-polarimetric sensors on very small platforms such as micro-UASs. A tightly integrated thin imager of this class could conform to the airframe, comprising in essence a very thin sensor system “painted” onto the aircraft. One could imagine many other applications, such as an ultra-lightweight solar vector magnetograph integrated as a thin sheet onto the outer surface of a satellite. Depending on the necessary polarimetric data, this could require the inclusion of circular polarizers in the filter array. This structure could also have an impact in optical testing, specifically in the Shack-Hartmann lenslet array sensor design.