The most common form of system performance check for thermal imagers is minimum resolvable temperature (MRT). Four-bar targets of various sizes are viewed one at a time. The minimum temperature between bar and space for bar visibility is the MRT. Two problems make laboratory MRT difficult to predict. First, because MRT is supposed to represent the best achievable sensor performance, the operator is encouraged to change sensor gain and level for each bar pattern size. This means that the imager is not in a single gain state throughout the MRT procedure. Second, aliasing of the bar pattern depends on sample phase; this makes the MRT for sampled imagers difficult to predict.
This work describes a new model for predicting laboratory MRT. The model accounts for variation of the sensor gain and level during measurement. Also, the model includes the visual bandpass properties of human vision, permitting sampled imager MRT to be accurately predicted. These model changes result in MRT predictions significantly different from previous models. Model results are compared to laboratory measurements.
Section 2 describes model theory. Section 3 compares model predictions to measured data. The comparisons in Sec. 3 demonstrate that the model predicts the wide variation in MRT that results when the imager gain state is not controlled or measured. Section 3 also shows that model predictions are accurate when imager gain state is known. Discussion and conclusions are in Sec. 4.
A model for predicting the effect of blur and noise on sine wave grating detection is described in Refs. 1, 2, 3, 4. To predict MRT, the sine wave model is adapted for bar patterns. Section 2.1 briefly describes the sine wave threshold model. Section 2.2 describes the threshold model for bar patterns. Section 2.3 introduces the effect of gain and level in establishing threshold contrast. Section 2.4 discusses the effect of sampling.
Effect of Noise on Sine Wave Threshold
This section describes a model that predicts the effect of noise on sine wave discrimination threshold. The model starts with measured naked eye thresholds, and then estimates the threshold elevation that results from adding imager blur and noise.
A function describing sine wave threshold for each spatial frequency and display luminance is called a contrast threshold function (CTF). Barten provides a numerical approximation to measured naked eye CTF.5 Eyeball modulation transfer function (MTF) is also needed to predict the effect of noise on threshold. Formulas to predict eyeball MTF are taken from Stefanik’s distillation of the data in Overington.6, 7 Starting with naked eye CTF and eyeball MTF, the model predicts the contrast threshold function that results from adding imager blur and noise.in fL and the square-root of the angular display size in degrees.
Eyeball MTF is predicted by multiplying optical, retina, and termer MTF. Optical MTF depends on pupil diameter. Pupil diameter versus light level is given by Table 1. For each pupil diameter, the parameters and are given by Table 2. Equation 4 gives optics MTF.
Pupil diameter in millimeters versus light level.
Parameters for optics MTF.
|Pupil diameter mm||f0||i0|
For statically presented stimuli, the visibility of foveally presented signals is established in the visual cortex.8 The effect of display noise can be explained by assuming the brain is taking the root sum square of display noise and some internal eye noise.9 Before threshold is established, display signal and noise are blurred by eyeball MTF and bandpass filters in the visual cortex.10, 11, 12, 13 In other words, after filtering by the eye and visual cortex, display noise is the root sum squared with eye noise in the visual cortex.
Equation 7 predicts for horizontal gratings. A similar formula is used for vertical gratings.is the 169.6 root-Hertz mrad for reflective models, is the 862 root-Hertz mrad for thermal models (see text), SMAG is the system magnification, is the vertical spatial frequency in , is the noise affecting threshold at grating frequency , is the noise spectral density in fL mrad, is the system MTF from scene through display, is the eyeball MTF, are filters in the visual cortex, and is the MTF of display blur.
If the noise is spectrally flat, then is the standard deviation of the display noise in fL taken over an angle in object space of one milliradian horizontally by one milliradian vertically and for one second.
The 169.6 value of is for models in the visible and near infrared spectral bands. Display luminance is proportional to scene luminance, and noise is proportional to the square root of the photoelectrons generated by scene luminance. In these models, signal is the integrated value of electron flux for a time . Noise is the square root of the signal at . The 169.6 value of results from these assumptions.
For thermal models based on detectivity, equals 862. The different results from different signal scaling in the detectivity models. In these models, noise is proportional to the square root of photoelectrons, but the display luminance is set to an arbitrary value Kelvins to enhance the contrast of thermal targets. Signal is taken as the one-second flux from , and noise is scaled up such that the signal to noise at one second matches the signal to noise at time . This practice of associating the signal to noise at time with the flux after one second changes the value of .
is taken from Barten, who created a numerical fit for the visual cortex filters by using psychophysical data14 [See Eq. 9]. In that equation, is the frequency of the sine wave grating, and is a dummy variable used to integrate over noise bandwidth.
Effect of Noise on Bar Pattern Threshold
Figure 1 plots the display intensity for an imager viewing a per milliradian four-bar pattern. The superimposed curve represents the bar signal after blurring by the eyeball MTF and visual cortex bandpass filter. The amplitude difference between a center bar and the adjacent space closer to the edge of the bar pattern is what determines bar pattern visibility. The amplitude difference between the locations indicated by the arrows is found and then compared to the threshold needed for sine wave detection.
The Fourier spectrum of the cortical signal is found by multiplying the Fourier transform of the four-bar pattern by the various filters. Amplitudes are found by taking inverse Fourier transforms. MRT is given by Eq. 14 using , , and from Eqs. 11, 12, 13, respectively. The difference substitutes for in Eq. 7. The following definitions are used:
is the bar pattern period in milliradian (mrad), is , is the bar width , is the bar length , and is the fractional intensity due to blur of bar length.Kelvin (K) is the change in scene temperature that raises display luminance from black to average. Imager gain is with units fL/K. Knowing and , variations in display luminance are related directly to variations in scene radiant temperature.
In Eq. 14, is calculated using Eq. 8 but substituting for . If detector noise is spectrally flat, then is the standard deviation of detector noise taken over one second and one milliradian both horizontally and vertically. has units K mrad. As explained in Sec. 2.1, the value of is 862.
In Eqs. 11, 12, note that is used to filter the four-bar Fourier transform but is not. We are comparing the bar threshold to CTF measured with gratings that are essentially a single frequency. Eyeball MTF degrades threshold during CTF measurements, but the visual cortex filters do not. Because of the Fourier spectrum of the bars, the Eq. 9 visual cortex filters do affect bar threshold. Since linearity is assumed, bringing the visual cortex filters forward to the display is allowed mathematically.
The factor is needed because high frequency MRT bars are short. High frequency four-bar patterns are hard to see, partly because of reduced spacing between the bars and partly because of the short length. CTF gratings are long and have constant length regardless of frequency. accounts for the effect of bar length on threshold.
Effect of Gain and Level on Establishing Threshold
When calculating laboratory MRT, imager gain and level are sometimes adjusted to optimize detection of the bar modulation (see Fig. 2). The imager is adjusted such that the bar modulation fills half the dynamic range of the display. Since is defined as the temperature difference that raises the display from black level to average luminance, the following relationship holds at threshold.14. Equation 16 is used when gain and level are optimized for each bar pattern.
Predicting Sampled-Imager Minimum Resolvable Temperature
The Fourier transform of the displayed image of a sampled bar pattern is calculated by replicating the presample frequency content at multiples of the sample frequency and then filtering with the postsample MTF.15 Let be the imager prefilter MTF and be the imager postfilter MTF. Prefilter MTF includes optics, detector, and perhaps line-of-sight jitter. Postfilter MTF includes display MTF and digital processing. is also a postfilter. Again, eyeball MTF is not in the postfilter because we are predicting contrast on the display. The eye/brain bandpass filter is included, because the MRT stimulus (a four-bar pattern) is different than the stimulus used to generate CTF data (a sine wave). Equation 17 assumes that only the first three sampling replicas contribute to the image in practical situations; the sum over is theoretically taken from minus infinity to plus infinity.
The sample phase is equal to , where is the offset of sample position from the bar center. MRT is found for both zero sample phase and sample phase. That is, MRT is found for the case where a sample point is at the center of the bar and also for the case where the sample points have moved by half a sample spacing. The imaginary terms in Eq. 17 cancel, resulting in Eqs. 18, 19 for sample phase of 0 and , respectively.and now become:
However, since sampling can cause some unusual images, the presence of the center space must also be checked. Equation 22 gives the amplitude at the center of a four-bar pattern.or is used in Eq. 14 or 16 to calculate MRT. If either or is equal to or greater than , then at least one bar space is not visible.
An example is used to illustrate what happens during the sampling process. The example imager is a midwave infrared imager with F/3 optics and a horizontal field of view. The FPA has detectors on a pitch (100% fill factor). The display is a flat panel (square pixels) high and viewed with two eyes from . The half-sample frequency for this imager is per milliradian.
Figure 3 shows intensity in object space for an per milliradian MRT pattern. Figure 4 shows both the display intensity and the signal after filtering by the eye. In both figures, the abscissa is angle in milliradians and the ordinate is intensity. The bar spatial frequency is a factor of 1.1 bigger than the sensor’s half-sample frequency. The ( ) show the sample positions; one sample is at the center, so this is a zero sample phase. In Fig. 4, the small wiggles in the display intensity result from the numerical inverse transform; the intensity across any one display pixel should be uniform. At this sample phase, the four bars are easily seen by the eye.
In Fig. 5, the sample positions have moved half of a sample spacing relative to the bar pattern, and sample phase is now . Figure 6 shows the display intensity and intensity through the eye filters. The bar pattern is now spatially distorted due to the sample phase, and the four bars are not visible. Substantial bar modulation is visible, but not at the frequency of the MRT pattern.
Comparison with Measured Data
Figures 7, 8, 9 show the horizontal MRT for sensors A, B, and C, respectively. A summary of the characteristics of sensors A, B, and C is shown in Table 3. The data are the averages for two observers each performing two runs. In all of these figures, the abscissa is spatial frequency in cycles per milliradian, and the ordinate is a thermal contrast in K. The black squares are laboratory data. The solid line is the model prediction for high gain; this prediction assumes that the gain needed to achieve Eq. 16 is available. The dashed line is the model prediction for low gain; this gain is established by a scene contrast temperature of . The dashed and solid lines represent worst and best performance, respectively.
|Parameter||Sensor A||Sensor B||Sensor C|
|Field of view (degrees)|
|Focal length (centimeters)||3||5||5|
|Detector active area (microns)||20||20||40|
|Noise equivalent temperature (K)||0.05||0.1||0.3|
The laboratory data tends to be flat out to about half the sample frequency, and the model matches that behavior. Since neither the imager maximum gain nor the instantaneous gain state is measured and recorded during the MRT procedure, the results of a particular MRT test cannot be predicted in an absolute sense. However, the model does provide best case (high gain) and worst case (very low gain) predictions that bound the measured data. Further, imager C is noisy with noise equivalent temperature difference (NETD) of . In this case, the imager gain is near minimum at all times due to the noise, and predictions are a good absolute match to the data.
In the mid 1980’s, 25 production thermal imagers were thoroughly evaluated. The imagers were the first generation of the Apache Helicopter Pilot’s Night Vision Systems (PNVS) with the design parameters shown in Table 4.16 The evaluation included MRT performed by experienced laboratory personnel. A wide variation in measured MRT was observed.17 However, the reason for the variation in MRT could not be discovered.
List of hardware parameters for PNVS.
|Field of view||by 40 H|
|Detector array||180 by 1 common module|
|Detector size||41 by|
|Display||IHADSS helmet display|
In all cases, the sensor NETD, MTF, and system intensity transfer function were measured. In rare cases, problems with the display were discovered and the display replaced. The systems were evaluated for electromagnetic interference, and no problem was found. Further, the systems were tracked through an extended field exercise, and individual operators were questioned about their experience with the imagers. There appeared to be no correlation between user experience and MRT.18
Figure 10 shows MRT data from 15 of the 25 systems; data from the remaining ten systems are lost. The solid line shows predictions of the high gain model; the solid line represents best possible performance. The dashed line shows predictions for the low gain model. All data lie between the high and low gain predictions.
Comparison of Model to Minimum Resolvable Contrast Data
Minimum resolvable contrast (MRC) measurements on image intensifiers are made with the imager in a known gain state. This means that the variability associated with an unknown gain state is removed, and the model predictions should be a good, absolute match to MRC data.
The MRC data presented here are particularly meaningful for several reasons. The manufacturer provides accurate and independent measurement of the intensifier’s MTF, signal to noise, gain, equivalent background input (EBI), and responsivity. Also, bar illumination levels range from 2.88 E-6 foot candles to 3.39E-3 foot candles. This variation in illumination means that the tubes are operated from noise-limited to resolution-limited conditions.
Further, measurements are made both with and without laser eyewear protection that reduces the light to the eye by a factor of ten. The three I2 tubes represent both poor and good MTF, and each tube is operated at three gain levels. The combination of laser eye protection and three gains means that light to the eye varies from as little as 3.6E-4 fL to as much as 50 fL. This is an excellent dataset because of the controlled nature of the physical sensor data, the wide range of scene illuminations, and the large variation of light to the eye.
The data were collected by the Image Intensifier Team at the Night Vision and Electronic Sensors Directorate of the United State’s Army’s Research, Development, and Engineering Command. The three observers were very experienced and consistent in making limiting-light and MRC measurements.
The maximum output luminance from the imagers is limited by regulating current from the power supply. When the whole field of view (FOV) is illuminated, average output brightness is approximately 3 fL; this approximate value varies between tubes and gain settings. In this experiment, the resolution chart in the test setup only illuminates one tenth of the FOV. Maximum output brightness easily exceeds 30 fL and on one occasion reached 50 fL.
Table 5 gives the physical parameters for each tube and gain setting. Transmission of the laser eyewear is also shown. When eyewear is used, adapting eye luminance is intensifier tube output luminance multiplied by the eyewear transmission. The maximum output brightness is a limit for each tube gain and for the test setup. Generally, tube output brightness is proportional to input luminance and is much lower than the maximum. Responsivity has units of microamperes per lumen . T# is F# divided by the square root of transmission. EBI is dark current with units of lumens per square centimeter (lm/cm2).
Parameters for I2 devices.
|Tube||Gain||S/N||Resp μA∕lm||Max fLoutput||E/Wtrans.||MTF lp/mm|
|Objective focal ,||0.99||0.96||0.92||0.85||0.79|
|Eyepiece focal ,||0.99||0.95||0.91||0.84||0.78|
|for 0106; for 0104 and 7285|
Two bar chart contrasts are used, 1 and 0.4. Five chart illuminations are used: 2.8E-6, 1.09E-5, 1.06E-4, 1.03E-3, and 3.3E-3 foot candles (fc). The high contrast chart data include three tubes with three gains each and with and without eyewear for a total of 90 data points. The low contrast chart data include three tubes operated at 25,000 gain and no eyewear for a total of 15 additional data points.
The bar chart used for MRC measurements is shown in Fig. 11. These are three-bar patterns and therefore not the same as for MRT measurements. Equations 10, 11, 12 become Eqs. 23, 24, 25 when three-bar patterns are used. Sampling is not an issue for image intensifiers. and from Eqs. 24, 25 are used in Eq. 14 to predict MRC. For MRC, however, equals 169.6 and is replaced by the cathode electrons generated in one second over an object space field angle of one milliradian both horizontally and vertically. is essentially the square root of cathode electrons, except that a slight adjustment is made to include EBI.
Figures 12, 13, 14, 15 compare the laboratory data to model predictions. In each figure, the ordinate plots the observed bar resolution and the abscissa plots the predicted resolution. If the model was perfect, and if the signal to noise, gain, and MTF measurements of the tube and optics were perfect, then all the points in the figures would lie on a straight line where model equals data. Figures 12, 13, 14 plot the results for the high contrast three-bar chart. Each figure shows results for all gains of one tube. The data with and without eyewear are plotted separately. Figure 15 shows results for all three tubes viewing the low contrast bar chart.
The fit between model predictions and MRC data is excellent. The biggest discrepancy between model and data is for tube 7285 at spatial frequencies above . All of these predictions are heavily influenced by the single 0.04 MTF measurement (see Table 5). This is an old tube, and the MTF measurement is difficult. Even if this measurement is accurate, however, the overall fit between model and measured data is quite good.
Discussion and Conclusions
In previous years, MRT was considered a reliable indicator of imager performance. This belief persisted despite experience that MRT results varied without an apparent physical cause. No physical cause could be found for the wide variation in imager MRT shown in Fig. 10. There was no difference in performance reported by the operators. And yet, the author and other engineers involved in the evaluation felt certain that an imager problem existed, and that this was proven by the MRT results. Early MRT models were based strictly on signal to noise and had no dependence on imager gain and level. The extent to which imager gain and level affected MRT results was not understood.
On the other hand, the effect of sample phase on MRT results was recognized decades ago, and procedures developed to either optimize sample phase or minimize the impact of sampling during MRT measurement.19, 20, 21, 22 However, earlier MRT models did not accurately predict the change in MRT values as sample phase varied.
The new MRT model predicts the trends seen in laboratory data. That is, measurements are flat out to near the half-sample frequency, and then rise sharply. If the bars are at zero sample phase, MRT does not rise until substantially beyond the half-sample frequency. The model also correctly predicts a lower and upper bound for the MRT data. These bounds are based on imager maximum and minimum gains. The model match to sensor C is interesting because the sensor noise resulted in little difference between low gain and high gain model predictions. In this case, the absolute match between model and data is excellent.
The ability of the model to predict the results in Fig. 10 is particularly compelling. The variation in MRT is not due to the blur or noise characteristics of the imagers, and the MRT observers are highly trained and experienced. The model predicts that the variation in MRT results from arbitrary manipulation of the imager gain and level controls during measurements.
Absolute agreement between MRT model and data is unlikely until the sensor gain state information is measured and recorded during the MRT procedure. MRT is subjective and therefore likely to vary between individuals. However, a better procedure would provide more quantitative data and reduce the measurement to measurement variations. The good match between model predictions and MRC data demonstrates that predictable results are obtained when imager gain state is known.