Radiometric calibration of infrared imagers using an internal shutter as an equivalent external blackbody

Abstract. Advances in microbolometer long-wave infrared (LWIR) detectors have led to the common use of infrared cameras that operate without active temperature stabilization, but the response of these cameras varies with their own temperature. Therefore, obtaining quantitative data requires a calibration that compensates for these errors. This paper describes a method for stabilizing the camera’s response through software processing of consecutive images of the scene and images of the camera’s internal shutter. An image of the shutter is processed so that it appears as if it were viewed through the lens. The differences between the scene and the image of the shutter treated as an external blackbody are then related to the radiance or temperature of the objects in the scene. This method has been applied to two commercial LWIR cameras over a focal plane array temperature range of ±7.2°C, changing at a rate of up to ±0.5°C/min. During these tests, the rms variability of the camera output was reduced from ±4.0°C to ±0.26°C.


Introduction
Obtaining quantitative data from thermal infrared imagers requires a radiometric calibration to relate the camera output digital number to a quantity such as radiance or brightness temperature. This is most commonly done by measuring the output while the camera views two or more blackbody sources to create a relationship between the digital number and the radiometric value. This relationship often is assumed to be constant over some length of time; however, for microbolometer detectors that lack a thermoelectric cooler (TEC) to stabilize the focal plane array (FPA) temperature, the calibration must be updated nearly continuously. In such TECless imagers, the camera output responds to changes in both scene temperature and FPA temperature. There is a need for methods to frequently update camera calibration during operation.
A general method of updating the calibration is to periodically view one or more external blackbody sources. The level of correction depends on the number of sources used: a single source provides a one-point offset calibration, whereas two or more blackbody references allow for correction of both offset and gain. A number of variations are described in the scientific and patent literature for achieving radiometric calibration with TEC-less cameras. These methods include a per-pixel calibration at every FPA-temperature to be experienced during operation, 1 a polynomial curve fit to relate variation in output digital number to variation in FPA temperature 2 and/or lens-temperature, 3 alterations to camera operating parameters, [4][5][6][7] Kalman filters to update the gain and bias of each pixel as it drifts over time and with FPA temperature, 8 corrections based on measurements of the camera response over a wide range of temperatures, 9,10 corrections based on consecutive images of a constant scene and a shutter as the camera temperature changes to derive an FPA-temperature-dependent calibration, 11 an autoregressive moving average to compensate for dynamic changes in the camera's FPA temperature, 12 and at least one method to estimate and remove nonscene energy to enable calibration. 13 Our motivation for pursuing high-accuracy and high-precision calibration of compact, TEC-less thermal imagers was provided by research into the use of microbolometer imagers to implement infrared cloud imager (ICI) systems for measuring the spatial and temporal variations of clouds and their radiative properties. [14][15][16][17][18] The ICI data are calibrated radiometrically to allow removal of the atmospheric emission to isolate the cloud signature. Previous versions of this system used a blackbody calibration source to track the changes in camera calibration over time, [15][16][17] but in the newer, compact ICI systems there is no room for a blackbody. 14,18 Furthermore, in many other remote sensing applications, it is desirable to use a compact system that does not include a large, heavy, and expensive large-area blackbody source. Examples of such applications include monitoring beehives, 19 measuring vegetation to detect leaking CO 2 gas, 20 and airborne imaging. Therefore, a method of providing real-time correction of the radiometric calibration on field-deployed infrared imagers is needed.
We previously described a method that relies on characterizing the camera's response over a range of camera temperatures, 21,22 and in this paper, we describe a method that can be used in combination with or in place of that method. The method described here provides a real-time update to the radiometric calibration by using the camera's internal shutter as an equivalent blackbody source. This method is implemented by modifying internal-shutter images to compensate for the optics, thereby simulating an image of an external blackbody source. The following sections describe the mathematical development of this method, describe how to determine the required calibration coefficients, and illustrate the application of the method to calibrate a TEC-less microbolometer camera subjected to a systematically varying ambient temperature.

Camera Response Model
The method described here is presented in terms of a linear camera response, r sc , relating output digital numbers to scene radiance L sc . We illustrate the application of this method to an FLIR Photon 320 microbolometer longwave infrared (LWIR) camera, which responds linearly over a large range of scene radiance, in which the response can be described in terms of camera gain G and offset D, according to In a calibration, this equation is rewritten to express the scene radiance in terms of the output digital value with g c representing the calibration gain and o c the calibration offset Typically, g c and o c are determined in a laboratory setting through the use of blackbody source images at a variety of temperatures and knowledge of the camera's spectral response.

Offset correction from a single blackbody source
During deployment, a calibration can be updated with a single blackbody source. This reference is used to cancel the offset term, thereby reducing the FPA-temperature dependence. To do this, the digital responses to two scenes, r 1 and r 2 , are observed for a scene radiance to be calibrated, L 1 , and known blackbody-source radiance L 2 . Assuming a constant offset and gain for these images, the difference between r 1 and r 2 gives or The difference removes the dark signal or offset of the camera, and multiplying this difference by the calibration gain The blackbody source radiance L 2 can be calculated from the known source temperature and the spectral response of the camera and then added to Eq. (6) to determine L 1 This shows that the scene radiance, L 1 , can be determined if the calibration gain is known, and this result is independent of the changing camera offset. This single-blackbody calibration technique is typically applied with an external blackbody source, but our technique instead uses the internal camera shutter [which typically is used to perform a flat-field nonuniformity correction (NUC)] as an equivalent blackbody source.

Internal shutter as an equivalent external blackbody
The use of the internal shutter as an equivalent external blackbody source requires careful characterization of the lens and the shutter, which is typically located between the detector array and the lens. The radiometry is illustrated in Fig. 1 for a pixel viewing the blackbody and in Fig. 2 for a pixel viewing the shutter. The expression for the optical power detected at a pixel viewing the blackbody is given in Eq. (8), and the corresponding expression for a pixel viewing the shutter is given in Eq. (9).
In Eq. (8) describing Fig. 1, the pixel views the blackbody through the lens with the following parameters. P dl is the power detected at the pixel through the lens, L bb is the radiance emitted by the external blackbody at temperature T bb , A l is the area of the lens (pupil), Ω ifov is the projected solid angle of the pixel's instantaneous field-of-view determined by the pixel area and lens-to-detector distance, τ l ðT l Þ is the  temperature-dependent spectral transmittance of the lens, τ f ðT f ; λÞ is the combined temperature-and wavelength-dependent filter spectral transmittance and detector responsivity, L l ðT l Þ is the temperature-dependent lens radiance that includes emitted and reflected components, A p is the pixel area, and Ω l is the projected solid angle of the lens as seen from the pixel. Many of these terms are wavelength dependent, but, in this paper, we assume the dependence is dominated by τ f ðλÞ.
In Eq. (9) describing Fig. 2, the pixel views the shutter directly. In this case, P ds is the power detected by the pixel from the shutter, L s is the temperature-dependent radiance emitted by the shutter and Ω s is the projected solid angle of the shutter seen from the pixel.
If the camera were to view an external blackbody whose temperature was set equal to the focal-plane-array temperature, T fpa , assumed equal to the lens and shutter temperatures, and if the shutter had emissivity ¼ 1, then the ratio P dl ∕P ds would become The function L l ðT fpa Þ (the emitted and reflected radiances from the lens) can be described with the lens emissivity ε l , blackbody emission at the FPA temperature (assumed equal to the lens temperature) L bb ðT fpa Þ, the lens reflectance ρ l , and the FPA emissivity ε fpa Substituting this into Eq. (10) for a shutter with emissivity ¼ 1 yields This expresses the detected-power ratio of P dl (power from the lens) and P ds (power from the shutter) in terms of camera and lens properties. The temperature-dependent Eq. (12) can be determined through laboratory measurements. First, the detectedpower ratio is measured over a range of FPA temperatures in a laboratory by alternately viewing the internal shutter and a blackbody at the same temperature. Since there is a direct relationship between power on the detector and the digital response of the detector, the ratio can be measured using digital-number images of the shutter and blackbody in close temporal succession. Once the ratio is determined over a wide range of shutter (FPA) temperatures, a first-order linear model can be determined for each pixel (relating response to T fpa ). This model will be referred to as S R ðT s Þ, where T s is the FPA temperature at the time the shutter image was acquired. This model is used to convert images of the shutter, represented by response r s , to equivalent-external-blackbody images represented by response r bb through the following equation:

Method to Determine the Conversion Coefficients
The function for the shutter-temperature-dependent ratio S R ðT s Þ, used to convert an internal shutter image to an equivalent external blackbody image, can be calculated from a series of measurements made with the camera viewing a blackbody in a thermal chamber. The chamber is held at a constant temperature, while the FPA-temperature stabilizes the blackbody set-point temperature is continually adjusted to equal the camera temperature (i.e., the FPA temperature).
Once the camera temperature is stable and the blackbody has been at the camera temperature for a sufficient length of time, a series of images is taken, alternating between images of the shutter and images of the blackbody (taking care not to change the shutter temperature by flipping it too often). The flat-field function of the camera should be used during this data collection if it is to be used during deployment. Once a large data set has been collected at one FPA temperature, the thermal chamber temperature is increased. The process is repeated for multiple FPA temperatures, and the resulting data are used to calculate S R as the per-pixel ratio between the blackbody image and the shutter image at each FPA temperature; for example, at at and so on; (16) After collecting data at a sufficient number of points, a function S R ðT s Þ can be derived for each pixel. This function allows the shutter ratio to be calculated for any shutter temperature T s that lies within the range of the thermal-chamber measurements.

Shutter-Based Offset Correction with FPA-Temperature-Dependent Response
Using the shutter as a blackbody reference taken consecutively with the scene image will cancel out the FPA-temperature-dependent offset. To show this mathematically, first we start with the microbolometer system model from Eq. (1) and extend it to describe an FPA-temperature-dependent camera by redefining D and G as a sum of temperature-independent terms, D o and G o , and temperature-dependent functions, D t ðT fpa Þ and G t ðT fpa Þ In this case, the response to the scene becomes In Sec. 2.1.2, it was shown that an image of the shutter could be converted into an equivalent external blackbody response, r bb . Using Eq. (19), the response to that equivalent external blackbody can be described as Subtracting the blackbody image r bb from the scene image r sc and reducing terms gives We have found that the temperature dependence in the gain can often be described as simply a scalar multiplied by T fpa , in which case Eq. (21) can be reduced to where G tc is a constant term that can be derived as part of the system calibration process by including it within the system calibration matrix. This equation can be rearranged to express the scene radiance L sc in terms of the blackbody radiance L bb , measurements of r sc , r bb , and T fpa , and the camera response terms G o and G tc , as follows: In the particular case when G tc T fpa is negligibly small, Eq. (23) reduces further to The values of G o and G tc can be derived from a laboratory calibration, and the value of L bb can be calculated by integrating the Planck function over the spectral response of the camera, with the Planck function evaluated at the measured FPA temperature (assuming the FPA temperature accurately describes the shutter temperature). With these values determined, Eqs. (23) or (24) can be used to determine the radiance of the scene from the raw digital number images of the scene and shutter.

Method to Derive the Gain Terms G o and G tc
To determine G o and G tc , first a large set of shutter images and blackbody images is collected over a large range of shutter and blackbody temperatures. For each image within this set, the difference in response between the camera viewing the shutter and the external blackbody, Δr, is determined. Unlike the procedure in Sec. 2.2, the blackbody and the shutter should not be at the same temperature. Next, the radiance difference between the blackbody and a blackbody at the shutter temperature, ΔL, is calculated. From Eq. (22), the response differences are written as a matrix equation of the form 2

Final Calibration Function
Combining Eqs. (13) and (23) yields the final calibration equation This equation allows calculation of the scene radiance L sc from the laboratory characterization of the shutter-to-blackbody power ratio S R ðT s Þ and the laboratory characterization of the gain and gain FPA-temperature dependence (G o and G tc ), using consecutive images of the shutter and scene. This function can be further reduced if it is assumed that G tc is small, as in Eq. (24), in which case Eq. (27) can be reduced to 3 Application of the Calibration The calibration methodology described in the previous sections was applied to a Photon 320 long-wave microbolometer camera with an Ophir 14.25 mm, f∕1.2, athermalized lens. The camera was placed in an environmental chamber while viewing a blackbody calibration source whose temperature was varied between 10°C and 50°C. The chamber temperature was varied between 10°C and 30°C. During this experiment, data were collected in the following manner. First, a standard NUC was performed using the shutter in the manufacturer's intended fashion. Next, an image of the blackbody (scene) was recorded, along with the FPA temperature. Next, the camera shutter was closed and an image of the shutter was acquired. Figure 3 shows the blackbody temperature (black dashed line) and the camera FPA temperature (red solid line) measured during this experiment. It was assumed that the FPA temperature was an accurate representation of the shutter temperature. Images of the blackbody acquired during this experiment were calibrated using three different methods: (1) without correction for FPA-temperature dependence; (2) using the shutter as an equivalent external blackbody, assuming G tc was insignificant [as in Eq. (28)]; and (3) using the shutter as an equivalent external blackbody, including compensation for the FPA-temperature dependence of the gain [as in Eq. (27)].

Calibration Without FPA-Temperature Correction
The calibrated response from the camera without any FPAtemperature compensation is shown in Fig. 4 and a histogram of the error relative to the blackbody set-point temperature is shown in Fig. 5. For computational efficiency without spatial or temporal averaging, the histogram was calculated using five randomly selected pixels from each frame. These figures show that there are significant errors caused by the camera output changing in response to its own temperature changes instead of to just the changing scene.

Shutter-Based FPA-Temperature Correction
To calibrate the camera output with shutter-based FPA-temperature compensation, first it was assumed that G tc , the FPA-temperature-dependent component of camera gain, was small. The results of this method, using Eq. (28), are shown as a time series of calibrated data and blackbody set-point temperature in Fig. 6, and as a histogram of the error between the calibrated temperature and blackbody set-point temperature in Fig. 7 (both of these figures were created using five randomly selected pixels from each frame, as was done for Fig. 5). The shutter-based calibration process greatly reduced the FPA-temperature-dependent errors, and the temperatures reported by the camera closely follow the blackbody set-point temperature. The histogram of the calibrated data is centered near zero (0.097°C), and has a Gaussian-like distribution with a standard deviation of 0.33°C.
Finally, the shutter-based FPA-temperature compensation, with an FPA-temperature-dependent gain term, was applied to the images using Eq. (27). The results of this    Optical Engineering 123106-5 December 2014 • Vol. 53 (12) method are shown as a time series of calibrated data and blackbody set-point temperature in Fig. 8, and as a histogram of the error between the calibrated temperature and blackbody set-point temperature in Fig. 9. Relative to the results using Eq. (28), the histogram for these results exhibits a slightly larger mean error (0.25°C) and a slightly smaller standard deviation (0.24°C). Table 1 lists the errors of the data calibrated both with and without a shutter-based correction. Application of the shutter-based calibration method induces a slight increase in the spatial standard deviation across an image, suggesting that the calibration has a slight negative impact on the factory NUC. Table 2 shows the final 1 − σ uncertainty, assuming that the spatial and random errors had a covariance of zero. In this case, these errors are independent and combine in quadrature to yield the final system variability.

Summary of the Calibration Results
The same process was applied to another Photon 320 with the same lens model. The results were very similar (with final 1 − σ uncertainties of 2.91°C, 0.26°C, and 0.21°C). In both cameras, the lens was smaller than the camera, and therefore had less thermal mass. However, when this technique was applied to a Photon camera with an IEM 6.8 mm, F∕1.1 lens, the algorithm produced degraded results. This lens was much larger and heavier than the Ophir lens, and was, in fact, larger than the camera body. This suggests that the temperature-induced changes in the lens could be tracked using the FPA temperature for small lenses, while this no longer held true with larger lenses. Finally, we note that this method compares well with the method we published previously, 22,23 which is based on characterizing the changing response as a function of camera temperature. The temperature-responsecompensation method 22,23 generally produces smaller errors over time than the shutter-based method alone, but it is computationally more intensive and tends to increase the spatial uncertainty of the sensor. For comparison, use of the earlier published method with the same test data that produced an uncertainty in time of 0.24°C and an uncertainty in space of 0.04°C (total of 0.26°C) shown in this paper, produced an uncertainty of 0.12°C in time and 0.19°C in space (total of 0.22°C).

Discussion and Conclusions
This paper has presented a method for reducing large calibration errors that result when a TEC-less microbolometer camera is used in a situation where the camera temperature changes. The method is based on interpreting images of the internal camera shutter in terms of an equivalent external blackbody source. This method is computationally simpler than a previously published method that relies on characterizing the camera response as a function of camera temperature. 21 The previous method is essentially the same as including the nonzero temperature-dependent gain discussed in this paper, but without the use of an internal shutter. Use of the shutter method alone simplifies the real-time calibration, but as mentioned at the end of Sec. 3, the FPA-temperature    compensation method is more robust than the shutter method, as presently implemented.
To be applied to a camera, the method described here requires the following: (a) an internal shutter; (b) an internal sensor that measures the temperature at or near the shutter; (c) an internal sensor that measures the temperature at or near the detector array (this could be the same as the shutter-temperature sensor); (d) laboratory measurements of an external blackbody to derive the effect of the intervening optics between the shutter and the external scene; and (e) laboratory measurements to quantify the imager's gain and the change in gain with temperature.
When applied to a Photon 320 with an Ophir 14.25 mm f∕1.2 lens, both methods performed well and were able to produce highly accurate measurements during laboratory tests, AE0.26°C to AE0.33°C (depending on which correction method was used) for an FPA-temperature range of 20°C to 32°C. However, when applied to a Photon 320 with a larger aftermarket lens, this method produced a degraded uncertainty relative to the correction with a smaller lens, presumably because of its larger thermal mass. As presently implemented, this technique relies on the FPA temperature, which we believe is insufficient to estimate the temperaturedependent properties of the larger lens because of its larger thermal mass.