3.1.

Sampling Point Spread Function

The SPSF combines the effects of the optical PSF and the pixel response function (assuming that the PSF is approximately constant for a small part of the camera’s field of view around the pixel of interest). As an example, let us assume a sensor pixel that is uniformly sensitive to light inside its borders and completely insensitive to light outside the borders. Then considering just one spatial dimension, the pixel response function will look as shown in Fig. 2(a).

Fig. 2

Pixel response function shown (a) alone, (b) together with the PSF (blue), and (c) together with the corresponding SPSF (green).

The SPSF in the across-track direction can be measured by first placing a point source well outside the pixel and then moving the point source in small steps across the field of view, i.e., in the across-track direction, while measuring the signal in the pixel for each position of the point source.⁷ When the point source is positioned far from the pixel, only a small percentage of the energy will hit the pixel due to the shape of the PSF. When the point source is positioned closer to the pixel—or especially inside the pixel—a larger portion of the energy will hit the pixel. The signal registered by the pixel as a function of point source position is the SPSF. Here is an example: when moving the PSF shown in Fig. 2(b) across the camera’s field of view and measuring the signal in a pixel that has a response function as shown in Fig. 2(a), the resulting SPSF will be as shown in Fig. 2(c).

The SPSF, when normalized so that the area under the curve is equal to 1, tells us how large fraction of the light ends up in the pixel for a given point source position. In the example in Fig. 2(c), 30% of the light ends up in the pixel when the point source is at some distance outside the pixel ($x=-0.8$), whereas 85% of the light ends up in the pixel when the point source is positioned at the center of the pixel ($x=0$). A more general definition of the SPSF can be found in previously published literature,^5,6,9 as well as information about how the SPSF could be measured in two dimensions.

In a hyperspectral camera with perfect spatial coregistration, the SPSFs will be identical across the entire wavelength range for any given spatial pixel. However, in a real camera, the SPSFs will be somewhat different in each spectral channel: they will be shifted somewhat relative to each other due to keystone, they will have somewhat different width at half-maximums, and the finer details of their shapes will also be somewhat different.

3.2.

Method 1

Method 1 calculates the misregistration between two channels based on the measured SPSFs.⁶ This gives an upper bound on the absolute radiometric signal error for extended sources.¹⁰ Figure 3(a) shows an example of the normalized SPSFs for two channels $i$ (green) and $j$ (blue). The two SPSFs have different shapes and widths and are shifted compared to each other, which leads to misregistration. For instance, when the point source is at the left border of the pixel ($x=-0.50$), about 72% of the light in channel $i$ (green) ends up in the pixel, whereas only 22% of the light in channel $j$ (blue) ends up in the same pixel.

Fig. 3

(a) Example of the normalized SPSFs for two channels $i$ (green) and $j$ (blue). (b) Border between two materials A and B. The border is positioned at the crossing point between the two SPSFs.

In order to calculate the misregistration, method 1 considers a scene consisting of two materials A and B [see Fig. 3(b)], where A is bright and B is completely dark. The amount of light from material A that ends up in the pixel of interest is in this case proportional to the integrals of the green and blue curves—in the area where material A is present—for channels $i$ and $j$, respectively. The difference between these two integrals, i.e., the area $\mathrm{\Delta }w$ in Fig. 3(b), is said to be the misregistration.^5,6

The misregistration is always calculated for the case where the border is at the crossing point between the two SPSFs [as shown in Fig. 3(b)], since this is the position of the border that would give the largest error. The maximum misregistration for a given spatial pixel is, for this method, defined as the largest misregistration measured across all pairs of spectral channels,⁶ whereas the mean misregistration is calculated as the average across all channel pairs.

3.3.

Method 2

Method 2 calculates the misregistration errors directly from the point source measurements without first forming the SPSFs.⁷ The method is based on a very basic principle: simply determine “how much of the energy collected by the hyperspectral camera ends up in the correct pixel in the final data cube.” For each point source position within the pixel, the misregistration is calculated as the relative difference between the energy measured in a given spectral channel [solid line in Fig. 1(b)] and the average energy across all spectral channels [dashed line in Fig. 1(b)]. The maximum misregistration is defined as the largest measured misregistration across all point source positions, whereas the mean misregistration is calculated as the average across all point source positions.

For easy comparison with method 1, method 2 can also be explained in the framework of SPSFs. Key differences between the two methods can then better be understood.

Figure 4 shows how misregistration for method 2 is defined and calculated in the SPSF framework. Green and blue curves represent the normalized SPSFs for channels $i$ and $j$, respectively, as before. The black curve is the mean of the green and the blue curve and is used as a reference. For a given point source position $x$, the black curve shows the fraction of light that would be recorded in the pixel of interest in each channel if there were no keystone or PSF differences in the system (i.e., zero misregistration). For instance, for position $x=-0.20$ approximately 60% of the light in a given channel would end up in the pixel of interest in the ideal case.

Fig. 4

Misregistration as defined by method 2, explained in the SPSF framework for easy comparison with method 1. The misregistration calculations for method 2 are based on the assumption of a point source. Only positions inside the pixel of interest (white area in the figure) are used when calculating the misregistration.

In reality, the pixel of interest will get more light in some spectral channels and less in others. In the example above, for position $x=-0.20$, the pixel of interest gets too much light in channel $i$ (green) and too little light in channel $j$ (blue). How large the deviation is ($\mathrm{\Delta }{S}_x^i$, green vertical line) compared to the ideal value ($S_x$, black vertical line) tells us how large the errors in the system will be. The misregistration (for channel $i$ and position $x$) is, therefore, defined as the ratio $\mathrm{\Delta }{S}_x^i/S_x$. This ratio is calculated for all channels and for different positions within the pixel of interest in order to determine the maximum and mean misregistrations. Note that only positions within the pixel of interest (white area in the figure) are used for the calculations, thereby avoiding using the tails of the SPSFs where the signal may be very low and difficult to measure.

4.1.

Simulated Misregistration Components (i.e., Camera Imperfections)

The following camera imperfections were simulated:

The performance of 59 different cameras was simulated. Each camera has a total of 21 spectral channels where one or more imperfections are present to various degrees. All imperfections and their magnitudes in the tested cameras are described below. For details on the PSFs used for the simulations, see Figs. 5 and 6. Table 1 gives an overview of the imperfections in all tested cameras. Image noise (photon noise, etc.) is not included in the simulations, since we want to assess the impact of misregistration errors isolated from other possible sources of error in the data.

Fig. 5

Different real PSFs used in the simulations. The PSF width is expressed as the MTF value at the Nyquist frequency. (a)–(d) Symmetric PSFs with varying widths, (e), (f) two PSFs with the same width ($\mathrm{MTF}=0.25$) but distinctly different shapes (one very asymmetric and the other quite symmetric) at two different wavelengths ${\lambda }_1$ and ${\lambda }_{101}$, and (g), (h) two PSFs with the same width ($\mathrm{MTF}=0.50$) but, again, two distinctly different shapes at two different wavelengths.

Fig. 6

Example of morphed PSFs across 101 spectral channels. (a) The first spectral channel with a PSF based on the PSF in Fig. 5(e) ($\mathrm{MTF}=0.25$, ${\lambda }_1$), (b)–(e) PSFs morphed between the first and the last spectral channel, and (f) the last spectral channel with a PSF based on the PSF in Fig. 5(f) ($\mathrm{MTF}=0.25$, ${\lambda }_{101}$).

Table 1

Main characteristics of the cameras that are simulated.

4.2.

Tested Cameras

The tested cameras can be divided into 13 groups. Each group explores a single misregistration factor or a particular combination of such factors. There are five cameras in each group. The misregistration factor (or factors) of interest gradually changes from the smallest value in the first camera of a group to the largest value in the fifth camera of the same group. Table 1 lists all tested cameras with a brief description of all tested misregistration factors.

4.3.

Test Scenes

Two airborne scenes were chosen as camera input, see Fig. 7. The scenes have many objects of various sizes and contrast. If spatial misregistration is present in a hyperspectral camera, these scene features will cause errors in the acquired spectra.

Fig. 7

Two airborne scenes used as input for the simulations: (a) scene A and (b) scene B. Scene A contains both manmade and natural objects. Scene B contains only natural objects. The images were acquired by HySpex Mjolnir V-1240 mounted in a gimbal on an octocopter. One of the near-infrared channels was used for the simulations. The images are radiometrically calibrated, but not georeferenced.

The first scene (scene A) has both manmade and natural objects. The features in this scene that are expected to cause large errors in the acquired data due to spatial misregistration are:

The second scene (scene B) has natural objects only. The features expected to cause large errors here are mainly small high-contrast objects.

4.4.

Recording the Images

The input scenes are processed by the Virtual Camera software. A single spectral channel in the input scene image is used for all 21 channels in the simulated camera, i.e., all input spectral channels are identical. This approach ensures that we have precise knowledge about all spectra in the scene and keeps the input data clean of any misregistration present in the original hyperspectral data. Keystone and optical blur (PSF) are then added to the input, and the resulting image is recorded into the final datacube.

Note that the image used as input scene contains 7 times more pixels in the across-track direction than the recorded image. This allows to simulate subpixel details (across-track) in the scene that is being depicted.

4.5.

Errors in the Recorded Images

Due to spatial misregistration (keystone and PSF differences), there will be errors in the recorded spectra. We want to estimate the size of these errors.

For a given spatial pixel, the true spectrum is a straight horizontal line, as shown in Fig. 8(a). This is because the input scene is identical in all spectral channels. How high or low this line lies depends on the content of that scene pixel. The corresponding true spectrum in the final data cube (also a horizontal line) may lie at a somewhat different height due to optical blur that causes leakage of the content of neighboring scene pixels into the pixel of interest [Fig. 8(b), red line]. This difference in height does not reflect an actual error in the spectrum. The recorded spectrum is still correct, i.e., the spectral integrity is preserved. However, due to the presence of keystone and PSF variations across spectral channels, the actual value recorded in different channels [red dots in Fig. 8(b)] will deviate from the true value (red line). This deviation represents actual errors in the recorded spectrum.

Fig. 8

Example of (a) true spectrum (blue line) in input scene versus (b) true (red line) and recorded (red dots) spectrum in the final data cube.

The relative error $\mathrm{\Delta }{E}_m^i$ for spatial pixel $\#m$ in spectral channel $\#i$ is given by

We can now calculate the standard deviation of the relative errors $\mathrm{\Delta }{E}_m^{\mathrm{std}}$ for pixel $\#m$

For each camera, we have determined the maximum relative error $\mathrm{\Delta }{E}^{\max }$ among all pixels in the image

4.6.

Simulated Point Source Measurements for Camera Characterization

The two characterization methods 1 and 2 require point source measurements in order to predict misregistration errors for a camera. We have, therefore, simulated point source measurements for all the cameras listed in Table 1.

The point source measurements are performed by moving a (simulated) point source in small steps across the pixel of interest in the across-track direction. In the simulations, we use 21 steps per pixel. Keystone and PSF variations across spectral channels are added to the input before recording into the final data cube. The measurements start and end well outside the pixel of interest in order to capture the whole SPSF. The SPSF will be used as a basis for the misregistration calculations by methods 1 and 2. Note that method 1 requires that the whole SPSF is captured, whereas method 2 only requires the part of the SPSF that is inside the pixel of interest to be captured.

4.7.

Misregistration: Method 1

Method 1 is based on point source measurements and the calculations are performed directly on the resulting SPSFs.⁶ The SPSFs are normalized to 1 in each channel, i.e., for every point source position, the total energy recorded in a given channel is normalized to 1. When referring to recorded energy in the equations below, this is always the underlying assumption.

The misregistration $\mathrm{\Delta }{E}_m^{ij}$ between spectral channels $\#i$ and $\#j$ for spatial pixel $\#m$ (the pixel of interest) is calculated from

The maximum misregistration $\mathrm{\Delta }{E}_m^{\max }$ for pixel $\#m$ can then be calculated from

4.8.

Misregistration: Method 2

Method 2 is based on point source measurements and the calculations can be performed directly on the resulting SPSFs.⁷ The SPSFs are normalized to 1 in each channel, i.e., for every point source position the total energy recorded in a given channel is normalized to 1. When referring to recorded energy in the equations below, this is always the underlying assumption.

The misregistration $\mathrm{\Delta }{E}_{mk}^i$ for spatial pixel $\#m$ (the pixel of interest) in spectral channel $\#i$ when the point source is at position $\#k$ is given by

We can now calculate the standard deviation of the relative errors $\mathrm{\Delta }{E}_{mk}^{\mathrm{std}}$ for pixel $\#m$ when the point source is at position $\#k$

We can then calculate the maximum misregistration $\mathrm{\Delta }{E}_m^{\max }$ for pixel $\#m$ (across all point source positions within the pixel)

5.1.

Maximum Misregistration Errors in Real Scenes

We will first verify that each of the spatial misregistration components (keystone, PSF width, and PSF shape) causes errors in the acquired spectra and check how large these errors are. We will look at the relative errors and, for now, consider the maximum relative error for each camera in a given scene.

Let us consider the five cameras of group 1 in Table 1. These five cameras have keystones of 0.1 to 0.5 pixel, respectively. The PSF widths and shapes correspond to modulation transfer function (MTF) = 0.5 at the Nyquist frequency and are identical in all five cameras and in all spectral channels, i.e., the only error source is keystone. The results of the simulations for these five cameras are presented in Fig. 9. The red curve shows how the magnitude of maximum errors in the acquired spectra depends on the camera’s keystone value. Naturally, a larger keystone causes larger maximum spectral errors.

Fig. 9

Maximum errors in scene A caused by keystone (red curve, five cameras from group 1 in Table 1 with progressively increasing keystone); by differences in PSF width (blue curve, five cameras from group 2 in Table 1 with progressively increasing PSF width differences, see also Table 2 for the PSF width differences in each of the five cameras, expressed as MTF variations at the Nyquist frequency); by differences in PSF shape (green curve, five cameras from group 6 in Table 1 with progressively increasing PSF shape differences).

Let us consider a different example with five cameras that have no keystone, and where all five cameras have identical PSF shapes in all spectral channels. However, the PSF width in each camera changes across all spectral channels (group 2 in Table 1). Table 2 shows how much the MTF at Nyquist frequency changes across the spectral range for each camera. The blue curve in Fig. 9 shows how these differences in MTF (i.e., in PSF width) correspond to the maximum errors in the acquired spectra.

Table 2

PSF width changes across the spectral range for five cameras in group 2 (Table 1). The PSF width values are expressed as MTF values. A larger MTF value corresponds to a narrower PSF.

In the third example, the five tested cameras have no keystone and a constant PSF width corresponding to $\mathrm{MTF}=0.5$ at the Nyquist frequency. However, for each camera, the PSF shape varies across spectral channels (group 6 in Table 1). In camera 30, the variations are the largest: Figs. 5(g)–5(h) show the two SPSFs with the largest difference in shape for this camera. PSFs for all other spectral channels are derived by morphing between these two PSFs [see Figs. 6(a)–6(f) for an example of such morphing]. The PSF shape variations in the other four cameras are smaller, as described in Sec. 4.1. The results of this simulation are presented in Fig. 9, where the green curve shows how the magnitude of the maximum errors in the acquired spectra depends on the differences in the PSF shape.

Based on the results above, it seems clear that misregistration is potentially an important source of error in hyperspectral data that may easily exceed the contribution of other error sources such as photon noise.

5.2.

Maximum Misregistration Errors in Real Scenes versus Camera Characterization of Misregistration

We will now compare maximum misregistration errors in real scenes to camera characterization by methods 1 and 2. Spectral errors in real scenes—caused by misregistration factors of different types and magnitude—were investigated for 59 cameras. The same 59 cameras were also characterized by methods 1 and 2. We can now evaluate how well each characterization method predicts the errors caused by misregistration when acquiring spectra of real scenes.

Note that the 59 simulated cameras have misregistration errors that are different not only in magnitude but also in nature (see Fig. 9 for 15 camera examples). In a nutshell, we need to find a way to evaluate two lab characterization methods by comparing their results with 13 datasets (each containing five cameras), where each dataset has errors of different magnitudes and the nature of the errors is different between different datasets. This does not appear to be an easy task. How can we do this comparison without resorting to multidimensional graphs?

First, we place points on the graph with each point corresponding to the maximum relative error in the spectral data captured from a real scene for a given camera. There are 59 cameras, so there are 59 points. The cameras are ordered according to the magnitude of the maximum error: the camera with the smallest maximum error is the first from the left, the camera with the largest maximum error is farthest to the right, with all other cameras in between—ordered according to the magnitude of their maximum errors. Now we can place misregistration for each camera as calculated by method 1 in the same camera order that was used for the errors in the real scene, and the same can be done for method 2. Figure 10 shows the result for scene A, where any given index along the horizontal axis shows (for the same camera):

Fig. 10

Maximum errors in scene A. The cameras are ordered according to the magnitude of their maximum error in scene A (index #1 represents the camera with the smallest maximum error, whereas index #59 represents the camera with the largest maximum error). The red curve shows the maximum error for each of the 59 tested cameras when depicting scene A. The blue curve shows the maximum error of the same 59 cameras as predicted by method 1. The green curve shows the maximum error of the same 59 cameras as predicted by method 2.

In order to reliably predict the relative performance between different cameras, the misregistration curve calculated by a given characterization method should follow the red curve, i.e., each given point of the method curve should be higher than the adjacent point to the left and lower than the adjacent point to the right. If this is the case, the method is good at predicting a camera’s performance on real data relative to other cameras. We see that, although there are noticeable deviations from this criterion, method 1 and method 2 follow the shape of the red curve reasonably well. Method 2 has the advantage of predicting the magnitude of the maximum error in the real data more accurately.

Let us check the two characterization methods on scene B (Fig. 11). We see that the conclusions are still valid: both methods follow the red curve reasonably well with method 2 better predicting the magnitude of the maximum errors for each camera. Note, however, that for scene B, method 1 creates a smoother curve that better follows the shape of the red curve.

Fig. 11

Maximum errors in scene B. The cameras are ordered according to the magnitude of their maximum error in scene B (index #1 represents the camera with the smallest maximum error, while index #59 represents the camera with the largest maximum error). The red curve shows the maximum error for each of the 59 tested cameras when depicting scene B. The blue curve shows the maximum error of the same 59 cameras as predicted by method 1. The green curve shows the maximum error of the same 59 cameras as predicted by method 2.

At first glance, it might seem that looking at the maximum error is not a reliable approach, because such an error would occur in a single specific location for a given scene and this error could have more or less random value. However, this turns out not to be the case. We have looked into the location and magnitude of the largest errors in the two scenes. Regarding the location of maximum errors, we find that the maximum errors occur in different locations of a given scene—depending on the nature of the camera’s spatial misregistration. Regarding the actual values of the largest errors, we find that the five largest errors (for a given camera) occur in different locations of a scene for different cameras. Further, for a given camera, we find that the five largest errors are very similar to each other in magnitude. A typical difference between the largest and the fifth largest error (for a given camera and scene) was found to be less than 3% of the error value, whereas the maximum observed difference was just above 11% of the error value.

Note that the two methods look at different scene features as sources of the maximum errors. Method 1 considers a sharp border between two different materials to give the largest error (Sec. 3.2), whereas method 2 considers a small (smaller than a pixel) bright object to give the maximum error (Sec. 3.3). In the two real scenes analyzed in this work, both the borders and the small (subpixel sized) objects were found to cause the largest errors—depending on the nature of spatial misregistration in the camera.

The cameras that we have tested so far have one thing in common: their optics is reasonably sharp with MTF at the Nyquist frequency of 0.25 in the worst case (0.5 in many other cases). This is typical for optics that is designed to retrieve as much information as possible from the scene when using a given sensor.¹² However, there are cameras where the resolution of the optics is significantly lower than the resolution of the sensor. We will, therefore, test one more set of five cameras where the PSF is so large that the MTF reaches 0 approximately at the Nyquist frequency [or, more precisely, at 0.95 of the Nyquist frequency, see Fig. 5(d) with the corresponding PSF]. The PSF size and shape will be identical for all cameras in all channels, whereas the keystone will be 10%, 20%, 30%, 40%, and 50% (of a pixel) in each camera, respectively. Let us now add errors from these five cameras to the graph (Fig. 12). This brings the total number of tested cameras to 64.

Fig. 12

Maximum errors: (a) scene A and (b) scene B. The cameras are ordered according to the magnitude of their maximum error in scene A and scene B, respectively (index #1 represents the camera with the smallest maximum error, whereas index #64 represents the camera with the largest maximum error). The red curve shows the maximum error for each of the 64 tested cameras when depicting the corresponding scene. The blue curve shows the maximum error of the same 64 cameras as predicted by method 1. The green curve shows the maximum error of the same 64 cameras as predicted by method 2.

Method 1 (blue curve) still follows the shape of the red curve (real scene) reasonably well, but this is no longer the case for method 2. Although method 2 is better at predicting the maximum errors for sharper cameras, it clearly underestimates the misregistration errors in the five blurry cameras [indices 5, 14, 26, 34, and 44 in Fig. 12(a); indices 5, 16, 24, 37, and 44 in Fig. 12(b)]. Method 1, therefore, seems to more reliably predict the relative performance between different cameras when more blurry cameras are also considered. In other words, if a camera has smaller maximum misregistration according to method 1, then this camera is likely to also have smaller maximum misregistration errors in an image of a real scene. However, method 1 consistently underestimates the magnitude of the maximum errors. In the next section, we will show how the two methods can be combined in order to optimize the ability to predict both the magnitude of the maximum misregistration errors and the camera performance relative to other cameras.

5.3.

Combining the Two Methods for more Accurate Camera Characterization

It is possible to improve the lab characterization of spatial misregistration by combining the two characterization methods. Note that even though method 2 may require fewer measurements for camera characterization, both methods can easily use the same measurement set (see Sec. 3 and Refs. 6 and 7). Below we will look at three possible approaches.

5.3.1.

Approach 1

Approach 1 combines the more accurate prediction of maximum errors for sharper cameras by method 2 with the more accurate prediction of maximum errors for blurry cameras by method 1. For a given camera, the misregistration values calculated by each method are compared and the largest value of the two is considered to be that camera’s misregistration. Figure 13 shows misregistration errors calculated by method 1 (blue) and approach 1 (green) compared to errors in a real scene (red). We see that the errors of blurry cameras are still underestimated, but not as much as when method 2 is used alone.

Fig. 13

Maximum errors: (a) scene A and (b) scene B. The cameras are ordered according to the magnitude of their maximum error in scene A and scene B, respectively (index #1 represents the camera with the smallest maximum error, whereas index #64 represents the camera with the largest maximum error). The red curve shows the maximum error for each of the 64 tested cameras when depicting the corresponding scene. The blue curve shows the maximum error of the same 64 cameras as predicted by method 1. The green curve shows the maximum error of the same 64 cameras as predicted by approach 1.

5.3.2.

Approach 2

We have previously discussed how method 1 seems to consistently underestimate the maximum misregistration errors. However, if the misregistration is always underestimated by the same percentage, then the misregistration prediction by method 1 could be improved by multiplying the calculated misregistration value by a given constant factor. For scene A, a factor of 1.25 seems to work well, see Fig. 14(a). It is, however, a necessary requirement that the same factor can be applied also to other scenes. Figure 14(b) shows the results when a factor 1.25 is used for scene B. Although this is a significantly different scene (as discussed in Sec. 4.3), we see that the factor 1.25 works well here, too. Results for approach 1 are also shown for comparison. Approach 2 (blue) seems to work better than approach 1 (green).

Fig. 14

Maximum errors: (a) scene A and (b) scene B. The cameras are ordered according to the magnitude of their maximum error in scene A and scene B, respectively (index #1 represents the camera with the smallest maximum error, whereas index #64 represents the camera with the largest maximum error). The red curve shows the maximum error for each of the 64 tested cameras when depicting the corresponding scene. The green curve shows the maximum error of the same 64 cameras as predicted by approach 1. The blue curve shows the maximum error of the same 64 cameras as predicted by approach 2.

As previously noted, method 1 fits data well with respect to comparing cameras, but not for the actual size of observed maximum errors. If a camera user would like to know the approximate magnitude of the expected maximum errors, it may be useful to know that this number can be found by multiplying the result by the factor 1.25.

5.3.3.

Approach 3

In approach 3, we compare the misregistration values calculated by method 2 and approach 2, and choose the largest value of the two to be the measured misregistration. Figure 15 shows the results. Results for approach 2 (light blue) are also included in the figure for comparison. Approach 3 (black) seems to work well for all tested cameras on both scenes and gives the best results of all three approaches. The magnitude of the maximum errors is now predicted reasonably well, and the misregistration curve also follows the shape of the red curve (errors in the scene) quite well.

Fig. 15

Maximum errors: (a) scene A and (b) scene B. The cameras are ordered according to the magnitude of their maximum error in scene A and scene B, respectively (index #1 represents the camera with the smallest maximum error, whereas index #64 represents the camera with the largest maximum error). The red curve shows the maximum error for each of the 64 tested cameras when depicting the corresponding scene. The light blue curve shows the maximum error of the same 64 cameras as predicted by approach 2. The black curve shows the maximum error of the same 64 cameras as predicted by approach 3.

5.4.

Mean Misregistration Errors in Real Scenes

So far, we have been discussing maximum errors—the errors that occur when a very high-contrast feature (a border or a small subpixel sized object) is placed in the worst position relative to the pixel center. When looking at the maximum misregistration error, we consider just two spectral channels (the two channels with the largest misregistration error between them). However, all other channels will have smaller errors. Therefore, in addition to knowing the maximum misregistration, it may be useful to know the mean misregistration when errors in all spectral channels and for various positions of high-contrast scene features are considered.

We will first check the magnitude of the errors in the acquired spectra, which are caused by each of the spatial misregistration components (keystone, PSF width, and PSF shape). Just as we did in Sec. 5.1 for the maximum errors, we will use the same 15 cameras and look at the relative errors. However, this time we will consider the mean relative error for each camera in a given scene. Figure 16 shows the results for scene A.

Fig. 16

Mean errors in scene A caused by keystone (red curve, five cameras from group 1 in Table 1 with progressively increasing keystone); by differences in PSF width (blue curve, five cameras from group 2 in Table 1 with progressively increasing PSF width differences, see also Table 2 for the PSF width differences in each of the five cameras, expressed as MTF variations at the Nyquist frequency); by differences in PSF shape (green curve, five cameras from group 6 in Table 1 with progressively increasing PSF shape differences).

We can clearly see that all three misregistration components affect the mean misregistration error in the spectra when acquiring an image of a real scene. The mean errors are significantly smaller than the maximum errors (Fig. 9). This happens for several reasons. First, not all spectral channels have equally large misregistration factors (keystone and PSF differences). Second, many scene features have low contrast, and therefore, cause relatively small errors. Finally, various scene features are positioned randomly relative to the pixel grid and the resulting misregistration errors are smaller for some of the positions than for others.

5.5.

Mean Misregistration Errors in Real Scenes versus Camera Characterization of Misregistration

We will now compare the mean misregistration predicted by methods 1 and 2 with the mean misregistration errors in the two scenes. This will be done for 64 different cameras—similar to what we did in Sec. 5.2 for maximum misregistration. Figure 17 shows the results for both scenes. The red curve shows the mean misregistration errors in the scene, whereas the blue and green curves show the mean misregistration measured by methods 1 and 2, respectively. The data are arranged in the same way as before: the cameras are ordered according to the magnitude of the mean errors, where the camera with the smallest mean error is the first from the left, and the camera with the largest mean error is the farthest to the right. We see that the red curve is much lower than the blue and green curves for the two characterization methods—for both scenes. This makes sense since the scenes have both high- and low-contrast features, whereas the two methods only consider high-contrast features. What is important here is how well the curves for the two characterization methods follow the shape of the red curve for the errors in the scene, i.e., for a good characterization method, a larger measured mean misregistration should correspond to a larger mean error in the real scene.

Fig. 17

Mean errors: (a) scene A and (b) scene B. The cameras are ordered according to the magnitude of their mean error in scene A and scene B, respectively (index #1 represents the camera with the smallest mean error, whereas index #64 represents the camera with the largest mean error). The red curve shows the mean error for each of the 64 tested cameras when depicting the corresponding scene. The blue curve shows the mean error of the same 64 cameras as predicted by method 1. The green curve shows the mean error of the same 64 cameras as predicted by method 2.

We see that the curve for method 1 (blue) follows the shape of the red curve quite well for both scenes, whereas method 2 (green)—as before—underestimates the misregistration of blurry cameras (indices 6, 18, 28, 38, and 44 for scene A; indices 6, 16, 27, 37, and 44 for scene B) compared to sharper cameras. Method 2 is, therefore, less suitable for evaluation of sharp and blurry cameras simultaneously. For sharper cameras, method 2 works reasonably well, but method 1 seems to give the best prediction of camera performance relative to other cameras—with respect to mean misregistration errors.

5.6.

Spatial Misregistration in the Along-Track Direction

The simulations in this paper were done in the across-track direction where the influence of the slit can be considered negligible.¹² In the along-track direction, a sufficiently narrow slit (similar to the Airy disk size and smaller^13–16) will contribute to the PSF shape. Although this contribution is naturally taken into account during lab characterization, incorporating this contribution into simulations may be more challenging because the optical design software may not be capable of reproducing it. Nevertheless, since the same spatial misregistration factors are present also in the along-track direction, we expect the conclusions based on our simulations in the across-track direction to be valid in the along-track direction as well.

The majority of pushbroom hyperspectral cameras use a continuous scanning motion in the along-track direction during image acquisition. This motion blurs the SPSF in the along-track direction, and therefore, affects the errors caused by spatial misregistration. In principle, this motion should be taken into account when quantifying the misregistration.¹⁴ However, in the majority of cases, it may be more practical if the manufacturer specifies spatial misregistration without the scanning motion, since it is the user—i.e., not the manufacturer—who determines how the camera is used. The scanning movement per frame may be as large as a couple of across-track pixels or as small as zero (when the camera is moved along-track between exposures, but stays still during the image acquisition). In any case, the misregistration specification provided for a static camera will be useful for all scanning scenarios, because a camera that is the best in terms of spatial misregistration when there is no scanning motion, will still be the best camera if scanning motion is present.