2.2.1.

Design and characteristics of developed hyperspectral camera

Figure 3 shows the development of the CubeSat LVBPF-based hyperspectral camera. The camera was small ($\sim 3\mathrm{cm}$ in length including the lens and image sensor). Two types of cameras to be installed on the CubeSats were developed in this study (Table 1). Both cameras had the same appearance but different filters and image sensors. However, because these cameras were intended to be installed as a sub-mission of a multi-mission CubeSat, they used a small lens with coarse resolution. Table 2 shows the specifications of each filter. The type A LVBPF was an initial experiment; therefore, we used a readily available existing filter with a long and thin shape. By contrast, the type B LVBPF is a filter cut to a size that matches the image sensor. The type A hyperspectral camera has a smaller pixel size in the image sensor, resulting in finer spatial and spectral resolutions compared with the type B camera. However, the type B camera has an advantage in its wavelength range as it covers both the visible and near-infrared spectra, whereas the type A camera is limited to the visible range only. The structures and detailed views of each hyperspectral camera are shown in Fig. 4. This figure shows the structure of the type B camera; the type A camera is the same except for the filter size. The filter is not deposited onto the image sensor but is fixed mechanically. However, in the case of mechanical fixation, the distance between the filter and image sensor becomes wider, which increases the effects of spectral cross-talk, causing light leakage of adjacent wavelengths and mixing of the wavelengths. Commercially available image sensors have a cover glass attached; therefore, the cover glass was removed, and the structure was designed for the filter to be as close as possible to the image sensor. Considering the spectral resolution in this configuration to be the width of the light beam passing through the filter, the spectral resolution is calculated as the product of the $F$-number, dispersion, and distance of the filter and focal plane. The $F$-number of the telescope lens is $F/2.5$, and the distance of the filter and focal plane is 0.67 mm; these are the same for both cameras. In the case of the type A camera, the dispersion of LVBPF is 42.1 nm/mm; thus, the spectral resolution is $\sim 11.3\mathrm{nm}$. In the case of type B, if the dispersion is 67.7 nm/mm, the spectral resolution is calculated as $\sim 18.1\mathrm{nm}$. However, this is merely a simplified calculation in which the half-band width of the filter itself is ignored. The measurement results of the spectral resolution of both cameras are shown in Sec. 2.2.2.

Fig. 3

Overview of the developed hyperspectral camera.

Table 1

Specifications of the developed hyperspectral cameras.

Table 2

Specifications of the LVBPF (condition: incident angle = 0 deg).

Fig. 4

Structure of hyperspectral type B camera.

The LVBPF-based hyperspectral camera offers flexibility as its spectral resolution and wavelength range can be easily modified by simply changing the filter, image sensor, and telescope lens. To avoid relying on the attitude control accuracy of the CubeSat for the demonstration experiment, the ground sampling distance was coarsened to provide a wide field of view, although the original ground sampling distance of both cameras was already coarser than 100 m.

2.2.2.

Pre-flight performances

The performance of the developed hyperspectral camera types A and B was evaluated as a ground test. The imaging performance of a hyperspectral camera is expected to depend on the wavelength, and the point-spread function (PSF) of each camera engineering model was measured with each wavelength. The PSF of each camera was measured using a collimator with a pinhole chart and band-pass filters of 400, 550, 600, 650, and 700 nm. Figures 5 and 6 show the PSFs of each camera with each wavelength. Based on the results obtained, each plot was approximated with a normal distribution function to derive the standard deviation ($\sigma$). As a result, the PSFs of the type A camera had an ($\sigma$) of 1.8 pixels at 450 nm, 1.7 pixels at 550 nm, and 1.6 pixels at 650 nm, whereas the type B camera had a PSF with an ($\sigma$) of 1.8 pixels at 450 nm, 1.7 pixels at 550 nm, 1.7 pixels at 650 nm, and 2.1 pixels at 700 nm. The result shows that the imaging performance in the visible range has no wavelength dependency. By contrast, there is some wavelength dependency in the near-infrared range of the type B. However, the difference is only $\sim 0.4\mathrm{pixels}$, which is sufficient for imaging performance across the entire wavelength range. The wavelength dependency of imaging performance is due to the optical design, and it is considered that there is no difference between the engineering model and the flight model; therefore, the PSFs of the flight model were measured only for green wavelengths (550 nm).

Fig. 5

PSF of hyperspectral type A camera for each wavelength. (a) 450 nm, (b) 550 nm, and (c) 650 nm.

Fig. 6

PSF of hyperspectral type B camera for each wavelength. (a) 450 nm, (b) 550 nm, (c) 650 nm, and (d) 700 nm.

The PSFs of each camera flight model were measured with ground tests using a collimator with a pinhole chart and green light. Both cameras were subjected to environmental tests, including random vibration, thermal cycling, and thermal vacuum tests. Figure 7 shows the PSF measured before and after the environment test. Thermal cycling and thermal vacuum tests were performed under $-10°\mathrm{C}$ to $+50°\mathrm{C}$. A random vibration test was performed with the camera mounted on each satellite under the conditions of the referenced launching vehicle. Before environmental tests, the PSFs of the type A camera had a standard deviation ($\sigma$) of 1.4 pixels, whereas the type B camera had a PSF with an ($\sigma$) of 1.4 pixels. By contrast, after the environmental tests, the results show that the type A camera had a PSF with an ($\sigma$) of 1.7 pixels and the type B had a PSF of 1.6 pixels. The difference in PSF before and after the environmental tests was 0.1 to 0.2 pixels, with both cameras achieving sufficient imaging quality based on the results of the environmental tests.

Fig. 7

PSF of hyperspectral type A and type B camera flight models.

Figure 8 shows the results of evaluating full width at half maximum (FWHM) as the spectral resolution of 400, 550, 600, 650, and 700 nm (type A did not have 700 nm).

Fig. 8

Spectral resolution of hyperspectral type A and type B camera flight models.

The horizontal axis is the pixel position of the image sensor, which is converted to a unit of wavelength using the result of spectral calibration. The conversion factor is 0.09 [nm/pixel] for type A and 0.336 (nm/pixel) for type B. The evaluation test was conducted by attaching a band-pass filter with an FWHM of 10 nm to an integrating sphere and using a halogen lamp. In the type A camera, the spectral resolution was $\sim 10.4\mathrm{nm}$ at 450 nm, 12.1 nm at 500 nm, and 20.7 nm at 600 nm. In the case of the type B camera, the spectral resolution was $\sim 19.8\mathrm{nm}$ at 450 nm, 23.7 nm at 500 nm, 36.4 nm at 600 nm, and 47.5 nm at 700 nm. As shown in Table 1, the design values of the spectral resolution are 14.1 nm for type A camera and 18.5 nm for the type B camera. At short wavelengths, the measured and calculated results are in agreement, but as the wavelength increases, the spectral resolution is reduced. This could be because the half-band width of the LVBPF has wavelength dependence as the filter specification shown in Table 2, and the half-width on the long wavelength side is wide. Table 2 lists the half-band width of the Type B filter; it is assumed that the half-band width of the type A filter is similar. As the half-band width increases, the angle of incidence that can be transmitted increases. Furthermore, the PSF is rougher on the near-infrared wavelength side. These combined causes could make the spectral resolution on the long wavelength side of the image sensor become coarse. For these reasons, the influence of spectral cross-talk between neighboring pixels, especially on the long wavelength side, becomes large, and the spectral purity becomes dilute so that the effective number of bands becomes coarser.

To ensure the success of the CubeSat demonstration experiment in this research, the camera adopted an $F/2.5$ fast lens to ensure that it captured the image of the Earth’s surface. As a result, the width of the light beam that passes through the filter becomes larger, and the spectral resolution is coarser. The spectral resolution of LVBPF-based hyperspectral cameras was found to be strongly dependent on the F-number of the lens. This should be considered in future research to find an optimal design solution.

2.2.3.

Demonstration using a drone

To evaluate the developed type B LVBPF-based hyperspectral camera, an aerial photography experiment was performed using a drone. The drone model used in this experiment is the DJI Phantom 4. The purpose of this experiment was to confirm that images and spectral information can be effectively acquired using the LVBPF-based hyperspectral camera. The experiment was performed at a soybean farm, where spectral information on soy, weeds, and soil was acquired. The specifications of the type B camera are shown in Table 1. The drone was flown at an altitude of 15 m, and the images were taken with a ground sampling distance of $\sim 1\mathrm{cm}/\mathrm{pixel}$. The ground surface is scanned in the direction of the drone’s movement, and the overlapping parts of the images are synthesized to generate a spectral image, which then forms hyperspectral data. However, due to the vibrations that occur when taking photos with a drone, the method of combining images at equal time intervals did not produce a successful combination. Therefore, overlap coordinates are estimated using image feature point matching and homography transformation, and images are synthesized to generate hyperspectral data. In the CubeSat imaging experiments described in Sec. 3, the satellite’s attitude was not stable, so a similar synthesis method was used to generate hyperspectral data.

The results of the RGB composite color image and the spectral information extracted from the acquired hyperspectral data are shown in Fig. 9. The results of soy and weed classification from the acquired hyperspectral data using the Spectral Angle Mapper method are summarized in Fig. 10.³⁰ As shown in Figs. 10(a) and 10(b), the results confirm that weeds can be classified by comparing the classification outcomes obtained using hyperspectral data and visual inspection. Next, the classification accuracy was calculated. The weeds were scattered in small areas, and it was not possible to create an accurate annotated mask; only the classification accuracy of soybeans was calculated. The classification accuracy for soybeans was obtained at 76.8%. This classification accuracy is calculated by comparing the annotated mask shown in Fig. 10(c) with the classification result in Fig. 10(a) and dividing the number of classified correct pixels by the total number of pixels of the annotated mask. The annotated mask was created manually using the high-resolution aerial photograph image with a ground sampling distance of $\sim 0.2\mathrm{cm}/\mathrm{pixel}$ shown in Fig. 10(d). These results reveal that the developed LVBPF-based hyperspectral camera has sufficient performance to acquire spectral information for agricultural remote sensing applications.

Fig. 9

Composite image consisting of 474, 564, and 614 nm and respective spectral information (2022-08-08 11:27:23 JST, captured by Drone).

Fig. 10

Classification results of the soybean farm. (a) Classification results for soy and weed, (b) comparison of the classification result of weed with a visual inspection, (c) manually annotated mask for the region of soy, (d) aerial photograph image of a high-resolution color camera.