Improvement of a mineral discrimination method using multispectral image and surrounding hyperspectral image

. Hyperspectral (HS) images are highly accurate for mineral discrimination, but available areas are limited. For this reason, several methods have been proposed to extend the mineral map of the overlap region between HS and multispectral (MS) images to the surrounding area with no HS image. One such method, proposed by Hirai and Tonooka, discriminates minerals using MS images by obtaining the endmembers of MS images from the positions of the end-member pixels of HS images in the overlap region. While this method (referred to as HT method) has the advantage of being less susceptible to the spectral distortions of HS and MS images, it also has the problem of reduced accuracy due to misalignment between HS and MS images. We proposed an improved HT method that reduces the effects of the above problems by incorporating a process that improves the robustness against misalignment by searching for the best MS endmember pixel around the position of the HS endmember pixel and a process that determines more optimum threshold value of each mineral in the spectral angle mapper method used in the HT method. As a result of evaluation using an AVIRIS image as an HS image and a World View-3 image as an MS image at Cuprite, Nevada, the improved method improved the overall accuracy by 2.6% compared with the original HT method, and in the case that the HS and the MS images were misaligned, the overall accuracy of the original method decreased by 7.0%, while the improved method decreased by only 1.5%. These results indicate that the improved method can perform as expected. © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 International License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI. [DOI: 10.1117/1.JRS.15.040501]


Introduction
Although multispectral (MS) images are useful for mineral discrimination by remote sensing, some minerals may be difficult to discriminate due to limitations caused by the small number of bands and wide band widths. 1 For example, Vural et al. succeeded in detecting the distribution areas of clay minerals in northeastern Turkey using MS images of Landsat 7 and 8, but detailed clay mineral classification has not been achieved. 24][5][6] However, as a trade-off for increasing the number of wavelengths, HS sensors have a narrower swath width, such as 7.6 km for Hyperion on EO-1, 7 30 km for Environmental Mapping and Analysis Program (EnMAP) 8 and PRecursore IperSpettrale della Missione Applicativa (PRISMA), 9 and 20 km for Hyperspectral Imager Suite (HISUI) 10 on the International Space Station.In addition, since there are few HS sensors that have been operated so far, archived HS images are limited compared to MS images, and there are many unobserved regions in the world.As a countermeasure to these problems, Kruse and Perry developed a method to extend mineral maps in an overlapped area of HS and MS to the surrounding area with an MS image and no HS image. 11,12he effectiveness of this method has been verified in the Cuprite region of Nevada, using images in the short-wave infrared (SWIR) band from the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) and the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER).On the other hand, because Kruse and Perry's method (referred to as KP method) can degrade due to the influence of spectral distortions caused by calibration errors, atmospheric correction errors etc., Hirai and Tonooka proposed an alternate method that classifies minerals from the MS image using MS-based endmember pixels extracted based on the locations of endmember pixels in the HS image.13 As a result of evaluation using AVIRIS and ASTER/SWIR images in the Cuprite region, Hirai and Tonooka's method (referred to as HT method) showed higher accuracy than KP method for data with spectral distortions, but the registration error between HS and MS images caused a decrease in accuracy.Under such background, in this study, we propose an improved HT method that improves the robustness against registration errors and also optimizes the threshold of each mineral used in the spectral angle mapper (SAM) method 14 included as a part of the HT method.

KP and HT Methods
Both the KP and HT methods use the analysis results of the overlapped area of MS and HS images to discriminate minerals in the neighboring area where only MS images are available.The KP method extracts the endmember spectrum of each mineral from the HS image in the overlapped area and multiplies it by the spectral response function of the MS sensor to obtain the endmember spectrum of each mineral in the MS image, while the HT method extracts the endmember pixels of each mineral from the MS image based on the locations of the endmember pixels derived from the HS image in the overlapped area.For spectral mismatch between MS and HS images due to calibration error, atmospheric correction error, etc., the latter is not affected while the former is affected and degraded. 13he following is a schematic procedure of the HT method.More details can be found in Ref. 13.
[Step 1] Atmospheric correction and surface reflectance transformation are applied to the HS and MS images to be used.Then, the alignment between the two images is strictly performed.[Step 2] The endmember pixels are extracted from the HS image through the following process: (1) the extraction of signal information and the reduction of spectral data volume by the minimum noise fraction transformation, 15 (2) the extraction of spectrally pure pixels by the pixel purity index analysis, 16 and (3) the extraction of endmember pixels by n-dimensional visualization. 16Step 3] The pixels of the MS image at the same position as the endmember pixels obtained from the HS image are selected as the MS-based endmember pixels.[Step 4] The SAM method is applied to the MS image, and the cosine similarity of each mineral is calculated for each pixel by where P is the spectral vector of the pixel, E is the mean spectral vector of the endmembers of the mineral, respectively, and θ is the angle between these vectors.If it is greater than or equal to the threshold value set for each mineral, the mineral is selected as a candidate mineral, where the threshold value of the cosine similarity of the mineral k is set so the distribution of the mineral k by this processing is consistent with the distribution of the mineral k obtained from the HS image in the overlapped area. 13Then, for each pixel, the mineral with the maximum cosine similarity among the candidate minerals is basically selected as the mineral of the pixel, where the mineral with the second or later cosine similarity among the candidate minerals may be selected if the consistency with the HS-based mineral map can be improved. 13Improvement of the HT Method The HT method assumes that the HS and MS images are accurately aligned, so if there is any misalignment, an error will occur in the selection of endmember pixels in Step 3.This is not a major problem if the geological unit has a relatively large area, because the same minerals are likely to be selected even if there is some misalignment.However, if the distribution range of the target mineral is narrow or if multiple minerals are distributed in a mixed manner, minerals different from those in the HS image may be erroneously selected as endmembers in the MS image.In addition, in setting the threshold of the SAM method for each mineral in Step 4, the method by which the analyst can match the distribution of the relevant mineral in the HS image with that in the MS image depends on the experience of the analyst.As countermeasures to these issues, we propose to apply the following two improvements to the HT method: (1) the MS-based endmember pixels are selected not from only the position of each endmember pixel in the HS image but also from the neighboring region of that based on the overall accuracy which can be calculated from provisional mapping results by the SAM method (see steps 3 and 4 below), and (2) the optimum threshold of the SAM method is selected not manually but by automatic search based on the overall accuracy (see steps 5 and 6 below).The following is the flow of the improved HT method, where steps 1 and 2 are the same as the original HT method (see the previous section), and steps 3 and after are updated. [ Step 3] For each mineral obtained in step 2, the pixel in the MS image located at the same position as each endmember pixel in the HS image is selected as an MS-based endmember candidate.[Step 4] For each mineral, the individual map of only the mineral is temporally generated from the MS image for each MS-based endmember candidate and its neighboring pixels (e.g., 5 × 5, 7 × 7 pixels) (referred to as a candidate pixel group) using the SAM method, and the overall accuracy is calculated by comparing with the individual mineral map of the mineral which is extracted from the HS-based mineral map generated in step 2. As many as the number of the HS-based endmember pixels of the mineral found in step 2 is selected from the pixels with higher accuracy in all the candidate pixel groups of the mineral, and the average value of the spectra of the selected ones is used as the MS-based endmember spectrum of the mineral.[Step 5] For each mineral, the threshold used in the SAM method is determined so the overall accuracy of the individual map of the mineral is maximized, by the following procedure: (1) For all pixels in the MS image, the cosine similarity to the MS-based endmember spectrum of the mineral is calculated.(2) While lowering the cosine similarity threshold from the maximum value, the overall accuracy against the HS-based individual map of the mineral is calculated for the MS-based individual map of the mineral derived from each threshold.(3) The threshold that shows the maximum overall accuracy is adopted as the threshold of the mineral.That is, the threshold T k of mineral k is determined by E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 2 ; 1 1 6 ; 2 8 0 E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 3 ; 1 1 6 ; 2 3 6 where a and b are the min and the max of the range that a threshold t can take, OA k ðtÞ is the overall accuracy for mineral k under a threshold t; N is the total number of MS pixels, and p k ðtÞ and q k ðtÞ are the numbers of the MS pixels correctly classified as mineral k and as not mineral k, respectively, under a threshold of t. [ Step 6] For each pixel in the MS image, a mineral with a cosine similarity greater than the threshold of that mineral is assigned.Since the thresholds are determined on a mineralby-mineral basis in step 5, two or more minerals may exceed each threshold.In this case, the mineral with the larger cosine similarity is selected for the pixel.

Test Site and Data Used
To evaluate the improved HT method, the Cuprite region in Nevada, was selected as a test site, and the AVIRIS image (wavelength range: 1967 to 2496 nm, spatial resolution: 15.7 m) observed on September 20, 2006, was used as the HS image, and the World View-3 (WV-3) ortho-ready image (spectral region: shortwave infrared, spatial resolution: 7.5 m) observed on September 19, 2014, was used as the MS image.Each image was then converted to surface reflectance by atmospheric correction using the ENVI FLAASH software.The WV-3 image was also registered to match the spatial resolution of the AVIRIS image.

HS-Based Mineral Map
Figure 1 shows the HS-based mineral map obtained from the AVIRIS surface reflectance image by steps 1 and 2, mapping five minerals, kaolinite, calcite, alunite, muscovite, and buddingtonite, where the number of endmember pixels for each mineral is 7, 1, 5, 1, and 3, respectively.This image is used as the true mineral map for evaluating the MS-based mineral maps obtained by the original and improved HT methods.

Evaluation of the MS-Based Mineral Maps
The MS-based mineral maps obtained by the original and improved HT methods can be evaluated by examining the validity of the MS-based endmember obtained, i.e., the consistency in the overlapping area between the mineral map generated from the MS-based endmember by these methods and the HS-based mineral map.Here, since the improved method was developed for increase of robustness against misalignment, the methods were evaluated in two cases: (case A) the case where the HS and MS images are accurately registered, and (case B) the case where they are misaligned.In case B, the misalignment was given by moving the MS image one pixel to the right and two pixels to the up.The upper two images of Fig. 2 were obtained in case A (accurate registration) and the lower two images were obtained in case B (misalignment), where the mineral maps by the original   method are on the left and those the improved method are on the right.In the improved method, the candidate pixel group was given as 7 × 7 pixels centering each HS-based endmember pixel.
As shown in the figure, there is a large difference in the distribution of muscovite and kaolinite between cases A and B in the original method, while there is not much difference between cases A and B in the improved method.Table 1 also shows the producer's and user's accuracies for each mineral and the overall accuracy for each method in cases A and B. In case A, except for the producer's accuracy of kaolinite and calcite, the accuracy is improved by the improved method, and the overall accuracy is improved by 2.6%.On the other hand, in case B with misalignment, the accuracy of the original method was 7.0% lower than that of case A, but the accuracy of the improved method was only 1.5% lower, and the overall accuracy of the improved method was 8.1% higher than that of the original method.These results show that the improved method can discriminate minerals with higher accuracy than the original method and is robust to HS-MS image misalignment which is a major error factor in the original method.

Conclusions
Although the original HT method is robust against spectral distortions, there are two issues: (1) the accuracy decreases due to registration errors between HS and MS images, and (2) the threshold setting for each mineral in the SAM method depends on the analyst.In this study, we proposed an improved HT method to mitigate these effects.As a result of evaluation using AVIRIS and World View-3/SWIR images from the Cuprite region, Nevada, it was confirmed that the improved method has higher accuracy than the original method, and in particular, the accuracy decrease is suppressed even when there is misalignment between HS and MS images, concluding that the improved HT method can discriminate minerals more robustly than the original HT method.

Fig. 2
Fig. 2 Mineral maps obtained by the original and the improved HT methods in cases A and B (blue: kaolinite; green: calcite; red: alunite; magenta: muscovite; yellow: buddingtonite).

Table 1
Producer's and user's accuracy (%) in cases A and B for the original and the improved HT methods (larger values are shown in bold).Nakayama and Tonooka: Improvement of a mineral discrimination method using multispectral image. . .