Imager and sounder data fusion to generate sounder retrieval products at an improved spatial and temporal resolution

Abstract. The Visible Infrared Imaging Radiometer Suite (VIIRS) imaging instrument on the polar-orbiting Suomi-National Polar-orbiting Partnership and National Oceanic and Atmospheric Administration-20 satellite platforms has infrared window bands but no carbon dioxide and water vapor absorption bands; the latter are essential for accurately deriving atmospheric variables (such as temperature and moisture profiles, cloud properties). An approach for fusing a high spatial resolution imager (e.g., VIIRS) and a high spectral resolution sounder [e.g., Cross-track Infrared Sounder (CrIS)] offers the opportunity to construct new spectral band radiances (radiance fusion) as well as sounder retrieval products (product fusion) at high spatial resolution. Furthermore, temperature and humidity profiles can also be provided at a high temporal resolution when geostationary imager [Advanced Baseline Imager (ABI)] data are used in combination with polar orbiting sounder (CrIS) data in a fusion approach. Promising results from VIIRS/CrIS and ABI/CrIS fusion are presented.


Introduction
Synergistic use of high spatial resolution imager data with high spectral resolution infrared (IR) sounder data provides advantages in various applications over the use of individual datasets alone. The "imager plus sounder" (or imager/sounder) data fusion method to construct high spatial resolution radiances (radiance fusion) have been demonstrated previously in Ref. 1. In this paper, we follow a similar fusion approach, called product fusion, where high vertical resolution (roughly 1 km) sounder temperature and water vapor (H 2 O) mixing ratio (also referred to as humidity in this paper) profile retrievals are being constructed at imager high spatial resolution (750 m to 2 km).
Product fusion transfers high spectral resolution (0.6 cm −1 in the longwave IR) soundings (at ∼14 km horizontal spatial resolution with an accuracy of 1 K temperature and 20% relative humidity for 1-km vertical layers in the troposphere) to imager high spatial horizontal resolution to reveal gradients in temperature and moisture at 750 m to 2 km horizontal scales at roughly 10 levels in the troposphere. The instruments of interest here include the following. The Cross-track Infrared Sounder (CrIS), onboard the National Oceanic and Atmospheric Administration (NOAA) operational Suomi-National Polar-orbiting Partnership (S-NPP) and NOAA-20 platforms in low Earth orbit (LEO), is a high spectral resolution (also simply referred to as hyperspectral) IR interferometer providing very accurate IR radiance measurements in more than 1300 channels, which are used to produce high vertical resolution temperature and humidity profile retrievals. On the same platforms, the Visible Infrared Imaging Radiometer Suite (VIIRS) is a high spatial resolution visible, near-IR, and IR imager of ocean, land, and atmospheric features providing data colocated with the CrIS. Onboard the Geostationary Operational Environmental Satellite (GOES)-R series, in geostationary orbit (GEO), the Advanced Baseline Imager (ABI) is a radiometer sensitive in the visible, near IR, and IR parts of the spectrum providing high spatial and temporal (i.e., hourly and better) resolution radiance measurements and imagery used for monitoring changes in the Earth's weather. Sounder and imager data from these LEO and GEO instruments are combined to generate retrieval products with both the high vertical resolution and high spatial resolution that can identify small-scale atmospheric features, especially changes in moisture concentration. ABI fusion with CrIS also offers hourly (or better) temporal changes in the tropospheric temperature and moisture that are important in nowcasting to monitor for the onset of convective activity.
In the radiance fusion approach, 1 a multidimensional nearest neighbor search, specifically a k-d tree search algorithm, is performed between imager radiances at the original high spatial resolution and imager radiances brought to sounder spatial resolution (through imager/sounder collocation and geographical averaging). This radiance search is used to find the coarse field-ofview (FOV) resolution radiances that best match the high pixel resolution radiances. For VIIRS, the split window radiances near 11 and 12 μm are used in the search. Then, the CrIS radiance measurements [convolved with the desired spectral response functions (SRFs) such as those from the Moderate Resolution Imaging Spectroradiometer or MODIS] associated with the previously found best matches are averaged to yield the final fusion estimates of the MODIS-like spectral band radiances at pixel resolution. It has been shown (in Ref. 1 and in global comparisons done since) that radiance fusion increases the spatial resolution of IR window radiances by an order of magnitude at the cost of adding roughly 0.5 K noise to the brightness temperatures. For the moisture and carbon dioxide (CO 2 ) sensitive spectral bands, the noise increases by about 1.0 K and 0.7 K, repectively, since the split window search emphasizes surface as well as total column moisture features, but not mid-or upper tropospheric CO 2 or H 2 O absorption characteristics.
To achieve sensitivity to atmosphere temperature and moisture vertical and horizontal gradients at imager spatial resolution, we explore two approaches for generating imager plus sounder fusion products. One approach is to use heritage sounding retrieval algorithms to derive atmospheric profile retrievals from the fusion radiances for each imager pixel. It is anticipated that these radiance fusion-based sounding products for the current S-NPP and NOAA-20 (and future NOAA) platforms will be more consistent with those from MODIS (on the NASA Earth Observing System Aqua and Terra platforms) than those derived from VIIRS radiances alone. Another approach, called product fusion, is to use the imager/sounder fusion approach directly on the sounder profile retrievals at sounder FOV resolution to infer them at imager pixel resolution. Specifically, in product fusion, the hyperspectral sounder retrieval products (i.e., instead of convolved radiance data) from the FOVs, selected in the k-d tree search between the imager radiances at coarse and high resolution, are averaged to create the new sounding products at imager high spatial resolution.
In both the VIIRS/CrIS radiance fusion as well the VIIRS/CrIS product fusion, the nearest neighbor search is using the VIIRS radiances from the split window near 11 and 12 μm. But, if the imager (e.g., ABI on GOES-platforms in GEO) has other bands in addition to the splitwindow bands, then the sensitivity to mid-and upper tropospheric levels can be improved. Therefore, the impact of performing the k-d tree search using the radiances in the split window plus additional IR spectral bands is studied as well.
A brief description of the satellite sensors and the data fusion methodology is provided in Sec. 2. Product fusion results for VIIRS/CrIS and ABI/CrIS pairings are presented and discussed in Sec. 3, whereas Sec. 4 summarizes the work and its implications for simulating geostationary hyperspectral soundings.

Cross-track Infrared Sounder
The CrIS is a Michelson interferometer with 1305 channels in normal spectral resolution covering from 3.92 to 15.38 μm with a nadir resolution of 14 km and cross-track scanning width of 2200 km. 2 The spectral resolution of CrIS is 0.625, 1.25, and 2.5 cm −1 , respectively, in the longwave, mid-wave, and short-wave spectral bands, with an option of providing observations with 0.625 cm −1 in all the spectral regions (the full spectral resolution option). Soundings derived from high spectral resolution CrIS measurements provide temperature and humidity profiles at single FOV resolution (13.5 km at nadir) with vertical accuracies of 1 K and 20% relative humidity per 1-km layer. Retrievals are computed at 101 levels going from 1100 to 0.005 hPa.

Visible Infrared Imaging Radiometer Suite
The VIIRS has 22 spectral bands covering the spectrum between 0.412 and 12.01 μm, including 16 moderate resolution bands (M-bands) at spatial resolution of 750 m, 5 imaging resolution bands (I-bands) at 375 m, and 1 panchromatic day-night band at 750-m spatial resolution throughout the scan covering a swath width of 3060 km. 3 The M-bands include 11 reflective solar bands and 5 thermal emissive bands (centered at 3.7, 4.0, 8.5, 11, and 12 μm). These IR bands are in atmospheric window spectral regions, so they are relatively insensitive to atmospheric absorbers, such as CO 2 and H 2 O.

Advanced Baseline Imager
The ABI, an imaging radiometer on the GOES, measures radiation in 16 spectral bands covering the visible, near-IR, and IR spectral regions. 4 The coverage rate for full disk scans is at least every 15 min, the continental U.S. (CONUS) is scanned every 5 min, and mesoscale coverage is provided at least every minute. ABI spatial resolution is 0.5 km for the 0.64-μm visible band, 1 km for selected near-IR bands, and 2 km at the subpoint for 10 IR spectral bands. The ABI bands 7 through 16 (centered at 3.9, 6.2, 7.0, 7.3, 8.5, 9.6, 10.3, 11.2, 12.3, and 13.3 μm, respectively) have some sensitivity to atmospheric CO 2 and H 2 O but not enough for retrieving temperature and moisture profiles.
It should be noted that ABI is not a sounder. 5 The operational ABI sounding products are minor modifications of the time-interpolated National Centers for Environmental Prediction (NCEP) Global Forecast System (GFS) model first guess. Hence, ABI/CrIS fusion offers the opportunity to include actual measurements sensitive to CO 2 and H 2 O in the troposphere (i.e., measured by the high spectral resolution CrIS) that provide independent information about the vertical structure of tropospheric temperature and moisture beyond that offered by the GFS.

Product Fusion Method
The imager/sounder radiance fusion method, which is described in more detail in Refs. 1 and 6, can be summarized as follows. To infer sounder radiances and products at imager spatial resolution, the fusion process requires two steps. In the first step, imager data are used as the input to a nearest neighbor search-specifically a multidimensional k-d tree search-to find the best pairing of imager single pixel radiances (usually in the split window at 11 and 12 μm) with a selected number (usually five) of nearby coarse resolution imager radiances. The latter are imager split window radiances averaged over collocated sounder FOVs. The collocation of VIIRS and ABI pixels within the CrIS FOV follows the process described in Ref. 7. The search finds, for each imager pixel, the closest, in radiance and geometrical space, low spatial resolution FOVs. In the second step, the mean of the sounder data (radiances or retrievals) at these selected FOVs is computed to provide the final fused product at imager pixel resolution. In the case of radiance fusion, the hyperspectral sounder data are first integrated over a selected SRF to produce a convolved radiance (and repeated for other desired spectral bands) before the nearest neighbor averaging is performed. These fusion radiances can then be used to infer profile, cloud and surface parameters; when MODIS SRFs are used, the heritage algorithms (such as those in the International MODIS and Atmospheric Infrared Sounder Processing Package, or IMAPP, described in Ref. 8) can be applied to the fusion radiances. Similarly, when performing product fusion, the sounder retrieval products (i.e., instead of convolved radiance data) from the FOVs selected in step one are averaged (for a given imager pixel) to create the new sounding products at imager pixel resolution; only CrIS soundings for FOVs determined to be clear of clouds by the sounder profile retrieval algorithm are averaged and displayed for imager pixels determined to be cloud-free by the imager cloud mask. These retrieval products, since they originate from hyperspectral retrievals of high vertical resolution and have been transferred to high spatial resolution through the fusion method, provide improved spatial mesoscale detail in temperature, moisture, and cloud features to the benefit of both hyperspectral sounder and imager applications, including severe weather nowcasting/prediction. In this paper, CrIS retrieval profiles are derived using the UW hyperspectral retrieval system, which is based on the dual-regression (DR) method, although any other retrieval algorithm can be used. DR (Refs. 9-11) is a fast physically based method that retrieves profiles as well as surface-and cloud-properties from high spectral resolution radiances measured in both clearand cloudy-sky conditions at single FOV resolution. The retrievals of surface skin temperature, atmospheric temperature, moisture, and ozone profiles (down to the surface in clear skies, above opaque clouds, and down below thin or broken clouds) are achieved with known vertical resolution and accuracy. Applying direct fusion to these CrIS retrieval products provides products now at high spatial (imager) resolution while maintaining high vertical resolution features.

VIIRS and CrIS Product Fusion
Although our focus here is on the direct product fusion (i.e., the k-d tree search results are directly applied to sounder retrievals), retrievals from the fusion radiance estimates are also presented to compare soundings at imager resolution produced in the two different ways: (1) VIIRS/ CrIS fusion is initiated to generate broadband CO 2 and H 2 O sensitive IR radiances at imager resolution (that are MODIS-like) and then a simple regression is used to produce retrievals with those radiances and (2) VIIRS/CrIS product fusion is performed, where DR CrIS high spectral resolution profile retrievals are directly transferred via the fusion process to high spatial imager resolution. The regression retrieval method used in (1) is based on the operational MODIS algorithm developed for IMAPP. 8 Figures 1 and 2 show the temperature and H 2 O retrieval results at selected pressure levels for two CrIS granules on April 10, 2018, at 0936 and 0942 UTC covering the western USA.
CrIS soundings at sounder resolution outline the tropospheric temperature and moisture gradients at coarse (∼14 km) resolution for clear and partly cloudy skies. Soundings from broad MODIS-like IR spectral bands at fine resolution (∼750 m) in pixels deemed to be clear in the  CrIS soundings mapped directly to fine resolution by the fusion process delineate mid-and low-level moisture features more in agreement with the CrIS soundings than the broadband retrievals. Fusion of the CrIS soundings directly to imager resolution captures more of the information content of the high spectral resolution measurements than fusion to broadband radiances followed by sounding retrievals. Although the fusion results are shown here only at 850, 500, and 300 hPa pressure levels, any of the 101 pressure levels provided in the CrIS profile retrievals could be shown.
CrIS DR retrievals and the product fusion results for another scene, namely the Chesapeake Bay area on May 19, 2018, around 18 UTC, are shown in Figs. 3 and 4. This case study will also be discussed in Secs. 3.2 and 3.3 with respect to LEO/GEO fusion. Overall, the fusion temperature products (Fig. 3) show increased coverage and fine-scale variations while maintaining the same features depicted by the hyperspectral retrievals.
Similar conclusions can be drawn for low-level H 2 O [ Fig. 4(b)], but the mid-level humidity fusion product [ Fig. 4(a)] shows some deviations from the hyperspectral retrieval products. These differences are due to the fact that the VIIRS split-window bands, which are not sensitive to mid-level moisture, are used in the k-d tree search.

ABI and CrIS Product Fusion
We now explore expanding the k-d tree search to include more imager spectral bands (when they are available). Using GEO and LEO data, specifically ABI data from 18 UTC and CrIS data from 1748 and 1754 UTC on May 19, 2017, we compare fusion results from k-d tree searches with just the ABI split window bands (11.2 and 12.3 μm) to those from k-d tree searches using eight out of the ten ABI IR bands. The center wavelengths of the eight bands are 6.2, 7.0, 7.3, 8.5, 10.3, 11.2, 12.3, and 13.3 μm, which correspond to ABI bands 8 through 11, and 13 through 16 (i.e., band 7 at 3.9 μm sensitive to reflected solar radiation and band 12 at 9.6 μm sensitive to stratospheric ozone are excluded).  Fig. 4(a), bottom] since the split windows for the two instruments are different (the ABI split window has more sensitivity to low-level moisture, which dominates the search). These moisture pockets are somewhat visible in Fig. 4, which shows the VIIRS/CrIS fusion results for the same case, but are less pronounced than the 8-band ABI/CrIS results. The low-level (850 hPa) H 2 O (Fig. 6, bottom) shows minor differences between the split-window versus eight band k-d tree search, implying that the information near the surface is mostly contained in the split-window bands (but the other window bands at 8.5 and 10.3 μm do contribute some additional information regarding surface features).

ABI and CrIS Product Fusion Time Sequence
The potential benefits of using hyperspectral sounder retrievals in the evaluation of atmospheric stability and real-time monitoring of extreme weather events are discussed in Ref. 13. Utilizing hyperspectral retrieval products transferred to high spatial resolution by means of the imager/ sounder product fusion method as well as employing a time series of prior and subsequent measurements may further contribute to the improvement of various severe weather forecasting and warning operations. With respect to the ABI/CrIS fusion process, opportunity now arises to perform additional k-d tree searches using ABI radiances every 15 min while connecting to the same fixed time CrIS retrieval product. The high spectral (which translates to high vertical) resolution retrieval products can then be generated not only at high spatial but also at high temporal resolution.
ABI radiances for the moisture sensitive band 8 (6.3 μm) at 16, 17 18, and 20 UTC on May 19, 2017, are shown in Fig. 7(a), illustrating the movement of mid-level moisture and intensification of clouds from the western to the northeastern part of the region. Figure 7(b) shows the mid-level (500 hPa) humidity ABI/CrIS product fusion results, which have been derived from combining the ABI radiances at these four times with the CrIS soundings at 1748 and 1754 UTC. Figure 7(c) shows the NCEP 13-km rapid refresh (RAP) model analysis for the humidity at 500 hPa for the same four times. The ABI/CrIS product fusion is spanning 4 h, which precede and follow the sounder overpass; it suggests the moisture increase as well as adjacent drying at the 500-hPa pressure level from 16 to 17 UTC (moistening in panel b is outlined by a red frame, drying by the black frame). The mid-level drying is followed by convective activity in eastern Virginia found independently in the ABI image at 20 UTC [convection is highlighted by the black frame in Fig. 7(a)] and in the RAP (although displaced somewhat to the north). Low-level moistening and mid-level drying are used in nowcasting as indicators of preconvective conditions, hence, the rapid changes in 500 hPa H 2 O mixing ratio are of interest in this time sequence. Finally, Fig. 7(d) shows the associated low-level (850 hPa) humidity ABI/CrIS product fusion results. The low-level moisture fields have more missing pixels due to cloud interference than the mid-level moisture results. But they show the low-level moistening from 16 to 17 UTC (eastern box outlined in black) connected with mid-level drying [shown in Fig. 7(b)] that presages convective development; the low-level drying to the west (outlined in red) shows where convection updrafts have already occurred and moisture has been transported to the mid-levels. The color scale has been adjusted to emphasize the low-level-moisture gradients over land. The RAP hourly 850 hPa H 2 O mixing ratio field is shown in Fig. 7(e); small scale features are not readily discernable but the general movement of low-level moisture eastward is apparent.
It should be mentioned that the model analysis incorporates the radiosondes as well as the available satellite data. The errors associated with the retrieval are documented in the literature (see, for example, Ref. 11) and those associated with imager sounder radiance fusion are cited in Ref. 1. The comparison of the moisture gradients with model analysis highlights that both (i.e., model analysis and product fusion) describe the same synoptic situation but the imager sounder fusion provides more small-scale features that are meteorologically consistent with developing weather events.
The results of pursuing fusion of the CrIS tropospheric stability retrieval with the ABI radiances are shown in Fig. 8 where convection occurs 4 h later. The indication from the ABI only product is much less pronounced. This example suggests that the imager/sounder fusion can be accomplished across different times, and therefore can provide the high temporal resolution requisite for predicting the onset and the evolution of severe convection.

Summary and Conclusions
While VIIRS does not have spectral bands sensitive to CO 2 , H 2 O, and O 3 , the fusion technique enables the construction of MODIS-like absorption band radiances using high spatial resolution VIIRS and high spectral resolution CrIS radiance data. The first step of the fusion method incorporates a k-d (multidimensional) tree search to find neighboring points from a training dataset to a query dataset. In this application, training and query are represented by low (FOV) and high (pixel) spatial resolution imager radiances, respectively. In the second step, the sounder data   for the FOVs selected in the first step are then averaged and associated with each imager pixel. In the case of the radiance fusion demonstrated in Ref. 1, radiance bands can be constructed by using the k-d tree search output to average over the neighboring sounder FOVs of the spectrally convoluted hyperspectral sounder radiances. This paper presents product fusion, where the k-d tree search output is directly applied to hyperspectral sounder retrieval products. Hence, for each imager pixel, the sounder retrievals of the sounder FOVs (found through the imager-only k-d tree search) are averaged. Therefore, the product fusion directly offers more spatial detail in temperature, moisture, and cloud features. Temperature and moisture profile retrievals derived from broadband VIIRS/CrIS radiance fusion are compared with those derived from VIIRS/CrIS product fusion. We found that product fusion retains more of the information content of the CrIS high spectral resolution radiances thus presenting more mid-tropospheric details in moisture and temperature.
Preliminary GEO/LEO product fusion results are also presented. The ABI instrument onboard GOES enables a k-d tree search with more spectral bands (beyond the split window of VIIRS). Using the additional ABI spectral bands enables better delineation of mid-level dry and moist features; this is attributed to the inclusion of H 2 O sensitive 6.2-, 7.0-, and 7.3-μm spectral bands. The very clean (least affected by atmospheric moisture) window band at 10.3 μm and the surface type sensitive window band at 8.5 μm influence the low-level moisture determinations more subtly. Since the ABI/CrIS fusion incorporates high spectral resolution CrIS measurements sensitive to CO 2 and H 2 O in the troposphere, it provides independent information about the vertical structure of tropospheric temperature and moisture well beyond that offered by the ABI alone. The high frequency of ABI measurements is also utilized when the k-d tree search is applied to each set of ABI radiances in time sequence and connected to a fixed time CrIS retrieval product; thus, high spectral (which translates to high vertical) resolution retrieval products are fused into high spatial as well as high temporal resolution. Changes in low-and mid-level moisture are more readily monitored in the ABI/CrIS fusion, helping to delineate areas of more likely tropospheric instability. Corroboration is offered with independent model analysis data in a convective development case study. Overall, the case studies presented here suggest the possibility of successfully transferring sounder high vertical resolution to imager high spatial and temporal resolution. It should be noted that improving spatial resolution via fusion does not alleviate sounder retrieval errors and that low-level RH retrieval errors need to be taken into account during severe weather mesoscale analysis.
Since high spatial resolution radiances can be constructed for any existing or newly desired IR spectral band from imager/sounder pairs, new opportunities are available for retrieving accurate atmospheric and surface products and for studying their impact on operational and research applications. We show promising results for temperature and moisture retrieval products, but, additionally, any hyperspectral retrieval parameter (e.g., severe weather indices, cloud parameters) can be transferred to high spatial resolution through the product fusion method. For example, ongoing product fusion-related studies include the investigation of H 2 O to track polar winds and trace gases like sulfur dioxide (SO 2 ) to detect volcanic plumes. Furthermore, any imager/ sounder pair on the same platform or on different satellites can be used. AVHRR/IASI (onboard the Metop satellites) fusion and the demonstrated VIIRS/CrIS fusion can be used to enable continuity of the Terra and Aqua MODIS cloud and moisture records. 14 The imager/sounder product fusion results demonstrated here suggest that the reach of either instrument can be extended through combination via the fusion approach. Specifically, the vertical resolution of hyperspectral sounding products and the horizontal and temporal resolution of imager radiances can be blended for use in, for example, severe weather monitoring and prediction systems. data have been provided courtesy of the Space Science and Engineering Center/University of Wisconsin-Madison.