Translator Disclaimer
16 September 2017 Earth-observing satellite intercomparison using the Radiometric Calibration Test Site at Railroad Valley
Author Affiliations +
This paper describes the current ground-based calibration results of Landsat 7 Enhanced Thematic Mapper Plus (ETM+), Landsat 8 Operational Land Imager (OLI), Terra and Aqua Moderate Resolution Imaging Spectroradiometer (MODIS), Suomi National Polar orbiting Partnership Visible Infrared Imaging Radiometer Suite (VIIRS), and Sentinel-2A Multispectral Instrument (MSI), using an automated suite of instruments located at Railroad Valley, Nevada, USA. The period of this study is 2012 to 2016 for MODIS, VIIRS, and ETM+, 2013 to 2016 for OLI, and 2015 to 2016 for MSI. The current results show that all sensors agree with the Radiometric Calibration Test Site (RadCaTS) to within ±5% in the solar-reflective regime, except for one band on VIIRS that is within ±6%. In the case of ETM+ and OLI, the agreement is within ±3%, and, in the case of MODIS, the agreement is within ±3.5%. MSI agrees with RadCaTS to within ±4.5% in all applicable bands.



Airborne and spaceborne sensors provide critical data to the scientific community for Earth-observation studies. Long-term environmental data sets are maturing and expanding with the launch of an ever-increasing number of sensors from both the government and commercial sectors.1 Scientific models are created using sensors with a wide variety of spectral, spatial, and temporal characteristics, so it is essential to place all of the data from these sensors on the same radiometric scale to ensure long-term accuracy and consistency in the models. Complex satellite platforms often have onboard calibration systems to monitor radiometric stability, but they must also be independently monitored to ensure their stability over time. The recent development of CubeSat and nanosat constellations has created new challenges for the vicarious calibration community. One ground-based solution is the development of automated sites that collect data in the absence of personnel. Recent efforts by the Remote Sensing Group (RSG) at the University of Arizona resulted in the development of the Radiometric Calibration Test Site (RadCaTS) at Railroad Valley, which is currently being used to calibrate such sensors as Landsat 7 Enhanced Thematic Mapper Plus (ETM+), Landsat 8 Operational Land Imager (OLI), the Moderate Resolution Imaging Spectroradiometer (MODIS) onboard Terra and Aqua, Suomi National Polar orbiting Partnership (SNPP) Visible Infrared Imaging Radiometer Suite (VIIRS), and the European Space Agency (ESA) Sentinel-2 Multispectral Instrument (MSI), among others.23.4.5 RadCaTS is also one of four current sites that constitute the Radiometric Calibration Network (RadCalNet), which is a Committee on Earth Observation Satellites (CEOS) Working Group on Calibration and Validation (WGCV). The goal of RadCalNet is to provide the user community with high-quality, harmonized radiometric calibration data with well-documented uncertainties. The RadCalNet working group has completed a 2-year prototyping phase, and the data are free to registered users as of late-2017.6

The reflectance-based approach to vicarious calibration is one of the traditional methods used by RSG and other research groups. On-site personnel typically make atmospheric and surface reflectance measurements during a sensor overpass. The results of the measurements function as input into a suitable radiative transfer code to determine the top-of-atmosphere (TOA) quantities of interest (e.g., spectral radiance or reflectance). Finally, the TOA results are compared to the corresponding imagery to validate the calibration of the sensor. An obvious requirement of the reflectance-based approach is on-site personnel, which is becoming increasingly difficult with the number of sensors in orbit. Collecting field data for constellations of satellites such as RapidEye creates a greater challenge to the calibration community in terms of both personnel time and cost.7 RSG developed RadCaTS in response to the need for automated accurate satellite calibration measurements in the solar-reflective regime. Railroad Valley was chosen as the location for RadCaTS since it had already been in use by the group since 1996 and it is well understood spatially, spectrally, and temporally.8,9

The work presented here is a summary of the current results obtained using RadCaTS for ETM+, OLI, MODIS, VIIRS, and MSI from the period 2012 to 2016. Section 2 describes the methodology used in RadCaTS, Sec. 3 describes the data used in this work, and Secs. 4 and 5 provide the results and conclusions, respectively.



The development of RadCaTS is a natural progression from the use of on-site personnel. The techniques involved in determining the TOA radiance and reflectance are similar, but there are slight differences in the instrumentation used to determine TOA spectral radiance and reflectance. Measurements of radiance reflected from the test site surface are made using all-weather multispectral ground-viewing radiometers (GVRs) that were developed by RSG specifically for RadCaTS.10 The GVRs have eight spectral channels that use narrow-bandwidth interference filters for spectral selection, and they are centered at 400, 450, 500, 550, 650, 850, 1000, and 1550 nm. The focal plane comprises seven silicon detectors for bands 1 to 7 and an InGaAs detector for band 8. The detector focal plane is designed with a 2 deg inward tilt so that each detector observes the same area on the ground when the head is placed at a height of 1.5 m. This translates to a spot size of 27  cm in diameter on the ground. The detector head is kept at 25.0°C±0.1°C throughout the day during clear-sky conditions to ensure that the detector responsivity during field use is the same as when the unit is calibrated in the laboratory. Each GVR is an autonomous instrument that is battery powered and solar charged. Data are downloaded daily to the base station via a wireless connection, and all data are downloaded to the University of Arizona for processing on a daily basis.

Figure 1 shows two GVRs, one of which is in a nadir-viewing configuration for satellites in a Sun-synchronous orbit. The other GVR is in a viewing configuration for sensors on geosynchronous satellites. Recent RSG work includes the advanced baseline imager (ABI), which is onboard the recently launched National Oceanic and Atmospheric Administration Geostationary Operational Environmental Satellite (GOES 16, which was named GOES R prior to launch). There are currently four GVRs in a nadir-viewing configuration and one with a viewing configuration similar to GOES 16. The GVRs are located throughout the 1-km2 site in such a manner as to balance spatial sampling with accessibility.

Fig. 1

(a) GVRs at RadCaTS in a nadir-viewing configuration and (b) GOES 16 viewing configuration.


Periodic radiometric calibration of the GVRs is used to determine the temporal stability and to check for potential issues. Currently, two lightweight portable transfer radiometers developed by RSG are used for the periodic on-site calibration of the GVRs.11 The Calibration Test Site SI-Traceable Transfer Radiometer (CaTSSITTR) instruments are multispectral portable radiometers that use the same seven silicon channels of the GVRs. The InGaAs, the GVR channel was not included to minimize the size and power requirements of CaTSSITTR. Prior to the development of CaTSSITTR, equipment was carried to each GVR to perform a solar-radiation-based calibration (SRBC), which limited the distance of the GVRs to the north–south access road that bisects the site.12,13 The main advantage in using the SRBC is that the Sun is effectively the calibration source, which can minimize spectral mismatches that can occur with the use of a spherical integrating source.10,12,14 The CaTSSITTR instruments will minimize the complexity of doing field calibrations of the GVRs, and they will also be used to assist in the calibration of the ground-viewing instruments at the various RadCalNet sites.

The GVRs are used to determine a multispectral surface reflectance of the 1-km2 RadCaTS site for a time of interest. The multispectral data are fit to a hyperspectral data set through the use of a library of hyperspectral data that were collected on the RadCaTS site for MODIS. The hyperspectral data set is shown in Fig. 2; it consists of 80+ individual data sets from 2000 to 2016. Once the GVR multispectral surface reflectance for a given time of interest is converted to a hyperspectral surface reflectance using a least-squares fit, it is used in MODerate resolution atmospheric TRANsmission (MODTRAN) to determine the TOA quantities of interest in a manner similar to the traditional reflectance-based approach.

Fig. 2

Bidirectional reflectance factor data collected for Aqua and Terra MODIS. These data are used as a reference library for RadCaTS to convert multispectral GVR data to hyperspectral for use in MODTRAN. The noisy features at 1375 and 1880 nm are due to atmospheric absorption features that reduce the signal-to-noise ratio of the data.


Atmospheric measurements at RadCaTS are made using a Cimel CE318-T Sun–Sky–Lunar multiband photometer, which is part of the Aerosol Robotic Network (AERONET).15 The Cimel CE318-T is a multispectral instrument that operates from 340 to 1640 nm. It makes direct solar measurements, sky radiance measurements, and nighttime lunar measurements (when the moon is illuminated >50%) to determine the aerosol optical properties. The aerosol optical depth, water vapor, and Angstrom exponent16,17 for the time of interest are downloaded from the AERONET web site and are used as input into the MODTRAN radiative transfer code.18 Initially, the surface reflectance for the time of interest has to be determined; this requires that MODTRAN be used to determine the diffuse sky irradiance. Once the surface reflectance has been determined, MODTRAN is run again to determine the TOA spectral radiance for the time of interest.18 Another input required by MODTRAN is the columnar ozone amount. Data from either the ozone monitoring instrument onboard the Aura spacecraft or the ozone mapping profiler suite onboard the SNPP spacecraft are used.1920.21.22

Railroad Valley is located in central Nevada, 5 h north of Las Vegas. The area of the dry lake bed, or playa, is 15×15  km (Fig. 3). The 1×1-km area chosen for RadCaTS is at an altitude of 1435 m. An example of the surface reflectance variation throughout the year is shown in Fig. 2, which illustrates why on-site instrumentation must be used to measure the temporal surface reflectance. There is minimal human contact, which means that the site is generally free of tire tracks. Changes in the surface characteristics are generally due to rain and snow. The atmospheric aerosol loading is generally low throughout much of the year, with an average aerosol optical depth of 0.060 at 550 nm.23

Fig. 3

True color image of Railroad Valley, Nevada, imaged by Sentinel-2A MSI. The 1-km2 RadCaTS region is shown in yellow.





Satellite Data

The Earth-observation sensors used in this study reside on flagship platforms operated by the National Aeronautics and Space Administration (NASA), the United States Geological Survey (USGS), and the ESA. The imagery was obtained from USGS Earth Explorer, NASA Level 1 Atmosphere and Archive Distribution System Distributed Active Archive Center, and the ESA Sentinels Scientific Data Hub web sites. The imagery used in this study is all freely available to registered users of the respective data portals. The period of study for this work is 2013 to 2016 for Landsat 8 OLI, 2015 to 2016 for Sentinel-2A, and 2012 to 2016 for all other sensors. The 2012 starting date for this study corresponds to the deployment of the current temperature-stabilized GVRs at RadCaTS.

The satellite data product used in this study is the radiometrically and geometrically corrected TOA spectral radiance, except for Sentinel-2A MSI, which is the TOA reflectance. In all cases except MSI, the exoatmospheric solar irradiance model used for each sensor is that recommended by the satellite operator. Table 1 describes the characteristics of the sensors in this study, which are multispectral and range from 10 to 1000 m in spatial resolution. In the case of those that have wide spectral coverage (e.g., midwave and thermal infrared), only the solar-reflective bands are used (400 to 2500 nm). The repeat cycle for a given view angle is 16 days, except for Sentinel-2A, which has a repeat cycle for a given view angle of 10 days.

Table 1

The satellites and sensors used in this study. Multiple view angles occur on different days. The exoatmospheric solar irradiance model, ESUN, is shown for each sensor except Sentinel-2A MSI.

Satellite and sensor
Landsat 7 ETM+Terra MODISAqua MODISSNPP VIIRSLandsat 8 OLISentinel-2A MSI
Launch date (mm dd, yyyy)April 15, 1999December 18, 1999May 04, 2002October 28, 2011February 11, 2013June 23, 2015
Spectral bands (used in study)7 (6)36 (9)36 (9)21 (13)9 (7)13 (11)
Wavelength range, all bands (μm)0.4 to 12.50.4 to 14.40.4 to 14.40.4 to 12.50.4 to 2.20.4 to 2.2
Pixel size (m)30250, 500, 1000250, 500, 1000375, 7503010, 20, 60
View angle(s) for RadCaTS (degrees)0.50.5, 11.2, 12.14.8, 7.00.6, 9.8,, 11, 3
Collection (if applicable)16631101N/A


Ground Data

GVRs take measurements every 2 min, but the detector focal plane is only temperature stabilized when a voltage threshold is reached. This reduces power consumption overnight, as well as during poor sky conditions, which is important since the instrument is battery powered. The meteorological station makes temperature, pressure, wind speed and direction, and relative humidity measurements every 2 min. The Cimel Sun photometer makes measurements based on the AERONET measurements protocol. The processed Cimel data are automatically downloaded by RSG on a daily basis.

Cimel measurements are being filtered for times that may indicate poor atmospheric conditions. Currently, Angstrom exponent values between 0.9 and 1.5 are accepted as good data, while overpass times that have Angstrom exponent values outside this interval are rejected. Values below the 0.9 threshold indicate that there may be thin cirrus present (large particles), and values >1.5 indicate that there may be smoke present (small particles). Work is currently underway to further understand the correlation between the Angstrom exponent, aerosol optical depth, and spatial homogeneity and their impact on the uncertainty of the final TOA result.




Landsat 7 Enhanced Thematic Mapper Plus and Landsat 8 Operational Land Imager

The comparison between the TOA spectral radiance reported by ETM+ and OLI using RadCaTS is shown in Fig. 4 for period 2012 to 2016 in the case of ETM+ and 2013 to 2016 in the case of OLI. The uncertainty bars in the results are the 4% uncertainty estimate of RadCaTS TOA radiance that was determined in a previous study.5 The number of successful overpasses for each sensor during this period is 11 for Landsat 7 and 12 for Landsat 8. Cloud cover is the typical reason for unsuccessful collections, although rain and snow can also be factors. Temporal results indicate that there is no discernable degradation in any band of ETM+ or OLI during the period of this study.

Fig. 4

RadCaTS results for Landsat 7 ETM+ and Landsat 8 OLI during the period 2012 to 2016 (L7) and 2013 to 2016 (L8). The uncertainty bars are the ±4% uncertainty of RadCaTS.



Terra and Aqua Moderate Resolution Imaging Spectroradiometer

The results for bands 1 to 9 of Terra and Aqua MODIS are shown in Fig. 5 for the period 2012 to 2016. The Terra MODIS results comprise 59 successful collects, while the Aqua MODIS results comprise 26 successful collects. As with the Landsat results, the uncertainty bars are the 4% uncertainty of RadCaTS. The temporal results do not indicate a discernable change in the calibration over the 2012 to 2016 period for either sensor.

Fig. 5

A summary of the Terra and Aqua MODIS RadCaTS results for the period 2012 to 2016.



Suomi National Polar orbiting Partnership Visible Infrared Imaging Radiometer Suite

The SNPP VIIRS results for the period 2012 to 2015 are shown in Fig. 6 for the relevant 375-m imagery (I) and 750-m moderate (M) bands. The year 2016 was not included in this study because collection 3110 data were only available up until February 2016. Collection 5000 is currently being processed, and the results will be updated once the imagery becomes available. There were a total of 51 successful collections during the period 2012 to 2016.

Fig. 6

A summary of the SNPP VIIRS results from 2012 to 2016. The uncertainty bars are the ±4% uncertainty of RadCaTS.



Sentinel-2A Multispectral Instrument

The Sentinel-2A MSI results from the launch in 2015 to the end of 2016 are shown in Fig. 7, which is a summary of 21 successful collections at RadCaTS. Figure 8 shows a comparison of TOA reflectance between similar bands of Landsat 8 OLI and Sentinel-2A MSI using the same 1km2 region of RadCaTS. It should be noted that the TOA reflectance product of Landsat 8 OLI follows a different calibration path (solar diffuser panel) than the TOA spectral radiance product.

Fig. 7

RadCaTS results for Sentinel-2A MSI. The uncertainty bars are the 4% uncertainty of RadCaTS.


Fig. 8

RadCaTS results for Sentinel-2A MSI (S2A MSI) and Landsat 8 OLI (L8 OLI) (a) VNIR and (b) SWIR bands.




The work presented here is a summary of the current RadCaTS radiometric calibration results for Landsat 7 ETM+, Landsat 8 OLI, Terra and Aqua MODIS, Suomi NPP VIIRS, and Sentinel-2A MSI. The ground-based measurements of RadCaTS generally agree to within ±5% of the satellite-derived TOA spectral radiance. The TOA reflectance products of OLI are in agreement to within the ±3% specification, and MSI is generally within ±4%, except for the 1.6-μm band, which is at the 5% agreement level. Temporal results during this time have not exhibited any appreciable degradation in any of the sensors.

RadCaTS has been using the current design of temperature-stabilized GVRs since 2012. The stability and accuracy of these instruments allow the RSG at the University of Arizona to use RadCaTS as the primary data collection source. It is one of four sites currently in operation as part of RadCalNet, which becomes operational and open to all registered users in 2017. Ground-based radiometric calibration and surface reflectance validation work at RadCaTS continues to evolve. New GVRs are being developed and deployed in response to work with ABI on GOES 16 (GOES-R prior to launch) and GOES 17 (GOES-S prior to launch), and a new Cimel CE318-T Sun–Sky–Lunar spectrometer was deployed to Railroad Valley in February 2017. RadCaTS surface reflectance data will continue to be provided to RadCalNet, where it will be processed to TOA reflectance on a daily basis for distribution to registered users. SNPP VIIRS results will continue to be uploaded to the NASA web portal.24 RadCaTS uncertainty analysis will also continue through collaboration with RadCalNet and by comparison with in situ measurements using the reflectance-based approach.


The authors declare that there are no conflicts of interest.


The authors would like to thank the Bureau of Land Management, Tonopah, Nevada, Office, for their assistance in gaining access to Railroad Valley. The authors would also like to thank Brent Holben and AERONET for processing and distributing the Railroad Valley Cimel data. This work was funded by the following sources: NASA research Grant Nos. NNX14AJ19G, NNX14AP68A, and NNX16AH44G, and USGS cooperative Agreement No. G14AC00371. Finally, the authors would like to thank the reviewers of this paper for their thoughtful comments and suggestions.



A. S. Belward and J. O. Skøien, “Who launched what, when and why; trends in global land-cover observation capacity from civilian earth observation satellites,” ISPRS J. Photogramm. Remote Sens., 103 115 –128 (2015). IRSEE9 0924-2716 Google Scholar


B. L. Markham et al., “Landsat-7 ETM+: 12 years on-orbit reflective-band radiometric performance,” IEEE Trans. Geosci. Remote Sens., 50 (5), 2056 –2062 (2012). IGRSD2 0196-2892 Google Scholar


J. S. Czapla-Myers et al., “The absolute radiometric calibration of Terra imaging sensors: MODIS, MISR, and ASTER,” Proc. SPIE, 9218 92180Y (2014). Google Scholar


B. Markham et al., “Landsat-8 operational land imager radiometric calibration and stability,” Remote Sens., 6 (12), 12275 –12308 (2014). Google Scholar


J. Czapla-Myers et al., “The ground-based absolute radiometric calibration of Landsat 8 OLI,” Remote Sens., 7 (1), 600 –626 (2015). Google Scholar


D. Naughton et al., “Absolute radiometric calibration of the RapidEye multispectral imager using the reflectance-based vicarious calibration method,” J. Appl. Remote Sens., 5 (1), 053544 (2011). Google Scholar


J. S. Czapla-Myers et al., “Temporal, spectral, and spatial study of the automated vicarious calibration test site at Railroad Valley, Nevada,” Proc. SPIE, 7081 70810I (2008). PSISDG 0277-786X Google Scholar


J. S. Czapla-Myers, K. J. Thome and J. H. Buchanan, “Implication of spatial uniformity on vicarious calibration using automated test sites,” Proc. SPIE, 6677 66770U (2007). Google Scholar


N. Anderson et al., “Design and calibration of field deployable ground-viewing radiometers,” Appl. Opt., 52 (2), 231 –240 (2013). APOPAI 0003-6935 Google Scholar


N. Anderson et al., “Design of an ultra-portable field transfer radiometer supporting automated vicarious calibration,” Proc. SPIE, 9607 960709 (2015). Google Scholar


N. Anderson et al., “Solar radiation-based calibration of laboratory grade radiometers,” Proc. SPIE, 6677 66770X (2007). Google Scholar


J. S. Czapla-Myers and N. P. Leisso, “Recent results from the Radiometric Calibration Test Site (RadCaTS) at Railroad Valley, Nevada,” Proc. SPIE, 7807 78070R (2010). Google Scholar


M. A. Kuester et al., “Solar radiation based calibration of an airborne radiometer for vicarious calibration of earth observing sensors,” Proc. SPIE, 4483 85 (2001). Google Scholar


B. N. Holben et al., “AERONET—a federated instrument network and data archive for aerosol characterization,” Remote Sens. Environ., 66 1 –16 (1998). Google Scholar


A. Ångström, “On the atmospheric transmission of sun radiation and on dust in the air,” Geografiska Annaler, 11 156 –166 (1929). Google Scholar


A. Ångström, “On the atmospheric transmission of sun radiation. II,” Geografiska Annaler, 12 130 –159 (1930). Google Scholar


A. Berk et al., “MODTRAN6: a major upgrade of the MODTRAN radiative transfer code,” Proc. SPIE, 9088 90880H (2014). Google Scholar


S. P. Ahmad et al., “Atmospheric products from the ozone monitoring instrument (OMI),” Proc. SPIE, 5151 619 (2003). Google Scholar


J. P. Veefkind et al., “Total ozone from the ozone monitoring instrument (OMI) using the DOAS technique,” IEEE Trans. Geosci. Remote Sens., 44 (5), 1239 –1244 (2006). IGRSD2 0196-2892 Google Scholar


L. E. Flynn, J. Homstein and E. Hilsenrath, “The ozone mapping and profiler suite (OMPS). The next generation of US ozone monitoring instruments,” in Proc. IEEE Int. Geoscience and Remote Sensing Symp. (IGARSS 2004), 152 –155 (2004). Google Scholar


J. V. Rodriguez et al., “An overview of the nadir sensor and algorithms for the NPOESS ozone mapping and profiler suite (OMPS),” Proc. SPIE, 4891 65 (2003). Google Scholar


K. Thome et al., “Ground-reference calibration results for Landsat-7 ETM+,” Proc. SPIE, 5882 58820B (2005). PSISDG 0277-786X Google Scholar


Jeffrey Czapla-Myers is an associate research professor in the College of Optical Sciences at the University of Arizona. He received his BS degree in optical engineering from the University of Arizona in 1997, his MSc degree in earth and space science from York University (Toronto, Canada) in 2000, and his PhD in optical sciences from the University of Arizona in 2006. He is currently a member of the Landsat Calibration Working Group, the MODIS Science Team, and the SNPP Science Team. He is a senior member of SPIE.

Joel McCorkel received his BS degree in optical engineering and his PhD in optical sciences from the University of Arizona. He joined the National Ecological Observatory Network as a staff scientist from the Airborne Observation Platform in 2009. He is currently a physical scientist in the Biospheric Sciences Laboratory at NASA’s Goddard Space Flight Center, Greenbelt, Maryland.

Nikolaus Anderson is an optical engineer with extensive mechanical and optomechanical design experience. He received his BS degree from the University of Arizona 2002. He has optical system design and testing experience, including end-to-end production and testing of several custom calibration instruments working in a small engineering team and with customers, external machinists, and vendors. His research projects include development of experimental lab and field characterizations and calibration instrumentation customized for specific customer needs.

Stuart Biggar is a research professor at the University of Arizona. He received his BS and MS degrees in physics from the United States Air Force Academy and the Ohio State University in 1971 and 1972, respectively. He received his PhD in optical sciences from the University of Arizona in 1990. He is interested in radiometry, optical system radiometric calibration, and reflectance measurement in the laboratory and field.

© The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.
Jeffrey Czapla-Myers, Joel McCorkel, Nikolaus Anderson, and Stuart Biggar "Earth-observing satellite intercomparison using the Radiometric Calibration Test Site at Railroad Valley," Journal of Applied Remote Sensing 12(1), 012004 (16 September 2017).
Received: 1 June 2017; Accepted: 25 August 2017; Published: 16 September 2017


Back to Top