The US National Park Service (NPS) preserves the natural night sky, to the greatest extent possible, as a natural resource and value. Our first step towards understanding and protecting the night sky is to assess the night sky quality. In the parks, we capture a series of overlapping high-resolution images to obtain a mosaic view of the entire night sky. The NPS Night Skies Program has specialized pipeline to perform data reduction and processing of these 45 one-million-pixel images taken by our mobile camera system. Specifically, the pipeline applies basic reduction, performs photometric calibration, mosaics the images to form the panoramic views of the entire night sky, generates the models for the natural sky brightness, and calculates values of different sky brightness metrics. This original data pipeline was developed mainly by a single person about a decade ago. These scripts are written in three different languages including Python, Java, and Visual Basic and interact with six different proprietary software packages including ACP Observatory Control, ArcGIS, MaxIm DL, Microsoft Excel, Photoshop, and Pinpoint. Although the current data pipeline is complete and functional, an upgrade is needed to make the pipeline distributable and manageable. Recognizing the challenges of moving forward, NPS Night Skies Program recently formed a pipeline development team for this upgrade effort. We, the members in the pipeline development team, are a multidisciplinary group of NPS staff with background in astronomy, geography, biology, and engineering. Geographically, we are located in both the mainland US and Alaska. Our goal is to transform the entire pipeline to be completely written in Python and minimize the number of required proprietary software. In addition, we will allow any future pipeline updates or new development to be made or suggested by any user instead of from a single-person initiated top-down model. For our pipeline upgrade project, we are using Git and GitHub for version control, project management, and source-code distribution. Once the upgrade is completed, the pipeline will be distributed via GitHub to NPS staff and partners located in Alaska, Nevada, Colorado, Tennessee, and Texas. All of our Python pipeline scripts will be open source where we hope to also benefit other scientists in the similar research field worldwide.
The Gemini Planet Imager Exoplanet Survey (GPIES) is a multiyear direct imaging survey of 600 stars to discover and characterize young Jovian exoplanets and their environments. We have developed an automated data architecture to process and index all data related to the survey uniformly. An automated and flexible data processing framework, which we term the Data Cruncher, combines multiple data reduction pipelines (DRPs) together to process all spectroscopic, polarimetric, and calibration data taken with GPIES. With no human intervention, fully reduced and calibrated data products are available less than an hour after the data are taken to expedite follow up on potential objects of interest. The Data Cruncher can run on a supercomputer to reprocess all GPIES data in a single day as improvements are made to our DRPs. A backend MySQL database indexes all files, which are synced to the cloud, and a front-end web server allows for easy browsing of all files associated with GPIES. To help observers, quicklook displays show reduced data as they are processed in real time, and chatbots on Slack post observing information as well as reduced data products. Together, the GPIES automated data processing architecture reduces our workload, provides real-time data reduction, optimizes our observing strategy, and maintains a homogeneously reduced dataset to study planet occurrence and instrument performance.
The Gemini Planet Imager (GPI) has been designed for the direct detection and characterization of exoplanets and circumstellar disks. GPI is equipped with a dual channel polarimetry mode designed to take advantage of the inherently polarized light scattered off circumstellar material to further suppress the residual seeing haloleft uncorrected by the adaptive optics. We explore how recent advances in data reduction techniques reduce systematics and improve the achievable contrast in polarimetry mode. In particular, we consider different flux extraction techniques when constructing datacubes from raw data, division by a polarized at-field and a method for subtracting instrumental polarization. Using observations of unpolarized standard stars we find that GPI's instrumental polarization is consistent with being wavelength independent within our errors. In addition, we provide polarimetry contrast curves that demonstrate typical performance throughout the GPIES campaign.
The Gemini Planet Imager (GPI) is a high-contrast instrument specially designed for direct imaging and spectroscopy of exoplanets and debris disks. GPI can also operate as a dual-channel integral field polarimeter. The instrument primarily operates in a coronagraphic mode which poses an obstacle for traditional photometric calibrations since the majority of on-axis starlight is blocked. To enable accurate photometry relative to the occulted central star, a diffractive grid in a pupil plane is used to create a set of faint copies, named satellite spots, of the occulted star at specified locations and relative intensities in the field of view. We describe the method we developed to perform the photometric calibration of coronagraphic observations in polarimetry mode using these fiducial satellite spots. With the currently available data, we constrain the calibration uncertainty to be <13%, but the actual calibration uncertainty is likely to be lower. We develop the associated calibration scripts in the GPI Data Reduction Pipeline, which is available to the public. For testing, we use it to photometrically calibrate the HD 19467 B and β Pic b data sets taken in the H-band polarimetry mode. We measure the calibrated flux of HD 19467 B and β Pic b to be 0:078±0:011 mJy and 4:87±0:73 mJy, both agreeing with other measurements found in the literature. Finally, we explore an alternative method which performs the calibration by scaling the photometry in polarimetry mode to the photometrically calibrated response in spectroscopy mode. By comparing the reduced observations in raw units, we find that observations in polarimetry mode are 1:03 0:01 brighter than those in spectroscopy mode.
The Gemini Planet Imager has been successfully obtaining images and spectra of exoplanets, brown dwarfs, and debris and protoplanetary circumstellar disks using its integral field spectrograph and polarimeter. GPI observations are transformed from raw data into high-quality astrometrically and photometrically calibrated datacubes using the GPI Data Reduction Pipeline, an open-source software framework continuously developed by our team and available to the community. It uses a flexible system of reduction recipes composed of individual primitive steps, allowing substantial customization of processing depending upon science goals. This paper provides a broad overview of the GPI pipeline, summarizes key lessons learned, and describes improved calibration methods and new capabilities available in the latest version. Enhanced automation better supports observations at the telescope with streamlined and rapid data processing, for instance through real-time assessments of contrast performance and more automated calibration file processing. We have also incorporated the GPI Data Reduction Pipeline as one component in a larger automated data system to support the GPI Exoplanet Survey campaign, while retaining its flexibility and stand-alone capabilities to support the broader GPI observer community. Several accompanying papers describe in more detail specific aspects of the calibration of GPI data in both spectral and polarimetric modes.