An initiative is under way at ESO Headquarters to optimise operations, in particular in the engineering, technical and associated management areas. A systematic approach to strengthen the operating processes is in preparation, starting with a mapping of the extensive existing process network. Processes identified as sufficiently important and complex to merit an in-depth analysis will be properly specified and their implementation optimised to strike a sensible balance between organisational overhead (documentation) and efficiency. By applying methods and tools tried and tested in industry we expect to achieve a more unified approach to address recurrent tasks. This will enable staff to concentrate more on new challenges and improvement and avoid spending effort on issues already resolved in the past.
In the past three years the QC group at ESO has installed a production line for science-grade data products. With the focus on spectroscopic observations, these in-house generated data products are complementary to the externally provided data products from the surveys. The production line combines efficient mass production (more than one million spectra have been generated so far), previews, and quality control. All data products are available to the community on the ESO archive interface.
A Low Humidity and Temperature Profiling (LHATPRO) microwave radiometer is used to monitor sky conditions over ESO’s Paranal observatory. It provides measurements of precipitable water vapour (PWV) at 183 GHz, which are being used in Service Mode for scheduling observations that can take advantage of favourable conditions for infrared (IR) observations. The instrument also contains an IR camera measuring sky brightness temperature at 10.5 μm. It is capable of detecting cold and thin, even sub-visual, cirrus clouds. We present a diagnostic diagram that, based on a sophisticated time series analysis of these IR sky brightness data, allows for the automatic and quantitative classification of photometric observing conditions over Paranal. The method is highly sensitive to the presence of even very thin clouds but robust against other causes of sky brightness variations. The diagram has been validated across the complete range of conditions that occur over Paranal and we find that the automated process provides correct classification at the 95% level. We plan to develop our method into an operational tool for routine use in support of ESO Science Operations.
ESO has implemented a process to automatically create science-grade data products and offer them to the scientific community, ready for scientific analysis. This process, called 'phoenix', is built on two main concepts: 1. a certification procedure for pipelines which includes a code review and, if necessary, upgrade; and 2. a certification procedure for calibrations which are processed into master calibrations, scored and trended. These master calibrations contain all information about the intrinsic instrumental variations and instabilities inevitable for ground-based telescopes. The phoenix process then automatically processes all science data using the certified pipeline and the certified master calibrations. Phoenix currently focuses on spectroscopic data. The first phoenix project has been the processing of all science data from UVES, ESO's high-resolution Echelle spectrograph at the VLT. More than 100,000 Echelle spectra of point sources, from begin of operations (March 2000) until now, have been reduced and are available to the public from the ESO archive, http://archive.eso.org/cms/eso-data/eso-data-products.html. The phoenix process will also feed future UVES data into the archive. The second project has been X-SHOOTER slit spectroscopy which currently has more than 30,000 Echelle spectra from the UV to the infrared (up to 2.5μm). The phoenix process will be extended to other, mostly spectroscopic, instruments with certified pipelines, like FLAMES. Also, all future VLT instruments will be supported by phoenix.
A Low Humidity and Temperature Profiling (LHATPRO) microwave radiometer, manufactured by Radiometer Physics GmbH (RPG), is used to monitor sky conditions over ESO’s Paranal observatory in support of VLT science operations. In addition to measuring precipitable water vapour (PWV) the instrument also contains an IR camera measuring sky brightness temperature at 10.5 μm. Due to its extended operating range down to -100 °C it is capable of detecting very cold and very thin, even sub-visual, cirrus clouds. We present a set of instrument flux calibration values as compared with a detrended fluctuation analysis (DFA) of the IR camera zenith-looking sky brightness data measured above Paranal taken over the past two years. We show that it is possible to quantify photometric observing conditions and that the method is highly sensitive to the presence of even very thin clouds but robust against variations of sky brightness caused by effects other than clouds such as variations of precipitable water vapour. Hence it can be used to determine photometric conditions for science operations. About 60 % of nights are free of clouds on Paranal. More work will be required to classify the clouds using this technique. For the future this approach might become part of VLT science operations for evaluating nightly sky conditions.
The European Southern Observatory Science Archive Facility is evolving from an archive containing predominantly raw data into a resource also offering science-grade data products for immediate analysis and prompt interpretation. New products originate from two different sources. On the one hand Principal Investigators of Public Surveys and other programmes reduce the raw observational data and return their products using the so-called Phase 3 - a process that extends the Data Flow System after proposal submission (Phase 1) and detailed specification of the observations (Phase 2). On the other hand raw data of selected instruments and modes are uniformly processed in-house, independently of the original science goal. Current data products assets in the ESO science archive facility include calibrated images and spectra, as well as catalogues, for a total volume in excess of 16 TB and increasing. Images alone cover more than 4500 square degrees in the NIR bands and 2400 square degrees in the optical bands; over 85000 individually searchable spectra are already available in the spectroscopic data collection. In this paper we review the evolution of the ESO science archive facility content, illustrate the data access by the community, give an overview of the implemented processes and the role of the associated data standard.
A new technological development, the laser driven light source (LDLS), in which a laser excited plasma emits intense
continuum radiation over a wide wavelength range from well below the atmospheric cut-off up to 800 nm, promises to
greatly improve our ability to provide high quality flat-fields for astronomical spectrographs. Its particular strength lies
in the ground-based ultraviolet (UV). We report on tests conducted with a LDLS using FORS2, UVES, X-Shooter and
CRIRES at ESO’s Very Large Telescope (VLT) in August 2013. Comparison with standard calibration sources such as
halogen and deuterium lamps shows that with the LDLS flat-fields with a better balanced dynamic range and excellent
signal to noise ratio can be achieved within short exposure times. This will enable higher quality science at the short
wavelength end of existing spectrographs at the VLT. Furthermore the LDLS provides exceptional stability and long
lifetime as important operational aspects. Optimised UV spectrographs such as the proposed CUBES (wavelength range
300-400 nm) project will be able to take full advantage of this development removing the long-standing limitation of
signal to noise ratios of UV flat-fields.
We present the performance characteristics of a water vapour monitor that has been permanently deployed at ESO’s
Paranal observatory as a part of the VISIR upgrade project. After a careful analysis of the requirements and an open call for tender, the Low Humidity and Temperature Profiling microwave radiometer (LHATPRO), manufactured by
Radiometer Physics GmbH (RPG), has been selected. The unit measures several channels across the strong water vapour emission line at 183 GHz, necessary for resolving the low levels of precipitable water vapour (PWV) that are prevalent on Paranal (median ~2.5 mm). The unit comprises the above humidity profiler (183-191 GHz), a temperature profiler (51-58 GHz), and an infrared radiometer (~10 μm) for cloud detection. The instrument has been commissioned during a 2.5 week period in Oct/Nov 2011, by comparing its measurements of PWV and atmospheric profiles with the ones obtained by 22 radiosonde balloons. In parallel an IR radiometer (Univ. Lethbridge) has been operated, and various observations with ESO facility spectrographs have been taken. The RPG radiometer has been validated across the range 0.5 – 9 mm demonstrating an accuracy of better than 0.1 mm. The saturation limit of the radiometer is about 20 mm. Currently, the radiometer is being integrated into the Paranal infrastructure to serve as a high time-resolution monitor in support of VLT science operations. The water vapour radiometer’s ability to provide high precision, high time resolution information on this important aspect of the atmosphere will be most useful for conducting IR observations with the VLT under optimal conditions.
In support of characterization of potential sites for the European Extremely Large Telescope (E-ELT) the European
Southern Observatory (ESO), the Institute for Space Imaging Science (ISIS) and the astrometeorology group of the
Universidad Valparaiso have jointly established an improved understanding of atmospheric precipitable water vapour
(PWV) above ESO's La Silla Paranal Observatory. In a first step, 8 years worth of high resolution near-IR spectra taken
with VLT-UVES have been statistically analysed to reconstruct the PWV history above Paranal. To this end a radiative
transfer model of Earth's atmosphere (BTRAM) developed by ISIS has been used. A median PWV of 2.1 mm is found
for Paranal based on UVES data covering the period 2001-2008. Furthermore we conclude that Paranal can serve as a
reference site for Northern Chile due to the stable atmospheric conditions in the region. The median offset between
Paranal and Armazones is derived to be 0.3 mm, but local arbitrary variations of a few tenths of a mm between the sites
have been found by measurement. In order to better understand the systematics involved two dedicated campaigns were
conducted in August and November 2009. Several methods for determining the water column were employed, including
radiosonde launches, continuous measurements by infrared radiometer, and VLT instruments operating at various
wavelengths: CRIRES, UVES, VISIR and X-shooter. In a first for astronomical instruments all methods have been
evaluated with respect to the radiosondes, the established standard in atmospheric research. Agreement between the
radiosondes and the IR radiometer (IRMA) is excellent while all other astronomical methods covering a wavelength
range from 700 - 20000 nm have also been successfully validated in a quantitative manner. All available observations
were compared to satellite estimates of water vapour above the observatory in an attempt to ground-truth the satellite
data. GOES can successfully be used for site evaluation in a purely statistical approach since agreement with the
radiosondes is very good on average. For use as an operational tool at an observatory GOES data are much less suited
because of significant deviations depending on atmospheric conditions. We propose to routinely monitor PWV at the
VLT and to use it as an operational constraint to guide scheduling of IR observations at Paranal. For the E-ELT we find
that a stand-alone high time resolution PWV monitor will be essential for optimizing the scientific output.
The ESO telescopes in Chile are operated in a geographically distributed scheme, in which some of the essential steps in
the end-to-end observing chain take place in Europe. Most notably, the health status of the instruments as derived from
the data themselves is monitored in Europe and the results fed back to the observatory within the hour. The flexibility of
this scheme strongly depends on the speed with which the data stream produced by the telescopes can be sent to Europe
for analysis and storage. The main challenge to achieve a fast intercontinental data transfer is the data volume itself,
which currently reaches an average 25 GB/night (compressed) for the four VLT Unit Telescopes. Since late 2008, this
stream has been entirely transferred through the internet via a 4.56 Mbit/s bandwidth assured via a Quality of Service
policy, which sufficed to transfer an average night of data within a few hours. A very recent enlargement of this capacity
to 9.12 Mbit/s will soon allow the addition of the calibration data for VISTA, the new infrared survey telescope on
Paranal, to the data stream transferred through the internet. Ultimately, the average data volume produced on Paranal
once the visible VLT Survey Telescope (VST) and the full complement of second-generation VLT instruments becomes
available is expected to exceed 200 GB/night. Transferring it over the internet will require a new fiber-based
infrastructure currently under construction, as well as the use of additional high bandwidth channels. This infrastructure,
provided by the European Union co-funded project EVALSO, should provide a data transfer capacity exceeding 1 Gbit/s
that will allow the transfer to Europe of the entire Paranal data stream, as well as that of the nearby Observatory of Cerro
Armazones and of the future European Extremely Large Telescope, with a delay of minutes at most since the data were
VIRCAM is the wide field infrared camera of the VISTA survey telescope on Paranal. VIRCAM, operated by
ESO since Oct. 2009, is equipped with 16 detectors and produces on average 150 Gigabytes of data per night.
In the following article we describe the back-end data flow operations and in particular the quality control
procedures which are applied to ESO VIRCAM data.
Quality Control (QC) of calibration and science data is an integral part of the data flow process for the ESO
Very Large Telescope (VLT) and has guaranteed continuous data quality since start of operations. For each
VLT instrument, dedicated checks of pipeline products have been developed and numerical QC parameters to
monitor instrumental behavior have been defined. The advent of the survey telescopes VISTA and VST with
multi-detector instruments imposes the challenge to transform the established QC process from a detector-by-detector
approach to operations that are able to handle high data rates and guarantee consistent data quality.
In this paper, we present solutions for QC of multi-detector instruments and report on experience with these
concepts for the operational instruments CRIRES and VIMOS. Since QC parameters scale with the number of
detectors, we have introduced the concept of calculating averages (and standard deviations) of parameters across
detectors. This approach is a powerful tool to evaluate trends that involve all detectors but is also able to detect
outliers on single detectors. Furthermore, a scoring system has been developed which compares QC parameters
for new products to those from already existing ones and gives an automated judgment about data quality. This
is part of the general concept of information on demand: detailed investigations are only triggered on a selected
number of products.
By 2010, the Paranal Observatory will host at least 15 instruments. The continuous increase in both the complexity and
quantity of detectors has required the implementation of novel methods for the quality control of the resulting stream of
data. We present the new and powerful concept of scoring which is used both for the certification process and the Health
Check monitor. Scoring can reliably and automatically measure and assess the quality of arbitrarily amounts of data.
GIRAFFE is an intermediate resolution spectrograph covering a wavelength range from 360-930nm and fed by
optical fibers as a part of FLAMES, the multi-object fiber facility mounted at the ESO VLT Kueyen. For some time we sought a new detector for GIRAFFE spectrograph to boost the instrument's red QE (Quantum Efficiency) capabilities, while still retaining very good blue response. We aimed also at reducing the strong fringing present in the red spectra. The adopted solution was an e2v custom 2-layer AR (Anti-Reflection) coated Deep Depletion CCD44-82 CCD. This device was made in a new e2v Technologies AR coating plant and delivered to ESO in mid 2007 with performance that matches predictions. The new CCD was commissioned in May 2008. Here we report on the results.
The ESO Very Large Telescope (VLT) started operations on Cerro Paranal (Chile) in April 1999 with one Unit Telescope and two science instruments. Seven years later it is still a growing facility consisting of four 8.2-m telescopes, three auxiliary telescopes for interferometry, and 11 science instruments. In addition two dedicated survey telescopes with wide-field cameras, VST and VISTA, a fourth auxiliary telescope, and several new instruments will become available in the coming months. Since the very beginning, VLT operations were planned to contain a substantial component of Service Mode observing, amounting to approximately 50% of the available time. The success of the full-scale implementation of Service Mode operations is reflected nowadays by the steady increase in its demand by the community, both in absolute terms and also relative to the demand in Visitor Mode, by the highly positive feedback received from the users, and also by the increasing flow of scientific results produced by programs that have exploited the unique advantages of flexible short-term scheduling. It is also fulfilling the requirement of creating a science archive and populating it with a data stream having through a quality control process. Here we review the current status of Service Mode observing at the VLT and the VLT Interferometer (VLTI), the challenges posed by its implementation on a wide variety of instrument modes, and its strong requirement of an integrated, end-to-end approach to operations planning with adequate tools and carefully defined policies and procedures. The experience of these seven years of VLT operations have led to a thorough exploration of operations paradigms that will be essential to the scientific success of ALMA and the extremely large optical telescopes in the coming decades.
The performance of all scientific instruments of the Very Large Telescope (VLT) is monitored by the Quality Control (QC) Group of the European Southern Observatory. Basic goals are to detect instrumental failures on a short time basis and to evaluate and detect long-term trends. The QC process mainly involves pipeline-produced calibration products and is set up on a file by file
basis. This implies that currently each detector or channel of an instrument is checked separately. All operational VLT instruments have a low number of detectors but with the advent of multi-detector instruments like OmegaCAM and VISTA, which have up to 32 individual detectors, this approach becomes unfeasible. In this paper, we present solutions for this problem for the VLT instrument VIMOS. With four detectors operating simultaneously, VIMOS can be regarded as a test bed for studying new QC concepts which can be implemented for other instruments with higher multiplicity.
With the completion of the first generation instrumentation set on the Very Large Telescope, a total of eleven instruments are now provided at the VLT/VLTI for science operations. For each of them, ESO provides automatic data reduction facilities in the form of instrument pipelines developed in collaboration with the instrument consortia. The pipelines are deployed in different environments, at the observatory and at the ESO headquarters, for on-line assessment of observations, instruments and detector monitoring, as well as data quality control and products generation. A number of VLT pipelines are also distributed to the user community together with front-end applications for batch and interactive usage. The main application of the pipeline is to support the Quality Control process. However, ESO also aims to deliver pipelines that can generate science ready products for a major fraction of the scientific needs of the users. This paper provides an overview of the current developments for the VLT/VLTI next generation of instruments and of the prototyping studies of new tools for science users.
Four years after its announcement at SPIE, FLAMES, the VLT fibre facility, has been completed, integrated into the VLT observatory and commissioned. It has been in operation since February 2003. More than 250000 scientific (single) spectra have been obtained, which have enabled the on-sky performance of the instrument to be compared to the predictions. We show that in several relevant aspects the real instrument significantly outperforms the specified astronomical performance. Some of the early scientific results are finally presented.
UVES-fiber is part of the FLAMES instrument mounted on the 8.2m
Kueyen Telescope (UT2) of the ESO VLT. Up to eight single object fibers can be linked from the FLAMES focus to the red arm of the echelle spectrograph UVES. Science and calibration data are pipeline-processed by the Data Flow Operations group of ESO. Parameters to monitor the performance of the instrument are routinely extracted from calibration frames, stored into a database, and monitored over time. In addition to the Quality Control parameters already present for UVES in slit mode, several specific procedures had to be added in
order to monitor the performance in the multi-object case. Particular attention is required for the positioning of the fibers on the detector and the transmission of the fibers. In this paper, we present details of the Quality Control process for UVES-fiber and results from the first year of operations.
GIRAFFE is a medium to high resolution spectrograph forming part of the complex multi-element fibre spectrograph FLAMES on the 8.2m VLT-UT2 telescope which also has a fibre link to the high-resolution spectrograph UVES. It has been operational since March 2003. GIRAFFE has been designed to be very stable and efficient. Here, first results concerning the Quality Control process are presented.
UVES is the UV-Visual high-resolution Echelle Spectrograph mounted at the 8-m Kueyen (UT2) telescope of the ESO VLT. In order to allow use of UVES at its highest resolution of up to 110 000, also during non-optimal seeing conditions, the instrument is equipped with Bowen-Walraven type image slicers. These devices have exit slits of 0.3, 0.44 and 0.68 arcseconds and possibility to view the sky next to the slicer for sky subtraction. This paper addresses the relevant UVES optical design aspects, image slicer design and manufacturing, observing procedures and usage statistics. In the last part of the paper we give examples of high-S/N observations made with the 0.3 arcsecond image slicer.
Currently four instruments are operational at the four 8.2m telescopes of the European Southern Observatory Very Large Telescope: FORS1, FORS2, UVES, and ISAAC. Their data products are processed by the Data Flow Operations Group (also known as QC Garching) using dedicated pipelines. Calibration data are processed in order to provide instrument health checks, monitor instrument performance, and detect problems in time. The Quality Control (QC) system has been developed during the past three years. It has the following general components: procedures (pipeline and post-pipeline) to measure QC parameters; a database for storage; a calibration archive hosting master calibration data; web pages and interfaces. This system is part of a larger control system which also has a branch on Paranal where quick-look data are immediately checked for instrument health. The VLT QC system has a critical impact on instrument performance. Some examples are given where careful quality checks have discovered instrument failures or non-optimal performance. Results and documentation of the VLT QC system are accessible under http://www.eso.org/qc/.
UVES is the UV-Visual high-resolution echelle spectrograph mounted at the 8.2m Kueyen (UT2) telescope of the ESO Very Large Telescope. Its data products are pipeline-processed and quality checked by the Data Flow Operations Group (often known as QC Garching). Calibration data are processed to create calibration products and to extract Quality Control (QC) parameters. These parameters provide instrument health checks and monitor instrument performance. Typical UVES QC parameters are: bias level, read-out-noise, dark current of the three CCD detectors used in the instrument, rms of dispersion, resolving power, CCD pixel-to-pixel gain structure, instrument efficiency. The measured data are fed into a database, compared to earlier data, trended over time and published on the web (http://www.eso.org/qc/index_uves.html). The QC system has evolved with time and proven to be extremely useful. Some examples are given which highlight the impact of careful QC on instrument performance.
Operations of the first ESO Very Large Telescope (VLT), the 8m-Antu, started on April 01, 1999, and two instruments, FORS1 (FOcal Reducer/low dispersion Spectrograph) and ISAAC (Infrared Spectrometer And Array Camera) became available to the user community on the same day. ESO's Data Flow Concept embraces the definition of observations (Phase 2 Proposal Preparation, P2PP), the execution (Paranal Science Operations), and the different functions of Data Flow Operations (DFO). As part of DFO, the Quality Control Scientists are in control of the following tasks: (1) distribution of data, (2) creation of master calibration data, (3) reduction of science data, (4) performance of quality control and trend analysis. In this paper we will describe in more detail the work within the Quality Control Group, with particular emphasis on the different methodologies applied to the two instruments, and give the insider point of view on operations with a large telescope after almost one year of experience.