22 January 2018 Astro-H/Hitomi data analysis, processing, and archive
Author Affiliations +
J. of Astronomical Telescopes, Instruments, and Systems, 4(1), 011207 (2018). doi:10.1117/1.JATIS.4.1.011207
Astro-H is the x-ray/gamma-ray mission led by Japan with international participation, launched on February 17, 2016. Soon after launch, Astro-H was renamed Hitomi. The payload consists of four different instruments (SXS, SXI, HXI, and SGD) that operate simultaneously to cover the energy range from 0.3 keV up to 600 keV. On March 27, 2016, JAXA lost contact with the satellite and, on April 28, they announced the cessation of the efforts to restore mission operations. Hitomi collected about one month’s worth of data with its instruments. This paper presents the analysis software and the data processing pipeline created to calibrate and analyze the Hitomi science data, along with the plan for the archive. These activities have been a collaborative effort shared between scientists and software engineers working in several institutes in Japan and United States.
Angelini, Terada, Dutka, Eggen, Harrus, Hill, Krimm, Loewenstein, Miller, Nobukawa, Rutkowski, Sargent, Sawada, Takahashi, Yamaguchi, Yaqoob, and Witthoeft: Astro-H/Hitomi data analysis, processing, and archive



Astro-H,1 renamed Hitomi after launch, is a facility-class mission launched on a JAXA H-IIA rocket into low Earth orbit on Feb. 17, 2016, at 5:45 pm JPS from Tanegashima Space Center in Japan. Hitomi was Japan’s sixth x-ray astronomy mission primarily developed at the Institute of Space and Astronautical Science of Japan Aerospace Exploration Agency (ISAS/JAXA) in collaboration with United States (NASA/GSFC) and Japanese institutions, with contributions from the European Space Agency, the Netherlands Institute for Space Research (SRON), the Canadian Space Agency, and additional US and European institutions. Hitomi was equipped with four different instruments that together cover a wide energy range of 0.3 to 600 keV. The soft x-ray spectrometer (SXS2), which pairs a lightweight soft x-ray telescope3 (SXT) with an x-ray calorimeter spectrometer, provides nondispersive spectroscopy with <7  eV resolution in the 0.3 to 12 keV energy range with a field-of-view (FoV) of about 3 arc min. Three additional scientific instruments extend the energy bandpass of the observatory. The soft x-ray imager (SXI4) expands the FoV with a new generation CCD camera in the energy range of 0.5 to 12 keV at the focus of the second lightweight SXT3; the hard x-ray imager (HXI,5,6 two units) performs sensitive imaging spectroscopy in the 5- to 80-keV band; and the nonimaging soft gamma-ray detector (SGD,7 two units) extends Hitomi’s energy band to 600 keV. In addition, there are three subsystems: two for the SXS, the modulated x-ray source (MXS8 to calibrate the gain) and the anticoincidence system; and one for the HXI, the Canadian Astro-H Metrology System (CAMS9) that tracks the movement of the extended optical bench (EOB). On March 27, 2016, JAXA lost contact with the satellite, and on April 28, JAXA announced that they would cease efforts to restore mission operations. The Hitomi instruments collected about one month’s worth of data that have been processed and archived.

The following sections describe the Hitomi data format, the software developed to calibrate and analyze the science data, how the data are processed on ground, the different levels of data creation, and how the data are archived. The latest updates on software, calibration, and archive are posted at Ref. 10.


Design of the FITS Files, Software, Processing and Archive

The design of the ground processing was developed with the same guiding principles and models used by previous high energy astrophysics missions, such as ASCA, RXTE, Swift, and Suzaku. The telemetry data are reformatted into FITS format at an early stage, and the analysis software directly operates on these FITS files. The FITS files are designed following the HEASARC/OGIP FITS conventions to facilitate multimission data analysis and the usage of existing software. The Hitomi software is built within the HEASoft environment and distributed by the HEASARC. HEASoft does not depend on any specific environment, and it is supported on many operating systems commonly used in high energy astrophysics. The calibration data are all ingested into the HEASARC calibration database (CALDB). The calibration information is stored in files, also in FITS format, and interfaces with the software via routines that query the CALDB metadata. The pipeline uses the same data pipeline environment as ASCA, Swift, and Suzaku, and the data processing uses the Hitomi software distributed with HEASoft.


Data Division and FITS Format

The Hitomi data are organized by sequence number and placed in the archive after processing. A sequence number may contain the entire observation of an object if shorter than 1 day. Observations longer than a day are divided into different sequences on the boundary between days from the start time of the observation. Each observation contains both pointing and incoming slew data for all instruments, subsystems, and housekeeping (HK). Figure 1 shows the different time boundaries. The science data files for each instrument are divided into slew and pointing data and may be further divided by the instrument mode or filter. The data from the instrument and spacecraft HK, as well as from the attitude and orbit, are instead not separated by slew and pointing. If an observation spans more than one sequence, then only the first sequence contains the slew data. The science files within a sequence have a size limited to 2 Gb.

Fig. 1

An observation contains all of the data collected between T_SA and T_EA, which indicate the start and end of the attitude. The T_SS and T_ES values indicate the start and the end of the slew and the T_SO and T_EO values are the start and end of the actual pointing. Within the sequence, only the science data are divided into separate files that contain only slew data or only pointing data.


All Hitomi telemetry data within a sequence are reformatted into FITS without loss of information, using either the BINTABLE or IMAGE extensions or the FITS primary header. The science data are organized in FITS BINTABLE extensions using the EVENT format, where each row contains columns with information specific to an event to characterize the coordinates, the time, and the energy of the event. The HK data are time-ordered FITS BINTABLE tables recording several parameters related to the instruments and subsystems. The files generated from the telemetry are first named FITS files (FFF) and after calibration are named second FITS files (SFF and SFFa). In addition to the events associated with the x-ray source, the science file for each of the instruments contains different types of signal related either to the calibration sources or electronic indicators that are used to derive corrections. The SXS and SXI files contain columns with either a single element or fixed array in one row with information related to one event. The FFF for the HXI and SGD instead contains variable length arrays. The x-ray interaction with the HXI or SGD detectors at any one time produces an ensemble of signals. The number of signals in one interaction (hereafter “occurrence”) is variable and stored in variable length array columns. The path of these signals (belonging to an occurrence) within the detectors is reconstructed to identify an event by the calibration software. The reconstructed events for the HXI and SGD are stored in the SFFa, which do not have variable length array columns.


Software and Calibration Data

The calibration and analysis of the Hitomi data require mission-specific tasks, as well as multimission tasks included within HEASoft.11 The Hitomi-specific tasks consist of 78 tools to support various stages of the calibration of the instruments and their subsystems, the data filtering, and to perform simulations for all instruments. Each task is built as an FTOOL (this is the standard within HEASoft) and uses a standard parameter file as interface, the CFITSIO library to access and write FITS files, and a common makefile to build the code. Each of the Hitomi tasks is dedicated to a specific function and is written in C or C++ or in Perl.

The tasks to calibrate the events detected by the different instruments include the assignment of time, energy, and coordinates for each event. The calibration of the subsystems is for the MXS, (used for time-dependent gain correction of the SXS data), the SXS anticoincidence signal, and the CAMS (used to calculate the coordinates for the HXI). There are tasks to calculate an ancillary response file [ARF, the effective area (EA) of the telescope which also includes the detector properties], an instrument response matrix file necessary for the spectral analysis, and also tasks to calculate an exposure map for imaging analysis.12 Several Perl scripts facilitate the data calibration and data filtering. A raytracing code—to model the soft and hard x-ray telescopes,13 as well as some general simulation tasks, to generate events for a specific source, and to model the sky background14—has also been developed. The analysis of the Hitomi data makes use of the standard multimission software packages existing within HEASoft: XSELECT, to extract events, light curves and/or spectra filtered for a region and/or specific pulse phases or source intensity; XSPEC,15 XRONOS,16 and XIMAGE17 for spectral, timing, and imaging analysis.

The Hitomi software uses the HEASARC CALDB to access the Hitomi calibration information.18 The Hitomi calibration data are in FITS files and include all the prelaunch information as well as the postlaunch updates. The calibration files are designed to accommodate dependencies either on time or specific mode or other parameters to allow software to retrieve the correct file via the CALDB metadata that lists these dependences. The updates of the calibration information do not require changes in the software but only updates of the CALDB files and metadata.


Data Flow: Prepipeline, Pipeline and Archive

The processing data flow consists of two steps: the prepipeline (run at ISAS Japan) that decodes the telemetry data into FITS file and organizes the observations into sequences and the pipeline (run at GSFC/USA) that calibrates and cleans the data and transfers the data to the archives (Fig. 2). In the prepipeline, the Hitomi telemetry data are divided into observations, using the time boundaries, as shown in Fig. 1, and stored in files with a FITS wrapper named raw package telemetry (RPT). The FFF are created by decoding the telemetry from the RPT. The FFF with the additional support files, such as those for the attitude and orbit, are packaged together and labeled with a sequence number (Table 1). These files have the same structure and the same number of columns as the final FITS files, but the prepipeline only populates the columns with the telemetered values (to maintain the original information without any conversion) leaving blank the columns to store the calibrated values. An exception to this is the TIME column containing the time values. This column is calculated using the tasks related to time and included in the Hitomi software package to ensure that the attitude and orbit data are also time-tagged consistently. The pipeline populates the remaining columns in the FITS file with the calibrated information for position and energy (the SFF) and creates high level data product files. The data processing in the pipeline outputs: (a) the unfiltered calibrated data (SFF file also level 1/1a), where the information for all events is maintained, i.e., no events are discarded; (b) the good time intervals (GTIs), which exclude unwanted times either derived from orbital or instrument HK parameters; (c) the filtered or cleaned files (level 2), where the data are screened for unwanted event flags and selected within the GTIs; (d) additional files to monitor the instrument performance (trend data); (e) the attitude corrected for aberration; (f) extended orbital information not included in the orbit file; (g) a filter file containing HK information necessary for screening; and (h) the database file containing high level information for each observation used by the archive. The tasks used to process the data in the pipeline as well as to calculate the timing information used in the prepipeline are all part of the Hitomi package so that users may always recalibrate or rescreen the data if new calibration information is available or special screening is needed in their analysis.

Fig. 2

The data processing flow: prepipeline for data preparation, pipeline for data processing, and archive.


Table 1

Data generated by the prepipeline are the RPT and FFF files. Data generated by the pipeline are the levels 1, 1a, 2, and 3 files and they populate the archive.

RPT: level 0
FITS wrapper (science + HK)
FFF: level 0Time assignedTime assignedTime assignedTime assignedTime assignedTime assigned
FITS decoded (science + HK)
SFF: level 1Calibrated eventsCalibrated eventsUncalibratedCalibrated eventsCalibrated positions and fine time of measurementsHK fine time assignment
Public archive
Start file for reprocessingFix length columnFix length columnVariable length array column Calibrated histograms
SFF= FFF + columns filled + GTIMSX time assignedDark frame eventRemapped channels
Multiple events in single row
SSFa: level 1a
FITSCalibrated pixel events NoCalibrated events NoCAMS delta attitudeNot applicable
Public archiveCalibrated baseline eventsFix length column
Single event in one row
Cleaned events: level 2Pixel eventsEventsEventsEvents Not applicableNot applicable
Public archive
Products images lightcurve spectra: level 3YesYesYesNoNot applicableNot applicable
Public archive

The pipeline data processing flow consists of a general part applicable to all instruments and a specific part for each individual instrument. The general part applies the aberration correction to the attitude; calculates the average pointing based on the time interval when the telescope was on source; derives the times for the South Atlantic Anomaly, the satellite day and night, cutoff rigidity, and other orbital quantities necessary for screening the data; and calculates observation-dependent quantities, as for the optical axis, that are stored as keywords in the header of the final FITS files. The data processing for the individual instruments involves several steps to calculate the coordinates and the energy of the events, set quality flags for the events with a quality status, calibrate the subsystem (MXS, SXS anticoincidence detector and CAMS) and derived quantities needed either for checking the instrument performance or to be used for further postprocessing, for example, to derive the non x-ray background (NXB). After the science data are calibrated, the processing applies screening criteria to calculate the GTIs and removes events flagged as not useful for the science analysis.

The pipeline processes the data by sequence and, at the end of each run, the files within a sequence are packaged and sent to the HEASARC (USA) and DARTS (Japan) archives, as is the case for the trend data. The data are organized in the archive by observation and by trend.19

The observation data are divided into directories, one for each sequence. Under each sequence directory, there are directories for each instrument and one for the auxiliary data. The CAMS data and the MXS data are included within the HXI and SXS directories, respectively. The different science data levels are stored in subdirectories within each of the instrument directories. Each sequence contains all the science, HK, and auxiliary files for all the instruments and subsystems.

The trend data are divided into instrument directories with one subdirectory dedicated to each trend type. The trend data consist of either HK information with no further processing or quantities calculated in the pipeline for monitoring special instrument characteristics.

During the 38 days of operation, Hitomi observed six astronomical objects and made several blank sky observations. These data have been divided into 42 sequences. These observations were taken during the commissioning phase and not all instruments were operational for all observations. The SXS started operating soon after launch and collected data from February 23, 2016, the SXI from March 6, 2016, the HXI1 and HXI2 from March 11 and March 14, 2016, respectively, and last, the SGD1 and SGD2 from March 20 and 22, respectively.


Time, Coordinates, and Energy Assignment

This section describes the main software tasks written to calibrate the time, coordinates, and energy for all Hitomi instruments. These tasks are run as part of the data processing. The calibrated values are stored in columns of the science FITS files. The final column names used in the analysis are TIME to record the time of the event; DETX/Y and X/Y to record the pixel position in the detector or sky coordinates systems; and PI to record the channel number that is used to extract a spectrum. The science FITS files for each instrument also contain other supporting columns that record previous steps necessary to derive TIME, DETX, and DETY, X and Y, and PI. The Hitomi software package also includes scripts to reprocess the data either to include new calibration updates or to test special parameter settings (ahpipeline, sxspipeline, sxipipeline, hxipipeline, sgdpipeline, ahcalctime).


Time Assignment

The time assignment framework is general to all science instruments and HK. Hitomi has a GPS receiver on board that dispatches the time to all subsystems as a time indicator (TI in units of 26  s). The instruments’ electronics assign the fine time to the science data. If the GPS is off, the time is assigned similarly to how it was done for the Suzaku satellite, where the TI is calibrated using the time assigned to the telemetry packets, the receiving time on ground, and the temperature–frequency relation of the on-board clock. Terada et al.20 contain a detailed description of the on-board time assignment.

The main tasks in the Hitomi package that assign time are “ahmktim” and “ahtime.” Additional instrument-dependent tasks are used to precalculate the instrument fine time. ahmktim computes a table (TIM table) of TIME versus TI (the lower 32 bit of the TI value, L32TI) used by ahtime. The calculation uses the combination of the GPS flags that indicate when the GPS is on, off, in transition or in an illegal state. If the GPS is on, the TIME versus TI is obtained using the values of the time to when the packets are tagged (stored in a column named S_TIME) and using the L32TI column. If the GPS is off, TIME versus TI is calculated using the closest GPS time to derive the temperature of the on-board clock in the HK and the frequency–temperature relation obtained from the clock ground calibration. ahtime computes the time for all science data and HK with a three-step procedure. The first step derives the fine time resolution that is assigned internally by the instruments either using support instrument dependent look-up tables or keywords in the header of the FITS file. The second step derives, for a given L32TI, the corresponding time using the TIM table. The third step adds a delay, when appropriate, between the on-board computer and the instruments. For HK, data steps 1 and 3 are skipped because there is no need for fine time resolution. The times are stored in the column TIME, which contains seconds since an epoch set to 2014-01-01 00:00:00 UTC. The epoch is written in the header of all FITS files as an MJD value in the keywords MJDREFI and MJDREFF. The time system is terrestrial time (TT), and the MJD value of the epoch corresponds to MJD 56658.0007775925926 (TT).


Coordinates Assignment

Hitomi has three instruments at the focal plane of the telescopes: the SXI, SXS (soft energy telescopes), and the HXI (high energy telescopes). The coordinates for these instruments are described by five different coordinate systems and their values are stored in the FITS columns named PIXEL/RAWn, ACTn, DETn, FOCn, where n is either X or Y, and the columns X/Y that represent the SKY system21 (see below). Since the SGDs are not imaging detectors, SGD files do not have any of the above columns for coordinates, however, the XYZ position in the detector is provided for any reconstructed event. The transformation of coordinates for all instruments is defined in calibration files, one for each instrument (the telescope definition, teldef). The SXI, SXS, and HXI coordinates are calculated by a common mission independent task coordevt that uses the information from the CAMS to account for the movements of the EOB, where the HXIs are located when transforming from RAW to ACT, the attitude to calculate X and Y, and the teldef for all other coordinate transformations. The characteristics for each instrument, such as pixel size, number of pixels, size of the FoV expressed in the different coordinate systems together with the definition on whether the coordinates are defined as a look-up/down system with respect to the detectors, are listed in Table 2.

Table 2

Instrument coordinate system characteristics.

InstrumentRaw/pixel coordinates (look-down)ACT coordinates (look-down)DET coordinates (look-down)FOC coordinates (look-down)
SizePixel (arc sec)SizePixel (arc sec)SizePixel (arc sec)SizePixel (arc sec)

The RAW coordinates comprise the basic coordinate system. For the SXS, they correspond to the telemetered pixel number where the event landed. This value is stored in the 1-d column PIXEL and ranges from 0 to 35, where one of the pixels (pixel 12) is the calibration source pixel located outside of the array that collects events from the sky. For the SXI, the RAW coordinates correspond to the telemetered event location into a segment of the CCD. Each CCD has two segments and the SXI has four CCDs for a total of eight segments. The RAWY values range from 0 to 639 and the RAWX values from 0 to 319. For the HXI, the RAW coordinates are calculated after the event is reconstructed using the telemetered strip location, and they are affected by the movements of the EOB, where the HXIs are located.

The ACT coordinates are derived from the RAW coordinates. The SXS ACT coordinates represent a 2-d look-down linearized coordinate system starting from the values stored in the PIXEL column. The SXI ACT coordinates correspond to the pixel locations in one SXI CCD. They range from 1 to 640 in the X and Y dimensions. The SXI RAW-to-ACT conversion depends on the window mode and readout node. The HXI ACT coordinates are calculated from the RAW coordinates; the movement of the EOB with respect to the main satellite body is removed using the CAMS time-dependent misalignments.

The DET coordinates are derived from the ACT coordinates. The SXI DET coordinates combine all four CCDs into a single-system frame, accounting for any misalignments among them. The ACT-to-DET conversion changes the configuration from look-down (from sky looking down at the detector) to look-up (from the detector looking up at the sky) for all instruments. Figure 3 shows the DET coordinates for all instruments overlaid.

Fig. 3

SXI look-up view of the DET system with the HXIs and SXS overlaid. The FoV centers of the two HXI sensors and SXS coincide with the SXI pointing position. For each instrument, the origins of the coordinate system are indicated with color-coded circles. The large gray circles represent the placement of the SXI calibration sources.


The FOC system combines all of the HXI, SXI, and SXS individual DET coordinates into a common system with the same pixel size and image size, accounting for small misalignments among them. Since the SXI has the largest FoV and smallest detector pixel size, the FOC coordinates for all sensors adopt the pixel scale and range for the SXI.

The X and Y coordinates (or SKY) are derived from the FOC coordinates using the satellite attitude, and they differ from the FOCX and FOCY according to the position angle or roll. The X and Y columns are in units of pixel. Each pixel value is associated with a position in the sky via the FITS WCS keywords. The SKY system defined by the WCS keywords is a tangent plane projection and oriented such that declination (δ) increases in the +Y direction and right ascension (α) increases in the X direction. Figure 4 shows the in-flight sky images for the G21.5-0.9 source for each individual detector as well as the overlay of the HXIs and SXS onto the SXI frame.

Fig. 4

(a) Each panel shows the image of in an individual detector in sky coordinates. The bright red spot in the SXS image is the calibration pixel. (b) SXI image with contour overlay of the HXI1, HXI2, and SXS. In this image, the SXI has the area discriminator activated so that a small fraction of the FoV is exposed to the x-ray source.



Energy Assignment

The HXI, SGD, SXI, and SXS spectra are derived using the FITS column pulse invariant (PI) that corresponds to the linearized pulse height (channel), where each channel has equal energy width (see Table 3). The PI is calculated with algorithms appropriate for each instrument due to the different physics of the x-ray interaction with the detector. Many intermediate quantities are necessary before computing the PI value, and all tasks for the SXI, SXS, HXI, and SGD record the intermediate values in additional columns of the event file.

Table 3

For each instrument, PI_CHAN gives the PI channel range and PI_BIN gives the size in energy of the channel in eV units.

SXI0-40956 eV
SXS0-327670.5 eV
HXI0-20470.1 keV
SGD0-20470.75 keV

The SXS telemetered “energy” information is recorded in the FITS columns PHA for all event grade types (high, med, and low grade both primary and secondary), where the grade information is stored in the column ITYPE. The PHA of the mid secondary grade events are first corrected by the task “sxssecor” and written in the column PHA2 that otherwise stores the telemetered PHA for primary events. The PI is calculated using “sxsgain” and “sxspha2pi” tasks. sxsgain calculates the SXS time-dependent energy scale correction using a calibration line and the appropriate temperature-dependent gain coefficients. The calibration lines originate either from the calibration pixel that is always operating or from the MXS calibration sources if they are turned on. Using the results of sxsgain and the SXS temperature-dependent gain file, sxspha2pi calculates the PI values and stores them in the PI column. The SXS anticoincidence data are also corrected for gain using a simple linear correction implemented in the task sxsanticopi. The first observation of the Perseus cluster was obtained when the SXS was not yet in thermal equilibrium that was reached on March 4. For the observations prior to that date, the gain is adjusted with an additional pixel-to-pixel correction, derived using an observation subsequent to that of Perseus conducted using an Fe55 source, on a filter wheel, that illuminated all pixels, and applied by the task “sxsperseus.”

The SXI telemetered energy is stored in the “PHAS” column that contains the 3×3  pixel array of telemetered charge around the event center. “sxipi” corrects the PHAS values in the 3×3 array for the video temperature part of the CCD electronics, for the charge trail, and for the charge transfer inefficiency. Using the corrected 3×3 array, sxipi assigns the grade and the pulse height for each event and populates the GRADE and PHA columns. In the last step, the gain is applied to each event and the computed PI value is stored in the PI column.

For the HXI and SGD, the telemetered pulse height corresponds to the signals detected by the instrument. The x-ray photon, when interacting with the detector, may create more than one signal. The ensemble of all signals constitutes an occurrence. Therefore, for one interaction, there are many signals that may contribute to the final PI calculation. At the first step, hxisgdpha gain corrects the PHA of each individual signal, deriving an energy. The PI is calculated using hxievtid and sgdevtid tasks for the HXI and SGD, respectively. These tasks determine whether the signals, due to the interaction of the x-ray with the detector parts, are consistent with a valid event by comparing the detected signals with the allowed template patterns of signals. If the pattern within an occurrence is consistent with an event, hxievtid and sgdevtid assign the PI to the reconstructed event, considering the energies of the signals within the occurrence, and the RAW coordinates.


Postprocessing Software: Data Selection, Effective Area, Response and Background

The data output from the pipeline processing is the starting point for the data analysis, namely the investigation of the timing, spectral and spatial source properties. All event files may be read by the multimission software XSELECT, distributed with HEASoft, to select, bin and calculate spectra, lightcurves and images.

To model the data, spectral analysis programs, such as XSPEC require for each instrument an ancillary response file (ARF) and a redistribution matrix file (RMF), or a RSP file (ARF and RMF together). The RMF, ARF, and/or RSP are calculated by instrument specific software. Figure 5 shows the SXS, SXI, and HXIs Hitomi spectra extracted from a region centered on the G21.5-0.9 supernova remnant together with the best fit model that has been convolved with the appropriate instrument responses. The RMF is a spectral redistribution matrix that includes the “line-spread function” (LSF), whereas the ARF is a multiplicative function of energy and contains all the information about the EA due to the x-ray telescope, quantum efficiency (QE), and a number of other detector-related efficiencies. For the SXS and SXI, there are tools to calculate both the ARF and RMF, but because of the different detector characteristics for the HXI, it is only possible to calculate the net response matrix file (RSP). For the SGD, the on-axis RSP files are provided in CALDB and the software corrects for the off-axis transmission.

Fig. 5

G21.5-09 spectra of the SXI (green), SXS (blue), HXI1 (black), and HXI2 (red) with the best fit model overlaid.


There are also specific tools to calculate the flat field for images as well as to calculate the instrumental background appropriate for the observation.


SXI and SXS ARF and HXI Response Generator

There is a common framework to calculate the telescope EA.12 aharfgen is a script that calls different tasks to calculate the ARF for the SXS and SXI (ahsxtarfgen), and RSP for the HXI (hxirspeffimg). Although the scheme for creating response functions differs among instruments, there are commonalities applicable to the three instruments. (a) The landing position in the focal plane of an event coming from a fixed position in sky varies with the attitude motions. This variation is accounted for by the outputs of the exposure calculation. ahexpmap calculates an histogram of time spent at different off-axis positions and, for the SXI and SXS, also calculates the fractional exposure of each detector pixel for each off-axis bin. (b) The telescope EA is calculated by the ray-tracing code13 (xrtraytrace), which is essentially a Monte Carlo simulation of photons injected into the telescope aperture. The run is made for each off-axis angle using a precalculated energy grid. The output is an event file containing the path of each photon through the telescope and the final impact coordinates on the focal plane. (c) The ray-tracing energy grid is coarser than the final energy grid of the ARF, because it utilizes an optimized precalculated grid with fine energy sampling in regions, where telescope or detector edges dominate and coarse sampling, where no features are expected due to the telescope or the detector. (d) There are four options to account for the spatial distribution of the x-ray source: point source, extended source flat within a circle, extended source described by a beta model, or a spatial distribution from an image obtained by a different mission. Events generated by the ray-tracing are filtered through the source region selection and propagated down to the detector. Any object encountered by the event down to the detector is accounted for at this stage. These are the filters for SXS and SXI and the baffle for the HXI. For the SXI and SXS, the final ARF calculation also includes the QE that is position dependent for the SXI but not for the SXS, the contamination and the exposure information. For the HXI, the code outputs an RSP file because the intrinsic detector characteristics are also time-and-position dependent. The LSF and QE are different for each of the five layers present in a single HXI unit. The HXI is located on the EOB that moves with respect to the main satellite body, where the telescopes are located. The EOB movements are tracked by the CAMS and recorded in the CAMS data. Therefore, the net response matrix is a weighted combination of the five responses. The weights depend on the relative time-and-position dependent x-ray source counts, which in turn depend on the movements of the EOB, with respect to the telescope and on the telescope EA.


SXI and SXS RMF Generator

The RMF for the SXS and SXI is calculated by the sxsmkrmf and sxirmf tasks, respectively.12 The SXS RMF depends on the grade distribution and the pixels selected to create the spectrum. sxsmkrmf first calculates an histogram of grades for the pixels of interest and then calculates the RMF for each pixel and grade, and averages the individual RMF according to the histogram. By default, the SXS RMF input and output grids result in an RMF with 32,768 channels of width 0.5 eV. The RMF for the SXI depends on the extraction region and sxirmf uses the WMAP, a coarse-binned image of the extraction region with the spatial counts distribution, to determine the weight distribution for the LSF from different positions on the detector. By default, an RMF is generated with an input energy grid in the range 0.200 to 23.974 keV, and with 5900 output channels of width 2 eV up to 12  keV and 500 channels up to 24  keV.


Flat Field

The counts images derived from the science events for the SXI, SXS, and HXI may be corrected for telescope and detector effects using the flat field image.

The SXI and SXS flat fields are calculated with ahexpmap by requesting an efficiency map as output. This is the same task that computes the exposure distribution at different off-axis angle necessary for the ARF calculation. The efficiency is calculated using several calibration quantities, the attitude information, and an input energy band. The calibration included in the efficiency calculation for both instruments are the SXT vignetting, the detector QE, and the time-dependent contamination column density. For the SXS only, the effect of the optical blocking filter, the gate valve, and any additional filter on the filter wheel is also considered. For the SXI, the optical blocking is already included in the QE. The contributions from these quantities, for either a single energy or for a range of energies, are multiplied together for each detector pixel considering the time-dependent variation of the attitude.

The HXI hxirspeffimg task calculates either the response or the flat field, the latter is achieved by requesting as output an efficiency map. Similarly to the SXI and SXS, the efficiency for the HXI is calculated using the vignetting from the hard x-ray telescope, the QE for each layer of the detector, the attitude information (output of ahexpmap), and an input energy range. However, for the HXI, the flat field also corrects for the movements of the EOB using the CAMS data. The efficiency map values for all instruments are coded as unitless numbers that range between 0 and 1.


Non-x-ray Background

The background in all Hitomi detectors includes contributions from x-rays that are focused by the telescope optics, referred to as x-ray background (XRB) and nonfocused background components from particles and internal detector effects collectively referred to as the NXB. There are various techniques to account for the XRB and NXB backgrounds in the spectral fitting as well as different methods to correct for the XRB (see also Hitomi analysis guide at Ref. 10). For the HXI, SXI, and SXS, the NXB is instead obtained using products of the pipeline. For each sequence and instrument, the pipeline creates event files, where the events are selected to belong to the night earth data when x-rays are blocked from the detector view. However, to derive spectra with sufficient statistics to characterize the NXB, night earth event data from several observations are combined together over a time period longer than a typical observation and archived together with all necessary support files. To derive the appropriate NXB spectrum for a given observation, additional filtering may be needed. The tasks hxinxbgen, sxinxbgen, and sxsnxbgen allow for the filtering of the NXB for the region of the x-ray source of interest, to select NXB time interval that includes the same geomagnetic cut-off orbital rigidity as the observation, and to perform additional filtering or calculation specific to each detector.



The Hitomi software package and calibration data are in the distribution with the HEASoft package and CALDB both available from HEASARC. There are two software guides to help the users in the data analysis, the Hitomi data reduction guide and the Hitomi step-by-step analysis guide, where the former is a comprehensive summary of the instrument descriptions, the available software, and how the calibration has been applied; and the latter includes actual example commands to recalibrate the data, extract spectra, lightcurve or images and calculate the response. Each software task also has an extensive help to explain the algorithm and meaning of each parameter. The calibration files have associated documents that describe how they have been derived and their limitations. The software and calibration documents together with any updates are posted at Ref. 10.

The Hitomi data archive is populated with the outputs of the pipeline processing of May 2017 that was run with the latest software and calibration. A subset of the sequences was reprocessed on September 2017 to include the final update of the SGD calibration. The data are available from the public archive at the HEASARC in United States and DARTS in Japan. The data were made public on October 31, 2017, following a propriety period. These few datasets have demonstrated the superiority of the nondispersive high-resolution spectroscopy and are an excellent platform for gaining familiarity with data to be obtained by similar instruments planned in the future, such as the x-ray astronomy recovery mission.


We thank the entire Hitomi collaboration team for the support and suggestions received during the preparation of the Hitomi software package in the prelaunch.


1. T. Takahashi et al., “The Astro-H (Hitomi) x-ray astronomy satellite,” J. Astron. Telesc. Instrum. Syst. (2018). Google Scholar

2. R. Kelley et al., “The Astro-H high resolution soft x-ray spectrometer (SXS),” J. Astron. Telesc. Instrum. Syst. (2018). Google Scholar

3. Y. Soong et al., “ASTRO-H soft x-ray telescope (SXT),” Proc. SPIE 8147, 814702 (2014).PSISDG0277-786X http://dx.doi.org/10.1117/12.894305 Google Scholar

4. H. Tsunemi et al., “Soft x-ray imager (SXI) onboard ASTRO-H,” Proc. SPIE 9905, 990510 (2016).PSISDG0277-786X http://dx.doi.org/10.1117/12.2230672 Google Scholar

5. K. Nakazawa et al., “The hard x-ray imager (HXI) onboard Hitomi (ASTRO-H),” J. Astron. Telesc. Instrum. Syst. (2018). Google Scholar

6. H. Awaki et al., “ASTRO-H hard x-ray telescope (HXT),” Proc. SPIE 9144, 914426 (2014).PSISDG0277-786X http://dx.doi.org/10.1117/12.2054633 Google Scholar

7. S. Watanabe et al., “The soft gamma-ray detector (SGD) onboard ASTRO-H,” Proc. SPIE 9905, 990513 (2016).PSISDG0277-786X http://dx.doi.org/10.1117/12.2231962 Google Scholar

8. C. P. de Vries et al., “Calibration sources and filters of the soft x-ray spectrometer instrument on the Hitomi spacecraft,” J. Astron. Telesc. Instrum. Syst. 4(1), 011204 (2017). http://dx.doi.org/10.1117/1.JATIS.4.1.011204 Google Scholar

9. L. Gallo et al., “The Canadian ASTRO-H metrology system,” Proc. SPIE 9144, 914456 (2014).PSISDG0277-786X http://dx.doi.org/10.1117/12.2054921 Google Scholar

12. T. Yaqoob et al., “Spectral response and effective area functions of the Hitomi imaging instruments,” J. Astron. Telesc. Instrum. Syst. (2018). Google Scholar

13. T. Yaqoob et al., “Raytracing for thin-foil x-ray telescopes,” in preparation (2018). Google Scholar

14. M. Loewenstein et al., “Heasim and skyback simulation tools, and their application to the Hitomi mission,” J. Astron. Telesc. Instrum. Syst. (2018). Google Scholar

15. K. A. Arnaud, “XSPEC: the first ten years,” in Astronomical Data Analysis Software and Systems V, , G. H. Jacoby and J. Barnes, Eds., Vol. 101, p. 17, A.S.P. Conference Series (1996). Google Scholar

16. L. Stella, L. Angelini, “XRONOS: a timing analysis software package,” in Astronomical Data Analysis Software and Systems, , D. M. Worrall, C. Biemesderfer and J. Barnes, Eds., Vol. 25, p. 103, A.S.P. Conference Series (1992). Google Scholar

17. P. Giommi et al., “XIMAGE a multi-mission x-ray image analysis package,” in Astronomical Data Analysis Software and Systems, , D. M. Worrall, C. Biemesderfer and J. Barnes, Eds., Vol. 25, p. 100, A.S.P. Conference Series (1992). Google Scholar

20. Y. Terada et al., “The time assignment system and its performance for the Hitomi satellite,” J. Astron. Telesc. Instrum. Syst. 4(1), 011206 (2017). http://dx.doi.org/10.1117/1.JATIS.4.1.011206 Google Scholar

21. H. Krimm et al., “ASTRO-H coordinates definitions,” (2017),  https://heasarc.gsfc.nasa.gov/docs/hitomi/calib/caldb_doc/astroh_SCT_020.pdfGoogle Scholar

Biographies for the authors are not available.

© The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.
Lorella Angelini, Yukikatsu Terada, Michael Dutka, Joseph Eggen, Ilana Harrus, Robert S. Hill, Hans Krimm, Michael Loewenstein, Eric D. Miller, Masayoshi Nobukawa, Kristin Rutkowski, Andrew Sargent, Makoto Sawada, Hiromitsu Takahashi, Hiroya Yamaguchi, Tahir Yaqoob, Michael Witthoeft, "Astro-H/Hitomi data analysis, processing, and archive," Journal of Astronomical Telescopes, Instruments, and Systems 4(1), 011207 (22 January 2018). https://doi.org/10.1117/1.JATIS.4.1.011207 Submission: Received 7 August 2017; Accepted 4 December 2017
Submission: Received 7 August 2017; Accepted 4 December 2017



Data archive systems

Data analysis

X-ray telescopes

Data processing

Content addressable memory


Ground based x ray calibration of the Astro H soft...
Proceedings of SPIE (July 24 2014)
Astro-H data analysis, processing and archive
Proceedings of SPIE (July 15 2016)
Chandra X-ray Observatory mirror effective area
Proceedings of SPIE (February 03 2004)
The Chandra X-ray Observatory data processing system
Proceedings of SPIE (June 30 2006)

Back to Top