CubeSats provide opportunities for science and technology demonstration missions with low-cost solutions and short project timescales, in particular, for studying gamma-ray bursts (GRBs) in the multi-messenger era. A robust operations strategy for scientific CubeSat projects is key to optimizing the results obtained from the experimental instruments. The Educational Irish Research Satellite-1 (EIRSAT-1) is a 2U CubeSat with three payloads, including a bespoke gamma-ray detector, gamma-ray module (GMOD), developed in-house for the detection of GRBs. The detection and reporting of GRB triggers to the scientific community complicates and drives the mission operational strategy. The operational procedures developed for commissioning and operating GMOD are detailed. The successful operation of EIRSAT-1 will facilitate the detection of ∼15 GRBs / year in low Earth orbit. To increase the likelihood of mission success, the project is following a prototype model philosophy, building, and testing, both an engineering qualification model (EQM) and flight model. The EIRSAT-1 operations manual is the document that will instruct operators in commanding the spacecraft correctly and efficiently throughout the mission lifetime. The operations manual must be refined in parallel to payload development. This two-model philosophy has provided time for the early development of the EIRSAT-1 operations manual with the EQM. The EIRSAT-1 operations manual has undergone incremental updates based on feedback from operational development tests (ODTs), and a version with 35 procedures was frozen prior to the month-long EQM mission test (MT). Specifically, the objective of our work is to validate and refine the operations manual using the EQM MT process. Although the ODTs were effective preparation, the MT process highlighted issues, such as procedures operators found convoluted, and scenarios not yet considered during the initial development stages. Two new procedures were identified, 8 procedures required major updates, 15 required minor updates, and the remaining 12 required no improvements after the MT. The validation process facilitated operator training in mission representative conditions, such as GRB triggering data downlink with GMOD, and the major lessons learned during the development and validation process are presented.
CubeSats are small-scale satellites, usually conforming to a design standardization, defined by California Polytechnic State University specifications.1 CubeSat designs are based on units where 1 unit, a 1U, is a cube with a mass . Their popularity is growing exponentially in recent years due to their science and technology demonstration capabilities.2–4 CubeSats were originally developed as an educational tool,5,6 and this led to the production of commercial-off-the-shelf (COTS) components. COTS components can be used for the fundamental subsystems of the satellite, e.g., power, communications, onboard computer (OBC), allowing for more resources to be applied to the science objectives of the mission.7 CubeSat missions must accept higher risk in return for low-cost and fast delivery.
Lower risk acceptance and lack of robust system-level testing have contributed to a high failure rate of CubeSat missions with half of the first 100 CubeSats launched by University teams being unsuccessful.8,9 The Educational Irish Research Satellite-1 (EIRSAT-1) mission is complex with three in-house developed payloads and an antenna deployment mechanism. The project aims to reduce risk and demonstrate a reliable spacecraft (S/C) by following, similar to the Aalto-1 mission,10 a prototype model philosophy. The EIRSAT-1 engineering qualification model (EQM) has been built,11 and the flight model (FM) is currently being tested. Rigorous system level testing has been performed with the EQM being subject to a full functional test,11 mission test (MT),12 and environmental testing13 to qualification levels. The FM will undergo the same suite of testing to acceptance levels.
The primary payload of EIRSAT-1 is the gamma-ray module (GMOD), a scintillation-based detector.14,15 The objective of GMOD is to demonstrate novel technology for the detection of gamma-ray bursts (GRBs) and the operation of the instrument in-orbit, which requires onboard triggering, modification of the beacons when a GRB is detected, and prioritization of data for downlinking. GRBs are violent and short-lived explosions of gamma-ray radiation that are involved in the birth of a black hole.16 GMOD will be used to scale up to a configuration of CubeSat compatible gamma-ray detectors, with localization capabilities, as part of the 6U Gamma-ray Investigation of the Full Transient Sky project, which is also based on in-house instrument heritage.14,15,17–19
In 2017, a breakthrough discovery was made when the electromagnetic (EM) counterpart to a gravitational wave (GW) event was detected (GW170817).20 The coincident detection proved the hypothesis of a progenitor of short GRBs and marked the beginning of a new era of multi-messenger astronomy.21 This ground-breaking observation highlighted the importance of gamma-ray observatories in the multi-messenger era; however, a major challenge to future gamma-ray discoveries is the gap in planned future missions. The current fleet of GRB detecting missions has had remarkable success; however, many of the missions have far exceeded their nominal lifetimes. The Fermi Space Telescope,22,23 Neil Gehrels Swift Observatory,24 and INTEGRAL25 have been in orbit since 2008, 2004, and 2002, respectively. Currently, there is no sequential mission in advanced development from any major space agency, which will likely result in a gap in the technological capabilities of gamma-ray observatories in the time of major GW detector enhancements. CubeSats are relatively low cost and have short launch timescales26 making them ideal candidates to bridge this gap. In addition to the current large-scale missions, a fleet of GRB detecting CubeSats (e.g., BurstCube,27 MoonBEAM,28 HERMES,29 GRBAlpha,30 CAMELOT,31 GRID,32 and GTM33) could allow for adequate detection and localization of GRBs and increased sky coverage, during future GW instrumentation operating runs.
It is a challenge to operate a satellite in the harsh space environment.34,35 The main goal of the primary payload of EIRSAT-1 is to trigger on GRBs in real-time and rapidly report to the scientific community, further complicating the operations strategy of the mission. The EIRSAT-1 project identified that early development of an operations manual and operator training for a student-led team with a lack of experience would increase the likelihood of mission success.36 The EIRSAT-1 operations manual provides operators with the prerequisite knowledge and step-by-step instructions required to efficiently and successfully command the S/C and achieve the science and technology goals. Development began once a full system had been assembled, i.e., the EIRSAT EQM, to allow for rigorous testing of the operations manual.
This paper will focus on the development of the EIRSAT-1 operations manual and the validation during the EQM MT. Section 2 will summarize the EIRSAT-1 mission with a focus on the primary payload, GMOD. An overview of the EIRSAT-1 operations manual will be presented in Sec. 3. Section 4 will detail the operational procedures specific to GMOD, including commissioning, experiment configuration, and downlink of GRB data, which is a key driving factor in the communication operations strategy for the mission. Section 5 will detail the validation of the operations manual via operational development tests (ODTs) and during the EQM MT. Section 6 discusses the major lessons learned by the team during the development of the operations manual and details the operational planning objectives that will be achieved before launch.
Educational Irish Research Satellite-1
EIRSAT-1, presented in Fig. 1, is a 2U CubeSat mission with science, technology demonstration, and education objectives being developed by a student-led team as part of the European Space Agency (ESA) Education Office’s Fly Your Satellite! (FYS) Program.37 As part of the FYS program, the team receives guidance and advice from ESA, and the team can make use of state-of-the-art ESA test facilities. All testing, results, and anomalies are tracked and reviewed by ESA experts. EIRSAT-1 has three novel payloads and an in-house developed antenna deployment module (ADM).38 The objectives of each payload are scientific or to demonstrate novel technology. GMOD, discussed in more detail in Sec. 2.1, will perform real-time detection of the most luminous explosions in the EM universe, GRBs, and in doing so, demonstrate the use of bespoke gamma-ray instrumentation with miniaturized electronic readout.14,15 The ENBIO Module (EMOD) will monitor ENBIO Ltd.’s thermal surface treatments SolarWhite and SolarBlack.39 These thermal coatings are on ESA’s Solar Orbiter mission,40 however, EIRSAT-1, will provide the unique opportunity to monitor their performance via resistance temperature detectors (RTDs). The wave based control (WBC) payload is a software demonstration testbed for a novel attitude control algorithm developed in University College Dublin (UCD).41,42 For communication, EIRSAT-1 makes use of the in-house developed ADM consisting of four coiled antenna elements. The elements are contained by doors held closed by two tensioned meltlines, which are cut by passing a current through one of two burn resistors.43
The other foundational subsystems, the electrical power subsystem, attitude determination and control system (ADCS), transceiver, and OBC are all COTS components procured from AAC Clyde Space. The software on the OBC has been developed to interface with the in-house developed payload firmwares.44 The prototype model approach was selected over building and launching one model (protoflight) as EIRSAT-1 is a complicated mission incorporating three experiments with different objectives and a novel antenna deployment mechanism. The project is being developed, tested, and launched by a student team with no previous experience in operating a S/C. The EQM can also be used during the mission for hands on operator training and as a ground-based testbed for anomaly investigations. Some trade-offs of this approach include longer project timelines and financing the cost of building and testing two models of the satellite. The model philosophy allowed for early development of the operational strategies and the EIRSAT-1 operations manual, which will be vital in ensuring the goals of all payloads are achieved and the mission operates successfully throughout its lifetime.
GMOD and GRB Triggering
The GMOD experiment was designed to further the goal of detecting GRBs in the multimessenger era as introduced in Sec. 1. The EIRSAT-1 operational strategy (Fig. 2) needs to account for the transient nature of GRBs, which makes them unpredictable sources to study.
GMOD consists of two main components, the detector assembly and the GMOD motherboard (MB). In the detector assembly, a tiled array of 16 silicon photomultipliers (SiPMs) is coupled to a scintillation crystal, which interacts with the gamma-ray photons to produce photons of characteristic energies that can be collected by the optical detectors. The outputs from the SiPMs are readout and digitized by an application-specific integrated circuit (ASIC), produced by Integrated Detector Electronics AS, called SiPM Readout ASIC (SIPHRA).45 The GMOD MB supports the electronics required to power and operate the SiPMs and SIPHRA. The firmware on the GMOD MB has been developed in-house46 to interface with the OBC of EIRSAT-1. The GMOD MB converts the channel data into time-tagged events (TTEs). There are TTEs generated for the 16 individual channels and for the sum channel (sum of 16 channels). Both sum and 16 channel TTEs are sent in pages to OBC, which stores them until a downlink of the data is requested from the ground station.
The OBC will use the GMOD TTE data to periodically generate lightcurves and spectra and store these in the GMOD lightcurve and GMOD spectrum storage channels for future downlink.47 GMOD trigger generation is calculated based on a rolling average of counts in a specified energy range, the average count is compared to a predefined trigger significance level. If the significance level of the signal exceeds the trigger level, TTEs from a specified time period before and after the trigger time are copied and stored in a specific storage channel by the OBC.15 Due to communication constraints, it is not feasible to downlink all the data generated on board by GMOD. Priority will be given to the protected TTE data associated with triggers for GRB candidates. After a GRB triggers the on-board software (OBSW), the satellite will emit beacons containing basic information about the trigger (e.g., duration, start time) and low-resolution spectra and lightcurves until the next communication pass with the EIRSAT-1 ground station.48 The subsequent communication passes will be used to downlink the higher resolution protected TTEs around the time of the trigger for analysis, classification, and reporting to the community. The EIRSAT-1 operations manual needs to prepare operators to optimize communication with the satellite. In a communication pass of to 7 min, the operators will need to confirm the health of the S/C and downlink science data from any or all three experiments depending on the priority of the data to downlink, i.e., GMOD GRB data has a higher priority than EMOD RTD readings.
EIRSAT-1 Operations Manual
The EIRSAT-1 operations manual is generated using Sphinx49 with the “read the docs” theme50 to produce user-friendly HTML and PDF documents compiled from a set of reStructuredText (RST) markup files. The manual has two major components, the first section introduces the operators to the tools they will be using to interface with the S/C and the knowledge that will be required for successful operation. The second section contains the procedures required to operate EIRSAT-1 in nominal scenarios and procedures to guide operators in the case of an anomaly, try to determine the root cause, and how best to recover the mission.
Mission Control Software
The main interface with the S/C is currently via Bright Ascension’s Mission Control Software (MCS).51 At the start of 2022, the project was selected for the Bright Ascension Bright Start Academic Program and received a year’s sponsorship with their flight and ground software. The month-long EQM MT and previous testing were performed using an older version of the MCS with fewer features. Due to the timing of the sponsorship, the updates to the operations manual based on feedback and lessons learned from the test campaign, discussed in more detail in Sec. 5.2, and the updates for interacting with the new MCS could be implemented simultaneously. The MCS is similar to the previous interface with the S/C with an overview of the interface initially presented and then the main improvements outlined.
The operator loads a spacecraft database (SCDB) into the MCS, which is used to interact with the parameters and actions of the onboard software. Parameters can be set, obtained, queried, downlinked from, or uplinked to, and actions can be invoked. These terms are the building blocks of the interface to the S/C and are defined in Table 1.
Summary of main components used to operate and command EIRSAT-1 via the MCS.
The MCS has a number of components, with different roles, that can be displayed. Only the ones used by the EIRSAT-1 project to date will be discussed here. The default EIRSAT-1 layout of the S/C interface, before operator customization for a specific procedure, is presented in Fig. 3.
The panel on the left is the mission explorer containing a drop-down of the SCDB and connection settings to the S/C. The packet monitor window (surrounded by a green box) in the center displays the telemetry (TM) and telecommands (TCs) exchanged between the MCS and the S/C. The transfer window (red) is located on the right and displays the percentage complete of data being downlinked from/uplinked to the S/C. The transfer window can be used to abort, suspend, and resume any transfer of data between the MCS and the S/C. At the bottom of the layout is the System/Event/Debug console (blue). The System tab displays logs from the MCS itself, e.g., failing to connect to the S/C. The Event tab displays any events from the OBSW, e.g., transition to safe mode. The debug log shows debug information from the OBSW.
In addition to these components, parameters and actions can be opened and positioned on the console via the SCDB and used to interact with the S/C, with an example of each in Fig. 3. Layouts were created for different operational procedures during the MT as it prevented operators from searching through the SCDB for a specific parameter during a communication pass. The first section of the EIRSAT-1 operations manual contains a comprehensive description of the aforementioned components of the MCS, how to load and change the SCDB, and how to set up communication via the MCS.
The EIRSAT-1 operations manual required a revision after transitioning to the MCS, the major updates to the introductory sections were to describe the new features available on the MCS summarized below:
Prerequisite Knowledge for Operators
In addition to describing the interface with the spacecraft, the operations manual contains details of all the knowledge that operators are required to have before commanding EIRSAT-1. This Prerequisite Knowledge section of the operations manual is an outcome of the ODTs, discussed in Sec. 5.1. Any background knowledge that applies to most, if not all, of the operational procedures is described. An in-depth description of the contents of this section can be found in Ref. 36, but the following points summarize the main topics:
The operations manual currently has 37 manual operations procedures for nominal and anomalous mission scenarios. The operational procedures can be split into four groups: early mission, nominal, payload, and fault analysis. Examples of each procedure category are presented, and the full list used in the EQM MT can be found in Ref. 36:
A more detailed description of the objectives of these example procedures is presented in Sec. 5.2. In summary, early mission procedures will only be required in the initial phases of the mission, such as making first contact with the S/C after launch and checking the functionality of the subsystems and payloads in the commissioning phase. Nominal procedures will be used on a daily basis by operators, some will be used during every pass. For example, Start A Communication Pass will be performed every time contact is made with the S/C to determine its current state and if any new GRBs have been detected. Nominal operations will also include the downlinking of housekeeping data to assess the health of the S/C and science data from the three experimental payloads. There are procedures specific to the configuration and experiment running of the two hardware payloads, EMOD and GMOD. The development of the WBC procedures will be part of the future work updates in preparation for the flight-ready operations manual.
The most challenging operational procedures to develop were the fault analysis procedures. Seminars were held to discuss the possible points of failure in the mission. These seminars resulted in a list of off-nominal operational procedures, which can be split into two categories—procedures to determine the cause of entering an unexpected mode or OBC image (e.g., Safe Mode Entered, Failsafe Entered) and procedures to determine the root cause(s) of the anomalous behavior and how to recover the mission (e.g., Low Battery Fault Analysis, S/C Reboot Fault Analysis). Ref. 36 contains a detailed example of a procedure from each of the four categories. This paper provides more detail on the GMOD operational procedures (Sec. 4) and describes the operational manual validation methods employed by the team (Sec. 5).
GMOD Operational Procedures
As the detection and generation of gamma-ray trigger alerts are key to the science output of EIRSAT-1, the operation of GMOD is discussed in more detail in this section. The operations are split into sections of commissioning, experiment configuration, and data downlink, with a focus on GRB data acquisition. These procedures were developed by the collaboration of the EIRSAT-1 operators and GMOD team members. All the procedures were subject to at least one ODT (Sec. 5.1) in their development and were tested during the EQM MT (Sec. 5.2). Feedback contributing to the improvement of these procedures is discussed. The GMOD procedures are also included in the MT validation approach discussed in Sec. 5.2.
Initially, GMOD was to be assigned one operational procedure during the commissioning phase of the mission. However, the ODTs exhibited that breaking the GMOD commissioning into two sections would be more beneficial. Once the core subsystem health checks (battery and ADCS) are completed, a GMOD health check will be performed to assess the basic functionality after launch. The second subset of commissioning procedures involves setting up the S/C for more routine operations. These include a GMOD operations procedure, which will set the experiment running in-orbit for a short duration of time to verify functionality. Both GMOD commissioning phase procedures are described in detail in Fig. 5. The MT verified this approach to commissioning GMOD, and the updates that are required for the flight operations of GMOD are based on advancements made in the firmware.
Experiment Configuration and Running
The GMOD configuration and GMOD experiment running operational procedures were developed and adapted from the commissioning procedures. The GMOD configuration procedure guides the operator through the different parameters and actions that can be used to configure the GMOD payload. There are eight sub-procedures that can be carried out individually to assess and update the GMOD experiment parameters. The sub-procedures contain the basic configuration instructions, e.g., to turn the experiment on/off, get and change the operational mode, and set voltage and current values. It also contains sections to customize the lightcurve and spectrum parameters (e.g., bin width, integration time) and set up and enable GRB triggering.
The GMOD experiment running procedure consists of items detailed in points A, B, C, and F in the outline of the GMOD commissioning operations procedure in Fig. 5. GMOD will primarily run with just sum channel data being generated and streamed to the OBC and the OBC triggers on sum channel data. There is a separate operational mode for the generation and stream of 16 channel data from GMOD to the OBC, this will be performed as a technical demonstration but will not be part of the daily operations of the GMOD experiment.
GRB Data Downlink
The scientific objective of the GMOD experiment is to detect GRBs and report triggers to the high-energy astrophysics community. This requires a complex operational strategy as these short-lived (approximately seconds long), and isotropic events need to be reacted to in real time. This operational requirement is unlike many other science-based observatories running on scheduled observations of known sources.
The OBSW includes a GRB trigger counter in the satellite beacon whose format will be made publicly accessible. This is to facilitate large communities of ground stations, e.g., SatNOGS54 in decoding the beacons and informing the team of new GRB data. The S/C will beacon basic trigger information, compressed lightcurves, and spectra until it communicates with the ground station. Once a trigger has been identified, the following communication passes will prioritize downlinking the protected higher resolution TTE data around the time of the event for ground-based analysis. This analysis will include confirming if the event was a GRB and reporting the observation to the community. As shown in Fig. 6, knowing about a new trigger before a communication pass will increase the efficiency with which the team can respond to GRBs. If the amateur radio community can inform the operators that a new trigger has been detected (grey panel on left of Fig. 6), the next pass plan can include downlinking GRB data. However, if the operators discover the GRB trigger counter has increased during a pass, the downlink of the data will be included in the subsequent pass plan but this delays initiating data acquisition by approximately hours, i.e., the right-hand side of Fig. 6 will need to be followed before proceeding to the downlink of GRB data (left-hand side) during the next communication pass.
It will take multiple passes to downlink the data from a GRB, so commencing the downlink rapidly is vital. The downlinking of GRBs will make use of the suspend/resume feature of data transfers in the MCS. At loss of signal, due to the S/C passing beyond the TM/TC horizon, the MCS will automatically suspend the transfer of GRB data, this can then be resumed by the operator during the next pass(es) until all the data have been successfully downlinked. At the start of each communication pass, the most recent housekeeping data will be downlinked before resuming any GRB data transfer to monitor the current health of the S/C.
Once that GRB has been successfully downlinked, the storage channel needs to be wiped manually by an operator to free the storage channel for future GRB trigger data. The OBC has 40 storage channels allocated to GRB data. There will be a trigger log on the ground that operators will be responsible for populating when data have been downlinked, from which storage channel and if the channel was wiped successfully. Once data for a GRB have been successfully transferred from the satellite, operators will need to determine the next highest priority task, which could include downlinking the next GRB, some RTD data from the EMOD experiment, or running the WBC experiment.
Operations Manual Validation
The EIRSAT-1 operations manual has undergone numerous iterations, and has been subject to rigorous testing. Examples of the procedures for the GMOD instrument were given in the previous section, and a similar process was followed for the commissioning and operation of the other subsystems and payloads on the S/C. Figure 7 outlines the approach taken in developing and verifying the operational procedures for all possible contingencies.
Initially, possible nominal and off-nominal mission failure scenarios were conceived and discussed. The accumulated list of operational procedures to be developed and tested were assigned to a member of the operations team, and a first draft was developed. The first procedures outlined the step-by-step instructions for an operator to command the S/C using MCS (Sec. 3.1) to achieve the operational objective, e.g., GMOD experiment configuration. The prerequisite knowledge topics (Sec. 3.2) were also discussed and assigned for an initial draft. Prior to any testing, operators were required review and be familiar with its contents, as the remainder of the procedures assume this fundamental knowledge of MCS and the S/C.
These first drafts were reviewed by the operations team and then subject to an operations development test, as described in Sec. 5.1. Feedback from the ODTs was used to improve the procedures, and for more complicated procedures, multiple ODTs were performed. In August 2021, the month-long MT was conducted, and a version of the EIRSAT-1 operations manual was frozen for this test campaign. The MT provided the opportunity to extensively test the procedures that had been written to date and also revealed procedures that needed to be created. An overview of the MT, the feedback on the operations manual, and the main lessons learned by the team will be presented in subsequent sections. The team is currently preparing the FM version of the operations manual, implementing feedback from the MT and updates based on the new MCS interface.
Operations Development Tests
ODTs were used to initially test the procedures individually or in small subgroups (e.g., Upload Payload Image and Rewrite Payload Firmware). Once an initial draft of a procedure was prepared and reviewed, an ODT was organized. Members of the team not involved in the development of that procedure would follow it with the EQM and provide feedback. Any procedures that required large updates were subject to more than one ODT.
The ODTs revealed the importance of initiating operations development once a full system configuration is accessible. Missing scenarios, procedures, and steps were all revealed in the iterative process of testing the individual operational procedures. ODTs were essential in introducing the team to the operations manual and familiarizing them with the prerequisite knowledge and format of the procedures. ODTs also facilitated all team members interacting with the S/C via an interface similar to the MCS. This S/C interface was used during the ODTs, as at that point, the project did not have access to Bright Ascension’s MCS. This predecessor to the MCS had the same major components but not as many user-friendly features, as discussed in Sec. 3.1. The ODTs were the initial stage of operator training of the EIRSAT-1 team.
The ODT approach also had its limitations, the tests were not conducted in a flight-like configuration to complete the first assessment of the procedures efficiently and with the available resources, i.e., the full operational chain had not been developed, and the analysis tools for processing the downlinked data had not been completed. In flight, there will be constricted communication pass times with approximately minutes to complete procedures. Communication pass times were not implemented during ODTs as the main objective was to improve the step-by-step instructions. Without time pressure, the operators could progress through the procedure, pause for feedback and discussions, and there was time to note the required updates for the operations team to implement. For efficiency, the ODTs were performed using serial communication with the S/C. In flight, all communication will be over radio frequency (RF), increasing the time taken to receive responses and data from the satellite. The EQM MT was a more rigorous assessment of the EIRSAT-1 operations manual, and Sec. 6 outlines the major differences between validating the EIRSAT-1 operations manual using the ODTs and MT.
Engineering Qualification Model Mission Test
The EQM MT was conducted during August 2021, and a full description of the approach taken to MT can be found in Ref. 12. The MT is a long-duration test campaign simulating flight-like scenarios, including early mission, nominal, and anomalous scenarios. The MT was designed to run through as many scenarios as feasible in 4 weeks and was therefore the logical advanced test of the EIRSAT-1 operations manual. The first week of the MT simulated initial AOS and commissioning of the subsystems and payloads. The remaining 3 weeks tested nominal operational procedures for EIRSAT-1, experiment running procedures for GMOD and EMOD (WBC was not advanced enough to be included in the EQM MT), and off-nominal scenarios, such as low battery events and unexpected S/C reboots.
Throughout the MT, the spacecraft operators could only interact with the S/C during simulated communication passes of to 5 min. There were 4 to 5 passes per day, 7 days a week. Test support operators monitored the test ground support equipment (GSE) and manipulated the S/C to inject non-nominal scenarios, e.g., turning off S/C charging and allowing the battery to fall below a certain threshold to trigger entry into safe mode. The MT was utilized for operator training, and the team was exclusively self-taught. Prior to the start of the test campaign, seminars were run for the team by the lead operations engineers to provide an overview of the EIRSAT-1 operations manual and the operator tools, e.g., MCS and Grafana. The manual was provided to the team in advance of the MT to allow time for them to become familiar with the prerequisite knowledge sections and the procedures. Two operators are always required to command EIRSAT-1, the lead operator commands the S/C via MCS while the supporting operator reads the instructions in the manual. During the EQM MT, there was always one experienced operator present for each communication pass but as the MT progressed, the operators in training took on the lead operator role. For the FM MT, the same approach was taken, where less experienced operators began with the supporting role, then as their confidence and competence in commanding the S/C increased, they were given the responsibility of being lead operator.
The majority of the operational procedures were tested during the MT, and the test campaign revealed two new procedures that needed to be developed around downlinking data in atypical scenarios. Of the 35 procedures tested during the MT, 12 required no updates due to sufficient development in the ODTs, 15 required minor updates, and 8 required major updates. Minor updates included typos or small clarifications in a step of the procedure, whereas major updates incorporated a reformatting of the procedure or adding multiple steps/sections. The procedure updates made from the EQM MT were validated during the FM MT. A frozen version of all of the procedures validated during the FM MT will comprise the EIRSAT-1 operations manual used during flight. Any new procedures or improvements required after the FM MT can be validated with the EQM in “mini-MTs,” which are 1-day MTs focusing on a specific scenario or procedure. Table 2 provides examples from each of the operational procedure categories, the number of ODTs that were conducted, and whether updates were required based on feedback from the MT.
Examples of each of the EIR_OPS procedure categories (early operations, nominal, payload, and fault analysis) are presented. For each example procedure, the number of ODTs that were included is given in the third column. The final column states the updates required based on EQM MT feedback, with minor updates including typos, adding single steps in, or clarifying a confusing step. Major updates required multiple step changes, addition of new sections, or re-visiting the entire structure of the procedure. The minor* procedures require updates based on advanced FM firmware, rather than issues revealed by the MT.
All of the early mission scenarios were subject to at least one ODT. Initial AOS required minor updates because during ODTs, the S/C was always ready for communication. During the MT, the operators had a time window where AOS was expected; only during the MT, it was realized that the operations manual only told operators to send the first command once. In reality, the first command should be sent frequently at the expected AOS time. The commissioning procedures, while undergoing ODTs, had never been completed in sequence for all subsystems and payloads. During the MT, it was found that the sequence of testing was not optimal and has been reorganized for the FM operations manual. The GMOD commissioning procedures only required minor updates related to improvements made to the firmware for the flight model.
The Start a Communication Pass procedure had the same issue as Initial AOS and required only a minor update to send the first command frequently around the expected AOS time until the S/C responds. The Operational Mode Change procedure never underwent an ODT, but the software team followed it regularly during testing and no issues were found during the MT.
The First Time Nominal Mode procedure was one of two procedures that were never subject to an ODT due to time and resource constraints. Each step is linked to a different procedure that could be followed to transition to nominal mode, set up EMOD, set up GMOD, or perform an ADCS health check. During the MT, operators found this procedure very confusing and hard to navigate leading to a re-evaluation of the procedure format. This procedure highlighted the importance of the ODTs in determining the optimal format and content for operational procedures that are both functional and user-friendly.
None of the payload procedures required major updates, which is likely due to extensive testing independent of the MT. All underwent at least one ODT and during payload functional testing, the operational procedures were regularly used to configure and run the experiments providing further validation of the steps. The MT did reveal that the payload procedures could make use of more informative steps for operators not familiar with the experiment. The GMOD configuration and experiment set up procedures are currently being updated based on feedback from the MTs and revision of the firmware on the GMOD MB in preparation for the FM test campaigns.
The fault analysis and recovery procedures all were subject to at least two ODTs to allow the majority of the team to operate during an anomalous scenario and provide feedback on all the scenarios considered for determining the root cause(s). This rigorous testing and involvement of the majority of the team proved beneficial as the procedures performed well during the MT and required only minor updates.
The EQM MT facilitated a long-duration test of the majority of the operational procedures required in flight-like scenarios for the EIRSAT-1 mission. In CubeSat projects, operations are often only developed late into the project due to time and resource constraints. The approach taken in developing the operations manual with the EQM highlights the importance of early development of the mission operational strategy as testing can be incorporated into other necessary test campaigns, such as the MT.
Developing and validating all the operational procedures for a mission with multiple payloads is a challenge. GMOD complicates and drives the operational strategy of the EIRSAT-1 mission with its requirements to detect GRBs, which are short-lived, unpredictable events. The operational procedures developed for GMOD are presented to provide insight into the considerations required for operating a GRB detecting instrument on a CubeSat mission with multiple scientific and technology demonstration goals. The EIRSAT-1 project made use of two forms of testing the operations manual, ODTs, and the EQM MT, to ensure the procedures could reliably instruct operators in nominal operations of the mission, such as GRB data downlink, and recovering from fault scenarios. The MT provided a comprehensive test of the EIRSAT-1 operations manual that could not have been performed via ODTs. Both the ODTs and EQM MT were important milestones in the testing and validation of the EIRSAT-1 operations manual.
The ODTs allowed for incremental development of the operational procedures from an initial version and each ODT focused on only one or two procedures. The EQM MT tested a well-developed frozen version of the operations manual and provided the opportunity to test the full suite of operational procedures over an extended period of time. Operators were aware of the procedures they would be following during initial ODTs, making it unrealistic from an operations perspective. However, during the MT, spacecraft operators only interacted with the satellite during communication passes and would only find out the state of the S/C (e.g., new GRB trigger or the S/C transitioned into safe mode) once contact had been made. This is much more realistic and an important aspect of operator training. The MT assessed how operators handled navigating between procedures in the operations manual. In the ODT, only one procedure would need to be opened, whereas the MT revealed that links to related or useful procedures could be useful but in moderation, e.g., first time nominal procedure was just composed of links to other procedures to set up GMOD and EMOD, and operators found this challenging to navigate. Other operational tools for data visualization, e.g., Grafana, were not developed during the ODTs. The MT facilitated observing how operators interacted with the different tools, such as the MCS (Sec. 3.1), EIRSAT-1 operations manual, and Grafana, simultaneously during and after communication passes to assess the health of the S/C and its experiments. The operational manual updates required after the MT were mostly minor due to the incremental development performed during the ODTs and regular feedback from the entire CubeSat team. The ODTs revealed the preference to split the GMOD commissioning procedures into a health check and experiment running procedure. The MT facilitated the development of the GRB data downlink procedure and training operators in reacting to new triggers and planning passes based on a new potential GRB triggering the GMOD instrument. The major lessons learned in developing and testing a robust operations manual are as follows:
Currently, the FM of EIRSAT-1 is undergoing its suite of tests in ambient and space representative conditions. Once these are completed successfully, the FM will be accepted as flight-ready. The two major updates to make to the operational manual before it is flight ready is to include information on scheduled operations via the MCS and to develop the WBC operational procedures. The EIRSAT-1 FM will be launched and operated by students in UCD, detecting and reporting cosmic gamma-ray events to the astronomical community using GMOD. Science data from the thermal surface treatment experiment, EMOD, and the attitude control testbed, WBC, will also be collected. The EIRSAT-1 mission will share the operational lessons learned during the early phases of the mission and reflect on how early development and rigorous testing of the operational procedures contributed to the success of the project in achieving its scientific and technology demonstration objectives.
The EIRSAT-1 project is employing a number of strategies to increase the likelihood of mission success. CubeSat projects are associated with low cost and short timelines, with a trade-off in that there is a higher level of risk acceptance and a significant rate of failure. The prototype model philosophy executed by EIRSAT-1 provided time for early development of the operational strategy and manual. The operational procedures of EIRSAT-1 must balance monitoring the health of the S/C and running three experimental payloads. The primary payload GMOD makes the operational strategy more complex with its main objective of detecting violent EM explosions, GRBs, and efficiently reporting them to the scientific community. In this paper, the rigorous validation processes of the operations manual at both a procedure level (ODTs) and during a long-duration flight representative test campaign (MT) are presented. Both ODTs and the MT highlighted the importance of early development of the operations manual. The training of operators on an educational team was easily incorporated into the testing of the operations manual. All members of the EIRSAT-1 team are now proficient in navigating the operations manual while communicating with the S/C and using it during nominal and anomalous mission scenarios. Ultimately, the final test of the EIRSAT-1 operations manual will be after launch; however, the testing approach taken has provided a high level of confidence in the EIRSAT-1 operations manual.
The EIRSAT-1 project is carried out with the support of ESA’s Education Office under the Fly Your Satellite! 2 program. The authors acknowledge the guidance from Jean-Philippe Halain of the ESA PRODEX Office. The authors acknowledge all students who have contributed to EIRSAT-1 and support from Parameter Space Ltd. R.D., M.D., D.M., J.T., and L.C. acknowledge support from the Irish Research Council (IRC) (Grant Nos. GOIPG/2019/2033, GOIP/2018/2564, GOIPG/2014/453, GOIPG/2014/684, and GOIPG/2022/1008), respectively. J.R., C.M., and A.E. acknowledge scholarships from the UCD School of Physics. G.F. acknowledges support from a scholarship associated with the UCD Ad Astra fellowship program. B.S. acknowledges a scholarship from the UCD School of Mechanical and Materials Engineering. S.M., D.M., A.U., and J.M. acknowledge support from Science Foundation Ireland (Grant No. 17/CDA/4723). C.B. and S.M. acknowledge support from Science Foundation Ireland (Grant No. 21/FFP-A/9043). J.F. acknowledges support from the European Space Agency (PRODEX) (Grant No. 4000138314). L.H. acknowledges support from SFI (Grant No. 19/FFP/6777) and support from the EU H2020 AHEAD2020 project (Grant No. 871158). This study was supported by The European Space Agency’s Science Program (Grant No. 4000104771/11/NL/CBi) and by the European Space Agency’s PRODEX Program (Grant No. C 4000124425). This work is based on content previously reported at the SPIE 2022 Observatory Operations: Strategies, Processes, and Systems IX conference, and will appear in the conference proceedings. The authors have no relevant financial interests in the manuscript and no other potential conflicts of interest to disclose.
https://doi.org/10.5821/conference-9788419184405.079 Google Scholar
https://brightascension.com/products/mission-control-software/ Google Scholar
Rachel Dunwoody is a current PhD student in the Space Science Research Group at the University College Dublin (UCD). Her research focuses on the development and testing of gamma-ray instrumentation for the study of GRBs in the multi-messenger era. She contributes to the GMOD (gamma-ray detector payload) and operations teams of EIRSAT-1, a 2U CubeSat being built in UCD. She received her BSc degree in physics with astronomy and space science from UCD in 2018 and her master’s degree in Dynamics and Control Systems Group at UCD, where she worked on the attitude determination and control system for EIRSAT-1. She is a student member of SPIE.