WFIRST-AFTA Coronagraph Science Yield Modeling with EXOSIMS

We present and discuss the design details of an extensible, modular, open source software framework called EXOSIMS, which creates end-to-end simulations of space-based exoplanet imaging missions. We motivate the development and baseline implementation of the component parts of this software with models of the WFIRST-AFTA coronagraph, and present initial results of mission simulations for various iterations of the WFIRST-AFTA coronagraph design. We present and discuss two sets of simulations: The first compares the science yield of completely different instruments in the form of early competing coronagraph designs for WFIRST-AFTA. The second set of simulations evaluates the effects of different operating assumptions, specifically the assumed post-processing capabilities and telescope vibration levels. We discuss how these results can guide further instrument development and the expected evolution of science yields.

as complicated as designing the mission. While each component of the system is modeled in great detail as it proceeds through its design iterations, fitting these models together is very challenging.
Making statements about expected science returns over the course of the whole mission requires a large number of often unstated assumptions when such results are presented. This makes it challenging to compare science simulation results and also to systematically test the effects of changing just one part of the mission or instrument design from different groups.
We seek to address this problem with the introduction of a new modular, open source mission simulation tool called EXOSIMS (Exoplanet Open-Source Imaging Mission Simulator). This software is specifically designed to allow for systematic exploration of exoplanet imaging mission science yields. The software framework makes it simple to change the modeling of just one aspect of the instrument, observatory, or overall mission design. At the same time, this framework allows for rapid prototyping of completely new mission concepts by reusing pieces of previously implemented models from other mission simulations.
Modeling the science yield of an exoplanet imager is primarily difficult because it is completely conditional on the true distributions of planet orbital and physical parameters, of which we so far have only partial estimates. This makes the mission model an inherently probabilistic one, which reports posterior distributions of outcomes conditioned on some selected priors. Since the introduction of observational completeness by Robert Brown, 1 it is common to approach exoplanet mission modeling with Monte Carlo methods. Various groups have pursued such modeling, often focusing on specific aspects of the overall mission or observation modeling. [2][3][4][5] A second challenge is correctly including all of the dynamic and stochastic aspects of such a mission. Given a spacecraft orbit, a target list, and the constraints of the imaging instrument, we can always predict when targets will be observable. Incorporating this knowledge into a sim-ulation, however, can be challenging if a single calculated value represents the predictions, i.e., the number of planets discovered. Similarly, while it is simple to write down the probability of detecting a planet upon the first observation of a star, it is more challenging to do the same for a second observation an arbitrary amount of time later, without resorting to numerical simulation. 2 EXOSIMS deals with these challenges by explicitly simulating every aspect of the mission and producing a complete timeline of simulated observations including the specific targets observed at specific times in the mission and recording the simulated outcomes of these observations. While one such simulation does not answer the question of expected mission science yield, an ensemble of many thousands of such simulations gives the data for the posterior distributions of science yield metrics. EXOSIMS is designed to generate these ensembles and provide the tools to analyze them, while allowing the user to model any aspect of the mission as detailed as desired.
In §2 we provide an overview of the software framework and some details on its component parts. As the software is intended to be highly reconfigurable, we focus on the operational aspects of the code rather than implementation details. We use the coronagraphic instrument currently being developed for WFIRST-AFTA as a motivating example for specific implementations of the code. In §3 we present mission simulation results for various iterations of the WFIRST-AFTA coronagraph designs using components that are being adapted to build the final implementation of EXOSIMS.
EXOSIMS is currently being developed as part of a WFIRST Preparatory Science investigation, with initial implementation targeted at WFIRST-AFTA. This development includes the definition of a strict interface control, along with corresponding prototypes and class definitions for each of the modules described below. The interface control document and as-built documentation are both available for public review and comment at https://github.com/dsavransky/EXOSIMS. Ini-tial code release is targeted for Fall 2015, with an alpha release in February of 2016 and continued updates through 2017.
Future development of EXOSIMS is intended to be a community-driven project, and all software related to the base module definitions and simulation execution will be made publicly available alongside the interface control documentation to allow mission planners and instrument designers to quickly write their own modules and drop them directly into the code without additional modifications made elsewhere. We fully expect that EXOSIMS will be highly useful for ensuring the achievement of the WFIRST-AFTA science goals, and will be of use to the design and planning of future exoplanet imaging missions.

EXOSIMS Description
EXOSIMS builds upon previous frameworks described in Ref. 3 and Ref. 6, but will be significantly more flexible than these earlier efforts, allowing for seamless integration of independent software modules, each of which performs its own well-defined tasks, into a unified mission simulation. This will allow the wider exoplanet community to quickly test the effects of changing a single set of assumptions (for example, the specific model of planet spectra, or a set of mission operating rules) on the overall science yield of the mission, by only updating one part of the simulation code rather than rewriting the entire simulation framework.
The terminology used to describe the software implementation is loosely based on the objectoriented framework upon which EXOSIMS is built. The term module can refer to either the object class prototype representing the abstracted functionality of one piece of the software, or to an implementation of this object class which inherits the attributes of the prototype, or to an instance of this object class. Thus, when we speak of input/output definitions of modules, we are referring to the class prototype. When we discuss implemented modules, we mean the inherited class definition. Finally, when we speak of passing modules (or their outputs), we mean the instantiation of the inherited object class being used in a given simulation. Relying on strict inheritance for all implemented module classes provides an automated error and consistency-checking mechanism, as we can always compare the outputs of a given object instance to the outputs of the prototype.
This means that it is trivial to pre-check whether a given module implementation will work with the larger framework, and thus allows for the flexibility and adaptability described above.  The simulation modules take the information contained in the input modules and perform mission simulation tasks. Any module may perform any number or kind of calculations using any or all of the input parameters provided. They are only constrained by their input and output specification, which is designed to be as flexible as possible, while limiting unnecessary data passing to speed up execution.

Input Modules
The specific mission design under investigation determines the functionality of each of the input modules, but the inputs and outputs of each are always the same (in terms of data type and what the variables represent). These modules encode and/or generate all of the information necessary to perform mission simulations. Here we briefly describe the functionality and major tasks for each of the input modules.

Optical System Description
The Optical System Description module contains all of the necessary information to describe the effects of the telescope and starlight suppression system on the target star and planet wavefronts.
This requires encoding the design of both the telescope optics and the specific starlight suppression system, whether it be an internal coronagraph or an external occulter. The encoding can be achieved by specifying Point Spread Functions (PSF) for on-and off-axis sources, along with (potentially angular separation-dependent) contrast and throughput definitions. At the opposite level of complexity, the encoded portions of this module may be a description of all of the optical elements between the telescope aperture and the imaging detector, along with a method of propagating an input wavefront to the final image plane. Intermediate implementations can include partial propagations, or collections of static PSFs representing the contributions of various system elements. The encoding of the optical train will allow for the extraction of specific bulk parameters including the instrument inner working angle (IWA), outer working angle (OWA), and mean and max contrast and throughput.
If the starlight suppression system includes active wavefront control, i.e., via one or more deformable mirrors (DM), 7 then this module must also encode information about the sensing and control mechanisms. Again, this can be achieved by simply encoding a static targeted DM shape, or by dynamically calculating DM settings for specific targets via simulated phase retrieval. As wavefront control residuals may be a significant source of error in the final contrast budget, it is vitally important to include the effects of this part of the optical train.
The optical system description can optionally include stochastic and systematic wavefronterror generating components. Again, there is a wide range of possible encodings and complexities.
They could be Gaussian errors on the contrast curves sampled during survey simulation to add a random element to the achieved contrast on each target. Alternatively, in cases where an active wavefront control system is modeled, stochastic wavefront errors could be introduced by simulating the measurement noise on the wavefront sensor (either again as drawn from pre-determined distributions, or additively from various detector and astrophysical noise sources). Systematic errors, such as mis-calibration of deformable mirrors, closed-loop control delays, and non-common path errors, may be included to investigate their effects on contrast or optical system overhead. In cases where the optical system is represented by collections of static PSFs, these effects must be included in the diffractive modeling that takes place before executing the simulation. For external occulters, we draw on the large body of work on the effects of occulter shape and positioning errors on the achieved contrast, as in Ref. 8.
Finally, the optical system description must also include a description of the science instrument or instruments. The baseline instrument is assumed to be an imaging spectrometer, but pure imagers and spectrometers are also supported. Each instrument encoding must provide its spatial and wavelength coverage and sampling. Detector details such as read noise, dark current, and quantum efficiency must be provided, along with more specific quantities such as clock induced charge for electron multiplying CCDs. 9 Optionally, this portion of the module may include descriptions of specific readout modes, i.e., in cases where Fowler sampling 10 or other noise-reducing techniques are employed. In cases where multiple science instruments are defined, they are given enumerated indices in the specification, and the Survey Simulation module must be implemented so that a particular instrument index is used for a specific task, i.e., detection vs. characterization.
The overhead time of the optical system must also be provided and is split into two parameters. The first is an integration time multiplier for detection and characterization modes, which represents the individual number of exposures that need to be taken to cover the full field of view, full spectral band, and all polarization states in cases where the instrument splits polarizations. For detection modes, we will typically wish to cover the full field of view, while possibly only covering a small bandpass and only one polarization, whereas for characterizations, we will typically want all polarizations and spectral bands, while focusing on only one part of the field of view. The second overhead parameter gives a value for how long it will take to reach the instrument's designed contrast on a given target. This overhead is separate from the one specified in the observatory definition, which represents the observatory settling time and may be a function of orbital position, whereas the contrast floor overhead may depend on target brightness. If this value is constant, as in the case of an observing strategy where a bright target is used to generate the high contrast regions, or zero, as in the case of an occulter, then it can be folded in with the observatory overhead.

Star Catalog
The Star Catalog module includes detailed information about potential target stars drawn from general databases such as SIMBAD, 11 mission catalogs such as Hipparcos, 12 or from existing curated lists specifically designed for exoplanet imaging missions. 4 Information to be stored, or accessed by this module will include target positions and proper motions at the reference epoch (see §2.1.6), catalog identifiers (for later cross-referencing), bolometric luminosities, stellar masses, and magnitudes in standard observing bands. Where direct measurements of any value are not available, values are synthesized from ancillary data and empirical relationships, such as color relationships and mass-luminosity relations. 13 This module will not provide any functionality for picking the specific targets to be observed in any one simulation, nor even for culling targets from the input lists where no observations of a planet could take place. This is done in the Target List module as it requires interactions with the Planetary Population module (to determine the population of interest), the Optical System Description module (to define the capabilities of the instrument), and Observatory Definition module (to determine if the view of the target is unobstructed).

Planet Population Description
The Planet Population Description module encodes the density functions of all required planetary parameters, both physical and orbital. These include semi-major axis, eccentricity, orbital orientation, and planetary radius and mass. Certain parameter models may be empirically derived 14 while others may come from analyses 15,16 of observational surveys such as the Keck Planet Search, 17,18 Kepler, [19][20][21] and ground-based imaging surveys including the Gemini Planet Imager Exoplanet Survey. 22,23 This module also encodes the limits on all parameters to be used for sampling the dis-  The specific implementation of the duty cycle function can have significant effects on the science yield of the mission. For example, if the observing program is pre-determined, such that exoplanet observations can only occur at specific times and last for specific durations, this significantly limits the observatory's ability to respond dynamically to simulated events, such as the discovery of an exoplanet candidate. This can potentially represent a sub-optimal utilization of mission time, as it may prove to be more efficient to immediately spectrally characterize good planetary candidates rather than attempting to re-observe them at a later epoch. It also limits the degree to which followup observations can be scheduled to match the predicted orbit of the planet. Alternatively, the duty cycle function can be implemented to give the exoplanet observations the highest priority, such that all observations can be scheduled to attempt to maximize dynamic completeness 2 or some other metric of interest.
The keepout definition determines which target stars are observable at a specific time during the mission simulation and which are unobservable due to bright objects within the field of view such as the sun, moon, and solar system planets. The keepout volume is determined by the specific design of the observatory and, in certain cases, by the starlight suppression system. The observatory definition also includes the target transition time, which encodes the amount of overhead associated with transitioning to a new target before the next observation can begin. For missions with external occulters, this time includes both the transit time between targets as well as the time required to perform the fine alignment at the end of the transit. For internal coronagraphs, this includes the settling time of the telescope to reach the bus stability levels required by the active wavefront control system. These may all be functions of the orbital position of the telescope, and may be implemented to take into account thermal effects when considering observatories on geocentric orbits. This overhead calculation does not include any additional time required to reach the instrument's contrast floor, which may be a function of target brightness, and is encoded separately in the Optical System Description.
In addition to these functions, the observatory definition can also encode finite resources that are used by the observatory throughout the mission. The most important of these is the fuel used for stationkeeping and repointing, especially in the case of occulters which must move significant distances between observations. We could also consider the use of other volatiles such as cryogens for cooled instruments, which tend to deplete solely as a function of mission time. This module also allows for detailed investigations of the effects of orbital design on the science yield, e.g., comparing the baseline geosynchronous 28.5 • inclined orbit for WFIRST-AFTA 24 with an alternative L2 halo orbit also proposed for other exoplanet imaging mission concepts. 25

Planet Physical Model
The Planet Physical Model module contains models of the light emitted or reflected by planets in the wavelength bands under investigation by the current mission simulation. It uses physical quantities sampled from the distributions defined in the Planet Population, including planetary mass, radius, and albedo, along with the physical parameters of the host star stored in the Target List module, to generate synthetic spectra or band photometry, as appropriate. The planet physical model is explicitly defined separately from the population statistics to enable studies of specific planet types under varying assumptions of orbital or physical parameter distributions, i.e., evaluating the science yield related to Earth-like planets under different definitions of the habitable zone.
The specific implementation of this module can vary greatly, and can be based on any of the many available planetary albedo, spectra and phase curve models. 26  The Rules module also encodes the calculation of integration time for an observation. This can be based on achieving a pre-determined signal to noise (SNR) metric (with various possible definitions), or via a probabilistic description as in Ref. 33. This requires also defining a model for the background contribution due to all astronomical sources and especially due to zodiacal and exozodiacal light. 5 The integration time calculation can have significant effects on science yield-integrating to the same SNR on every target may represent a suboptimal use of mission time, as could integrating to achieve the minimum possible contrast on very dim targets. Changing the implementation of the Rules module allows exploration of these tradeoffs directly.

Post-Processing
The Post-Processing module encodes the effects of post-processing on the data gathered in a simulated observation, and the effects on the final contrast of the simulation. In the simplest implementation, the Post-Processing module does nothing and simply assumes that the attained contrast is some constant value below the instrument's designed contrast-that post-processing has the effect of uniformly removing background noise by a pre-determined factor. A more complete implementation actually models the specific effects of a selected post-processing technique such as LOCI 34 or KLIP 35 on both the background and planet signal via either processing of simulated images consistent with an observation's parameters, or by some statistical description.
The Post-Processing module is also responsible for determining whether a planet detection has occurred for a given observation, returning one of four possible states-true positive (real detection), false positive (false alarm), true negative (no detection when no planet is present) and false negative (missed detection). These can be generated based solely on statistical modeling as in Ref. 33, or can again be generated by actually processing simulated images.

Simulation Modules
The simulation modules include Target List, Simulated Universe, Survey Simulation and Survey Ensemble. These modules perform tasks which require inputs from one or more input modules as well as calling function implementations in other simulation modules.

Target List
The Target List module takes in information from the Optical System Description, Star Catalog, Planet Population Description, and Observatory Definition input modules and generates the input target list for the simulated survey. This list can either contain all of the targets where a planet with specified parameter ranges could be observed, 36 or can contain a list of pre-determined targets such as in the case of a mission which only seeks to observe stars where planets are known to exist from previous surveys. The final target list encodes all of the same information as is provided by the Star Catalog module.

Simulated Universe
The Simulated Universe module takes as input the outputs of the Target List simulation module to create a synthetic universe composed of only those systems in the target list. For each target, a planetary system is generated based on the statistics encoded in the Planet Population Description module, so that the overall planet occurrence and multiplicity rates are consistent with the provided distribution functions. Physical parameters for each planet are similarly sampled from the input density functions. This universe is encoded as a list where each entry corresponds to one element of the target list, and where the list entries are arrays of planet physical parameters. In cases of empty planetary systems, the corresponding list entry contains a null array.
The Simulated Universe module also takes as input the Planetary Physical Model module instance, so that it can return the specific spectra due to every simulated planet at an arbitrary observation time throughout the mission simulation.

Survey Simulation
The Survey Simulation module takes as input the output of the Simulated Universe simulation module and the Time, Rules, and Post-Processing input modules. This is the module that performs a specific simulation based on all of the input parameters and models. This module returns the mission timeline -an ordered list of simulated observations of various targets on the target list along with their outcomes. The output also includes an encoding of the final state of the simulated universe (so that a subsequent simulation can start from where a previous simulation left off) and the final state of the observatory definition (so that post-simulation analysis can determine the percentage of volatiles expended, and other engineering metrics).

Survey Ensemble
The Survey Ensemble module's only task is to run multiple simulations. While the implementation of this module is not at all dependent on a particular mission design, it can vary to take advantage of available parallel-processing resources. As the generation of a survey ensemble is an embarrassingly parallel task-every survey simulation is fully independent and can be run as a completely separate process-significant gains in execution time can be achieved with parallelization. The baseline implementation of this module contains a simple looping function that executes the desired number of simulations sequentially, as well as a locally parallelized version based on IPython Parallel. 37

WFIRST-AFTA Coronagraph Modeling
While the development of EXOSIMS is ongoing, we have already produced simulation results with the functionality out of which the baseline EXOSIMS implementation is being built. In this section, we present the results of some mission simulations for WFIRST-AFTA using optical models of coronagraph designs generated at JPL during the coronagraph downselect process in 2013, as well as post-downselect optical models of the Hybrid Lyot Coronagraph (HLC) 38  In addition to the HLC, the first set of optical models includes models for a Shaped Pupil Coronagraph (SPC) 39

and a Phase-Induced Amplitude Apodization Complex Mask Coronagraph
(PIAA-CMC). 40 In the downselect process, the SPC and HLC were selected for further develop- The Star Catalog is based on a curated database originally developed by Margaret Turnbull,4 with updates to stellar data, where available, taken from current values from the SIMBAD Astronomical Database. 11 Target selection is performed with a detection integration time cutoff of 30 days and a minimum completeness cutoff of 2.75%. 36 Revisits are permitted at the discretion of the automated scheduler, 3 and one full spectrum is attempted for each target (spectra are not repeated if the full band is captured on the first attempt). The total integration time allotted is one year, spaced over six years of mission time with the coronagraph getting top priority on revisit observations.

Comparison of Pre-Downselect Coronagraph Designs
As a demonstration of EXOSIMS ability to compare different instrument designs for a single mission concept, we compare mission simulation results based on optical models of the pre-downselect SPC, HLC and PIAA-CMC designs. As all of these represent preliminary designs that have since been significantly improved upon, and as our primary purpose here is to demonstrate the simulations' utility, we will refer to the three coronagraphs simply as C1, C2, and C3 (in no particular order). Table 1 lists some of the parameters of the three coronagraphs including their inner and outer working angles, their minimum and mean contrasts, and maximum and mean throughputs.
Each design has significantly different operating characteristics in its region of high contrast (or 'dark hole'). C3 provides the best overall minimum contrast and IWA, but has a more modest mean contrast, whereas C2 has the most stable, and lowest mean contrast over its entire dark hole, at the expense of a larger inner working angle. C1 has the smallest angular extent for its dark hole, but maintains reasonably high throughput throughout. C2 has a constant, and very low throughput, while C3 has the highest throughput over its entire operating region. Finally, while C1 and C3 cover the full field of view with their dark holes, C2 only creates high contrast regions in 1/3 of the field of view, and so requires three integrations to cover the full field.
We consider five specific metrics for evaluating these coronagraph designs: 1. Unique planet detections, defined as the total number of individual planets observed at least once.
2. All detections, defined as the total number of planet observations throughout the mission (including repeat observations of the same planets).    Table 1 assuming either a factor of 10 or 30 in post-processing contrast gains. Of particular importance here is the probability of zero detections-all of the designs at 10x suppression, and C1 in particular, have a significant (> 5%) chance of never seeing a planet.
From the tabulated values, we see that the three coronagraphs have fairly similar performance in terms of number of planets found and spectrally characterized. Overall, C2 is most successful at detecting planets, due primarily to the stability of its contrast over the full dark hole. However, because of the very low overall throughput, this does not translate into more spectral characterizations than the other two designs. C1 and C2 benefit more from the change from 10x to 30x contrast improvement due to post-processing than does C3, which already has the deepest overall contrast, but whose contrast varies significantly over the dark hole. The largest differences in the metrics are the total number of observations. These illustrate the direct trade-off between acquiring spectra, which take a very long time, and doing additional integrations on other targets. In cases such as  C2 with only 10x contrast improvement, the spectral characterization times are typically so long that most targets do not stay out of the observatory's keepouts and so the mission scheduling logic chooses to do more observations rather than wasting time on impossible spectral integrations.

All Detections Nor malized Fr equency
Turning to the figures of the full distributions for these metrics, we see that despite having similar mean values for unique planet detections, the full distributions of detections are quite different, leading to varying probabilities of zero detections. As this represents a major mission failure mode, it is very important to track this value, as it may outweigh the benefits of a given design. C1 with only 10x contrast gain does particularly poorly in this respect, with over 15% of cases resulting in no planets found. However, when a 30x gain is assumed, C1 and C2 end up having the lowest zero detection probabilities. We again see that the effects of even this simple post-processing assumption are not uniform over all designs. This is due to the complicated interactions between each Total Vis its Nor malized Fr equency instrument's contrast curve and the assumed distributions of planetary parameters. In essence, if our priors were different (leading to different completeness values for our targets) then we would expect different relative gains for the same post-processing assumptions. This is always a pitfall of these simulations and must always be kept in mind when analyzing the results. It should also be noted that there have been multiple iterations of all these coronagraph designs since downselect, resulting in significantly lower probabilities of zero detections, as seen in the next section.
Another interesting feature are the very long right-hand tails of the all detections and total visits distributions. These do not actually represent outliers in terms of highly successful missions, but rather typically imply the existence of one or a small number of very easy to detect planets.
The logic of the scheduler allows the mission to keep returning to these targets for followup observations when it has failed to detect any other planets around the other targets in its list. This situation arises when the design of the instrument and assumptions on planet distributions leave Unique Tar gets Nor malized Fr equency  The input mass distribution shown here is derived from the Kepler radius distribution as reported in Ref. 42 and is calculated by assuming that this distribution is the same for all orbital periods and via an assumed density function. 6 The frequency spike seen at around 20 Earth masses is due to a poor overlap in the density functions used in this part of the phase space. This results in an equivalent spike in the posterior distributions, which slightly biases the results.
All of the instruments have fairly similar selection biases, although C1 and C3, which have smaller inner working angles and higher throughputs, detect more lower mass/radius planets. The effects of the instruments are readily apparent in all cases: lower radius planets, which are predicted to occur more frequently than larger radius ones, are detected at much lower rates.

Comparison of HLC Parameters
In this section we present the results of survey ensemble analyses for a single instrument-a postdownselect HLC design-again assuming either 10x or 30x post-processing gains, and assuming either 0.4, 0.8, or 1.6 milliarcseconds of telescope jitter. The jitter of the actual observatory will be a function of the final bus design and the operation of the reaction wheels, and its precise value is not yet known, which makes it important to evaluate how different levels of jitter may effect the achieved contrast and overall science yield. The jitter is built directly into the optical system model    Fig. 3. The input mass distribution is derived from sampling the radius distribution shown in Fig. 8 and converting to mass via an assumed density function.
The mean and 1σ of the five metrics of interest described in §3.1 are tabulated in Table 3, and the full PDFs for all metrics are shown in Figs. 10 -14.
One important observation made immediately obvious by these results is the relatively large effect of increased jitter versus the gains due to post-processing. Tripling the assumed gain factor of post-processing on the final achieved contrast has a significantly smaller effect on the number of detections, gaining only one unique detection, on average, as compared with halving the amount of telescope jitter, which increases the number of unique detections by over 30%, on average. This shows us that the telescope jitter may be an effect that fundamentally cannot be corrected after the fact, and therefore needs to be tightly controlled, with well defined requirements set during mission design. Much of the current development effort for the project is focused on low-order wavefront sensing and control to mitigate these effects. 43  It should be noted that the change in assumed post-processing gain has a significantly smaller effect than the increased telescope jitter. We also note that the 1.6 mas jitter cases still have a small (0.6 to 1.8%) probability of never seeing a planet, whereas the 0.4 mas jitter ensembles do not contain a single simulation with zero plaents detected.
We can also see significant improvements in the coronagraph design since the versions evaluated in §3.1, as the probability of zero planet detections is less than 2% in the case of the highest jitter level, and is well below 1% for all other cases. In fact, for both the 0.4 mas jitter ensembles, no simulations had zero detections, indicating a very low probability of complete mission failure for this coronagraph at these operating conditions. Similar to the results of the previous section, the trend in the number of total visits does not simply follow those seen in the unique and total detection metrics, but is a function of both the number of detections and how much time is spent on spectral characterizations. We can see how the cases with the highest jitter and lowest post-processing gains are pushed towards larger numbers of observations, and unique targets, as they are able to achieve fewer full spectral characterizations, leaving them with additional mission time to search for new candidates. This is equally reflected in Fig. 14 where, despite the good performance seen in Fig. 10, all jitter levels have over 5% chance of zero full spectra at the 10x post-processing gain level, and only the 0.4 mas case at 30x gain has no instances of zero full spectra in its ensemble of results.
These metrics, taken together, clearly show that further optimization is possible via modification of mission rules, which were kept constant in all these ensembles. For example, the low numbers of spectral characterizations at higher jitter levels suggest that it may be worthwhile to attempt shallower integrations in order to be able to make more total observations and potentially find a larger number of bright planets. This would bias the final survey results towards larger planets, but would increase the probability of spectrally characterizing at least some of the planets discovered. Alternatively, this may point to the desirability of investigating whether full spectral characterizations can be achieved for a small number of targets over the course of multiple Total Vis its Nor malized Fr equency 0.4m as 30x 0.4m as 10x 0.8m as 30x 0.8m as 10x 1.6m as 30x 1.6m as 10x  Fig. 10. Here, the post-processing improvement factor makes more of a difference than in the previous two figures, as more time must be devoted to spectral characterizations, limiting how much time is available for further observations. independent observations.

Conclusions
We have presented the design details of EXOSIMS-a modular, open source software framework for the simulation of exoplanet imaging missions with instrumentation on space observatories. We    Fig. 10. The trend here tracks closely to the one observed in the total visits metric, and shows that this coronagraph design is not target limited in any of the studied cases.
As both the tools and the coronagraph and mission design continue to mature we expect the predictions presented here to evolve as well, but certain trends have emerged that we expect to persist. We have identified the portions of design space and telescope stability ranges that lead to significant probabilities of zero detections, and we expect instrument designs and observatory specifications to move away from these. We have also identified a mean number of new planetary detections, for our particular assumed prior distributions of planetary parameters, that are consistent with the science definition team's mission goals for this instrument.
As we continue to both develop the software and to improve our specific modeling of WFIRST-AFTA we expect that these and future simulations will prove helpful in guiding the final form of the mission, and will lay the groundwork for analysis of future exoplanet imagers.  Fig. 10. In the worst case, there is an ∼ 15% of not getting any spectra. Only the case of 0.4 mas jitter with 30x post-processing gain has no simulations in its ensemble with zero full spectra achieved.

Daniel Garrett is a PhD student in the Sibley School of Mechanical and Aerospace Engineering at
Cornell University. His research interests include dynamics and control theory, planetary science, and space exploration.  a Contrast improvement factor due to post-processing. b Number of individual planets detected one or more times. c Total number of detections (including repeat detections of the same planets). d Total number of planets where spectra can be obtained over the whole wavelength range (400-800 nm). planets about some targets, detected one or more times) for the coronagraph designs described in Table 1 assuming either a factor of 10 or 30 in post-processing contrast gains. Of particular importance here is the probability of zero detectionsall of the designs at 10x suppression, and C1 in particular, have a significant (> 5%) chance of never seeing a planet.

4
PDF of all detections (including repeat detections) for instruments as in Fig. 3.
Note that values of 15 or more typically represent a small number of easily detectable planet that are re-observed many times. Re-observations of a single target were capped at four successful detections in all simulations.

5
PDF of total number of observations (including repeat observations of some targets) for instruments as in Fig. 3.

6
PDF of unique targets observed for instruments as in Fig. 3. While all three instruments have fairly narrow distributions of this parameter, only C2 with 10x postprocessing gains is completely target limited. 7 PDF of number of spectra achieved over the whole band from 400 to 800 nm for instruments as in Fig. 3. C3 does comparatively well in this metric due to its lower IWA and high throughput. The input distribution is based on the Kepler results reported in Ref. 42. 9 Input and output distributions of planetary mass for instruments as in Fig. 3. The input mass distribution is derived from sampling the radius distribution shown in Fig. 8 and converting to mass via an assumed density function.
10 PDF of unique planetary detections (number of individual planets, potentially with multiple planets about some targets, detected one or more times) for the postdownselect HLC design, assuming either a factor of 10 or 30 in post-processing contrast gains and telescope jitter of 0.4, 0.8 or 1.6 mas. It should be noted that the change in assumed post-processing gain has a significantly smaller effect than the increased telescope jitter. We also note that the 1.6 mas jitter cases still have a small (0.6 to 1.8%) probability of never seeing a planet, whereas the 0.4 mas jitter ensembles do not contain a single simulation with zero plaents detected.
11 PDF of total number of planetary detections (including repeat detections) for instruments as in Fig. 10. The trend here closely follows the one observed in the results for the unique detections metric.
12 PDF of total number of target observations (including repeat observations) for instruments as in Fig. 10. Here, the post-processing improvement factor makes more of a difference than in the previous two figures, as more time must be devoted to spectral characterizations, limiting how much time is available for further observations.
13 PDF of unique targets observed for instruments as in Fig. 10. The trend here tracks closely to the one observed in the total visits metric, and shows that this coronagraph design is not target limited in any of the studied cases.
14 PDF of number of spectra achieved over the whole band from 400 to 800 nm for instruments as in Fig. 10. In the worst case, there is an ∼ 15% of not getting any spectra. Only the case of 0.4 mas jitter with 30x post-processing gain has no simulations in its ensemble with zero full spectra achieved.