Parametric cost models are routinely used to plan missions, compare concepts, and justify technology investments. Unfortunately, there is a wide range of parametric models, with no definitive model for the cost of a space telescope optical telescope assembly (OTA). Part of this problem is the limited number of telescopes that have flown in space for which we have data. Another part is the tendency to extrapolate ground telescope rules of thumb to space telescopes. In 2010, these authors published the article “Single-variable parametric cost models for space telescopes.”1 That paper presented new single-variable cost models for space telescope OTA. These models were created by applying standard statistical methods to data collected from 30 different space telescope missions. The results were compared with previously published models. This paper updates and revises the findings of our 2010 paper. To fully understand the content of this update, the authors recommend that one also read the 2010 paper.
After the publication of the original paper,1 we were invited to present our findings to the National Reconnaissance Office (NRO) in September 2010. The NRO Cost Model Office reviewed our database and, while they did not provide us with access to their database or give us any specific data, they did identify specific missions where our two databases disagreed. In response, we conducted a systematic review of all missions in our database and reconciled them with source documents. As a result, some missions have been removed from our cost database. The costs for some missions have been revised, with revisions ranging from slight to dramatic, and several new missions have been added to the database. As a consequence, there have been changes to the 2010 published results and models. But our two most important findings remain unchanged: aperture diameter is the primary cost driver for large space telescopes, and it costs more per kilogram to build a low-areal-density low-stiffness telescope than a more massive high-stiffness telescope. One significant difference is that we now report telescope cost to vary linearly from 5% to 30% of total mission cost instead of the previously reported average of 20%. The key lesson learned from this process is the need to be precise in all definitions and consistent in their application. Finally, note that the cost model coefficients reported in this paper are inflated to 2011 values.
The OTA is defined as the subsystem that collects electromagnetic radiation and focuses it (focal) or concentrates it (afocal) into the science instruments (SI). An OTA consists of the primary mirror, secondary mirror, auxiliary optics, and support structure (such as optical bench or truss structure, primary support structure, secondary support structure or spiders, straylight baffles, mechanisms for adjusting the optical components, and electronics or power systems for operating these mechanisms). An OTA does not include SIs or spacecraft subsystems. An SI is defined as the subsystem that converts electromagnetic radiation into data. An SI includes conditioning optics (e.g., beam splitters, reimaging optics, spectral filters, dispersive elements), mechanisms, detectors, focal planes, and electronics.
OTA cost is defined as the prime contractor’s cost to design, build, and integrate the OTA. OTA cost includes allocated subsystem-level management and systems engineering as well as program-level costs that can be allocated to the subsystem. For all missions in our database except the Orbiting Astronomical Observatory (OAO), OTA cost does not include NASA labor cost. The reason is that, before the James Webb Space Telescope (JWST), NASA’s accounting system did not track the cost of NASA personnel to a given program, such as Hubble or Kepler. NASA did start tracking these costs in the early phases of JWST when it implemented full-cost accounting. But, to compare apples to apples between JWST and prior missions such as Hubble, we remove NASA labor costs from the JWST cost. Regarding the OAO program, NASA personnel performed most of the design, build and integration effort, and these costs were tracked and reported. Therefore, these costs are included in the database. Total mission cost is not life cycle cost. It is defined as only phase A–D cost, excluding launch cost; costs associated with NASA labor (civil servant or support contractors) for program management, technical insight/oversight, or any NASA-provided ground support equipment, e.g., test facilities. Note that excluding NASA labor costs underestimates the true cost of a given mission by at least 15% and maybe by as much as 30%.
Careful review found that we were inconsistent in the application of the above definitions for OTA cost. In the original paper, the database OTA cost for large missions (Hubble, Kepler, JWST, etc.) were correct. They were for only the OTA. But the database cost for the smaller missions (GALEX, IUE, TRACE, and WIRE) were incorrect. The database costs of these missions were not just for the OTA. Rather, they were an “instrument cost,” where an instrument was defined to be an integrated system consisting of an OTA and an SI. Removing the SI cost to yield just an OTA cost dramatically reduced the database values for these small-aperture missions. Additionally, we established a new definition which resulted in significant changes to the database. We decided to exclude thermal/cryogenic control systems from the definition of an OTA. For example, the JWST OTA does not include the cost of the JWST Sunshade. But the old IRAS and Spitzer OTA costs did include the cryogenic system. Removing these costs dramatically changed their previous database values.
Finally, additional resources were used for source documents, including: NAFCOM (NASA/Air Force Cost Model) database, NICM (NASA Instrument Cost Model), NSCKN (NASA Safety Center Knowledge Now), RSIC (Redstone Scientific Information Center), REDSTAR (Resource Data Storage and Retrieval System), SICM (Scientific Instrument Cost Model), project websites, and interviews.
After careful review of source CADRe documents (Cost Analysis Data Requirements), several changes were made to the database. The costs of the Kepler and Wise missions were increased to include program management, systems engineering, and integration and test cost. The costs of GALEX, HiRISE, HUT, OAO-3, UIT, WIRE, and WUPPE were decreased to remove SI costs. The costs of IRAS and Spitzer were decreased by separating cryostat and OTA cost. The cost of SOFIA was decreased by removing the cost of the gimbal structure that holds the SOFIA OTA in the 747 airframe. Several missions were added to the database, specifically: CloudSat, OAO-B/GEP, Herschel, and Planck. JWST costs were updated to the 2011 estimate to complete. Finally, the Hubble OTA and total missions costs were reduced as shown in Table 1. Previously, the cost of the fine guidance sensor (FGS) had been excluded from the OTA cost, because this cost should be allocated to the spacecraft. But management and systems engineering costs allocated to the FGS had not been properly removed. Additionally, the previous total mission cost included phase E operations costs. Finally, the Hubble OTA mass was increased from 2150 to 3180 kg based on better documentation.2
Refinement of Hubble cost knowledge.
|Old (FY11$)||Revised (FY11$)||Notes|
|Total Cost Phase A–D (Design and Build)||4.0 B||2.8 B||Old: NGST cost model database|
|Total optical telescope assembly (OTA)||0.9 B||0.9 B|
|OTA||0.7 B||0.47 B||Old: allocated fine guidance sensor (FGS) and C&DH PM & SE costs to OTA|
|Optics||0.07 B||New: REDSTAR 121-4742|
|Optics control||0.08 B|
|Optical structure||0.08 B|
|Electrical power||0.02 B|
|Structures, mechanisms, support equipment||0.05 B|
|System level 53%||0.14 B|
|Space telescope level 53%||0.01 B|
|FGS||0.2 B||0.26 B||New: REDSTAR 121-4742|
|C&DH||0.08 B||New: REDSTAR 121-4742|
|Thermal control||0.01 B||New: REDSTAR 121-4742|
|System level 47%||0.12 B||New: REDSTAR 121-4742|
|Space Telescope level 47%||0.01 B||New: REDSTAR 121-4742|
|Total SSM (spacecraft)||1.14 B||New: REDSTAR 121-4742|
|Science instrument (SI)||0.5 B||New: REDSTAR 123-1064 (p. 108)|
|European Space Agency contribution||0.25 B||New: REDSTAR 123-1064 (p. 108)|
|Total cost phase A–E||5.1 B||4.6 B||Old: NGST cost model|
|Launch||0.62 B||New: REDSTAR 123-1064 (p. 108)|
|Phase E (Operations)||1.2 B||New: REDSTAR 123-1064 (pp. 108 and 122)|
Pearson Cross-Correlation Analysis of Parameters
Because the database has changed, the cross-correlation matrix has also changed. However, the methodology of using the cross-correlation matrix to guide the statistical analysis has not changed. Therefore, for the sake of brevity, there is no need to show a corrected matrix.
OTA Cost Versus Total Cost
Changes to the database significantly changed the ratio of OTA cost to total mission cost. Previously, the ratio of OTA cost to total mission cost was spread from a few percent to 65%. The net effect of this spread was to make it appear that on average, the OTA was approximately 20% of total mission cost. But, with the corrections, the OTA cost as a percentage of total mission cost varies linearly as a function of aperture diameter size from a few percent to nearly 30% (Fig. 1). It is hypothesized that the reason for this increase with aperture size is infrastructure and technology reuse. Smaller-aperture missions tend to use existing manufacturing and testing infrastructure, whereas larger-aperture missions often require the design and fabrication of expensive custom infrastructure. Also, smaller missions tend to have higher reuse of existing designs. Finally, the data imply that for small missions, other major subsystems (such as the spacecraft) are a much larger cost for the total mission than the OTA.
An analysis of detailed work breakdown structure documents of the seven missions for which we have such documents shows that, on average, the spacecraft accounts for approximately 25% of the cost, SIs account for 25%, OTAs account for 12%, program management and systems engineering account for 12%, integration and testing account for 10%, and the balance is “other” (Fig. 2). Obviously, there is significant uncertainty in these percentage values.
Single-Variable Cost Models
Using the revised database, single-variable models are created by regressing OTA cost data versus selected parameters in single-variable models (Fig. 3). Note that Fig. 3 shows the regression results for 15 free-flying OTAs in the database. Also, the aperture diameter result changes if CloudSat, Planck, or Herschel is excluded from the regression.
As discussed in the original paper, each regression is evaluated for its goodness of fit and significance via a range of statistical measures, including Pearson coefficient, Student -test value, and standard percent error (SPE). Pearson (typically denoted as just ) describes the percentage of agreement between the model and the actual cost in log-log space. For multivariable models, we use adjusted Pearson (or ), which accounts for the number of data points and the number of variables. In general, the closer (or ) is to 1.0% or 100%, the better the model. SPE is a normalized standard deviation of the fit residual (difference between data and fit) to the fit. The closer SPE is to 0, the better the fit. Please note that since SPE is normalized, a small variation divided by a very small parameter coefficient can yield a very large SPE. The value is the probability that a fit or correlation would occur if the variables are independent of each other. The closer the value is to 0, the more significant the fit or correlation. The closer it is to 1, the less significant. For the purpose of our study, we consider any value of less than 0.10 to be “good” and any value greater than 0.10 to be “bad.” Also, it is important to consider how many data points are included in a given correlation, fit or regression. Again, for the purpose of our student, based correlation statistical significance, we urge caution before using any regression with fewer than 12 data points.
The variables that yield a significant regression for OTA cost are aperture diameter (meters), primary mirror focal length (meters), volume (cubic meters), pointing stability (arc-seconds), and mass (kilograms). Of these, aperture diameter is the independent variable. The others are dependent variables because they are correlated with diameter. All space telescope OTAs tend to have similar values, so larger apertures have longer focal lengths. Also, pointing stability is directly proportional to resolution, which is defined by aperture diameter. And of course, larger-aperture OTAs have larger volumes and are more massive than smaller-aperture OTAs. Unfortunately, while these authors disagree with the practice, many cost models only use mass to estimate cost. Therefore, we do report mass models in addition to aperture models.
Cost as a function of aperture diameter cost-estimating relationship
The difference between this update paper and the original paper is that the diameter exponent has increased from 1.2. The reason for this increase is that the small-aperture mission OTA costs decreased when the SI costs were removed. Also, in the original paper, we asserted that because the diameter exponent of 1.2 was so much less than 2 that the areal cost of large-aperture telescopes was significantly less than the areal cost for small-aperture telescopes. As will be discussed below, the diameter exponent has increased, and it is unclear at this time if this assertion is still valid. We will revisit this issue in a future multivariable paper.
Based on a sample size of 15 free-flying space telescopes, a single-variable cost-estimating relationship (CER) was developed for OTA cost as a function of primary mirror diameter:3:
However, as indicated by the SPE, the new model is a bit noisy. The reason for this noise is that the revised database includes three additional missions that provide significant wavelength diversity because of their longer diffraction limited wavelengths: , , and . Because the original paper did not include CloudSat, Planck, or Herschel, if we exclude these three OTAs, the regression becomes:
Figure 4(a) shows the data plotted for 17 free-flying telescopes and four attached telescopes (whose data was excluded from the regression). As discussed in the original paper, the attached OTAs are excluded because, while their trend line slope is similar to that of the free-flying telescopes, their leading coefficient is 2 to 3× lower. As a reminder, the attached OTAs are defined as UIT, WUPPE, and HUT, which flew attached to the space shuttle orbiter, and SOFIA, which flies attached to a 747. Note that, in the original paper, the cost for SOFIA was too high because it included the cost of the gimbal. As a result, in the original paper SOFIA’s cost was in family with the three space shuttle missions. However, after removing the cost of the gimbal, SOFIA’s cost dropped to a level between that of the shuttle attached OTAs and the ground OTAs. Regardless, the implication is that the basic engineering issues that drive cost as a function of aperture apply equally to all OTAs but that the lower the mass, i.e., the less stiff the OTA, the more difficult and more expensive it is to fabricate. The relationship between the free-flying and attached OTAs becomes evident if one eliminates the CloudSat, Planck, and Herschel data points [Fig. 4(b)].
Finally, a regression of 21 free-flying missions yields a total mission cost versus aperture diameter CER of
Cost as a function of mass
The difference between this update paper and the original paper is that the mass exponent has increased from 0.7 to 1.0 and the value has decreased from 92% to 55%. The reason for the exponent increase is that removing the instruments from the smaller missions lowered the OTA cost more than it lowered the OTA mass, which indicates that instruments have a higher cost per kilogram than OTAs. The reason that the decreased is because of the change in the Hubble OTA mass. Previously HST’s OTA mass was approximately the same as JWST’s OTA mass. But the new documentation yields an HST OTA mass that is 150% that of JWST’s OTA. Also, excluding CloudSat, Planck, and Herschel from the OTA mass regression has a negligible effect. Therefore we will not exclude them. Finally, while the mass and cost of the four attached OTAs were also reduced, they continued to lie on a cost curve parallel to and below that of the free-flying model. Because we removed the gimbal cost and mass from SOFIA, it is no longer on the same line as the three missions that flew attached to the space-shuttle orbiter.
Based on a sample size of 14 free-flying space telescopes, a single-variable CER was developed for OTA cost as a function of OTA mass (Fig. 5):
A new graphical tool developed since the original paper is cost density (cost per kilogram). Figure 6 plots OTA cost per kilogram versus OTA aperture diameter for free-flying space, attached, and ground OTAs. Several obvious conclusions can be drawn. All free-flying space telescopes have approximately the same cost per kilogram, independent of aperture diameter. All ground telescopes also have approximately the same cost per kilogram, independent of aperture diameter. Space telescopes cost about more than ground telescopes. Additionally, UIT, WUPPE, and HUT, which flew attached to the space shuttle, are 5 to 10× less expensive per kilogram. SOFIA, which flies attached to a 747, is 20 to 30× less expensive. One explanation is that each of these mission types are built to different design rules. While all three types need similar wavefront shape and pointing stabilities as a function of aperture diameter, they have different static gravity and dynamic jitter environments. They also have different mass budgets with which to achieve the required wavefront shape and pointing stability. Free-flying telescopes have mass budgets that are severely constrained by the launch vehicle. Therefore, significant engineering cost is required to achieve the required performance for the allowed mass. While the attached missions did fly on the space shuttle and SOFIA flies on a 747, the carrying capacities of these vehicles allows for different mass margin design rules.
Finally, as shown in Fig. 7, a regression of 26 free-flying missions yields a total mission cost versus total mission mass CER of
Discussion on Mass
As with the original paper, mass appears to be an attractive CER for both OTA cost and total mission cost. however, it is the belief of these authors that in both cases mass is a surrogate for other engineering parameters. In the case of OTA, mass depends on aperture diameter. And, as indicated by Fig. 6, changing the design rule to allow for a larger mass budget might reduce total cost. Additionally, there are multiple cautionary indicators against a pure mass model. For example, consider JWST and HST. Hubble is more massive than JWST at both the OTA (150%) and mission (200%) level (Figs. 5 and 7). But JWST is 2× more expensive than HST at both the OTA and mission level. A pure mass model would predict that JWST would be half as expensive as HST—and it would be wrong. The reason is that HST is more massive, stiffer, and less complex than JWST. Next, consider the three shuttle-attached missions. For the same aperture diameter, attached missions are × more massive and × less expensive. This is an interesting comparison because the attached missions were space-rated. Of course, there are multiple mitigating factors, such as their limited design life, but given that most OTAs have very few limited lifetime components, these authors believes it to be a valid comparison. Finally, ground-based OTAs are 10 to 100× more massive than free-flying space OTAs and 1000×/kg less expensive.
Parametric cost models are an important tool routinely used to plan missions, compare concepts, and justify technology investments. In 2010, “Single variable parametric cost models for space telescopes”1 presented new single-variable cost models for space telescope OTAs. Its models were created by applying standard statistical methods to data collected from 30 different space telescope missions. The results were compared with previously published models. After its publication, we were invited to present our findings to the NRO. The NRO Cost Model Office reviewed our database and, while they did not provide us with access to their database or give us any specific data, they did identify specific missions where our two databases disagreed. In response, we conducted a systematic review of all missions in our database and reconciled them with source documents. This paper updates and revises the findings of our 2010 paper. To fully understand the content of this update paper, the authors recommend that one also read the original 2010 paper.
Our review determined that we had been inconsistent in the application of our definition as to what is and is not included in an OTA. For all of our modeling, we define an OTA to be the subsystem that collects electromagnetic radiation and focuses it (focal) or concentrates it (afocal) into the SIs. An OTA consists of the primary mirror, secondary mirror, auxiliary optics, and support structure (such as optical bench or truss structure, primary support structure, secondary support structure or spiders, straylight baffles, mechanisms for adjusting the optical components, and electronics or power systems for operating these mechanisms). An OTA does not include SIs or spacecraft subsystems. In the 2010 paper, the database OTA cost for large missions (Hubble, Kepler, JWST, etc.) were correct. They were for only the OTA. But the database cost for the smaller missions (GALEX, IUE, TRACE, and WIRE) were incorrect. The database costs of these missions were for both the OTA and SI. Removing the SI cost to yield just an OTA cost dramatically reduced the database values for these small-aperture missions. As a consequence, there have been changes to the 2010 published results and models. But our two most important findings remain unchanged.
First, from an engineering and scientific perspective, aperture continues to be the best parameter with which to build a space telescope cost model. Aperture defines the observatory’s science performance (sensitivity and resolution) and determines the payload’s size and mass. Depending on which telescope OTAs are included or excluded from the regression, the CER for an OTA as a function of aperture diameter (inflated to 2011 values) is bounded between
Finally, one significant difference with the 2010 paper is that we now report telescope cost to vary linearly from 5% to 30% of total mission cost instead of the previously reported average of 20%. The larger the OTA, the greater percentage of the total mission cost it consumes.
H. Philip Stahl is a senior optical physicist at NASA MSFC, currently leading an effort to mature technologies for a new large-aperture UV/optical/IR telescope to replace Hubble. Previous assignments include astrophysics division deputy assistant director for technology and mirror technology lead for the James Webb Space Telescope. He is a leading authority in optical metrology, optical engineering, and phase-measuring interferometry. Many of the world’s largest telescopes have been fabricated with the aid of high-speed and infrared phase-measuring interferometers developed by him, including the Keck, VLT, and Gemini telescopes. He is a member of OSA and SPIE (fellow). He earned his PhD in optical science at the University of Arizona Optical Sciences Center in 1985.
Todd Henrichs received his BS degree in mathematics from Middle Tennessee State University (MTSU) in 2010. He is currently a graduate student at MTSU and plans to receive an MS degree in statistics in December 2013. He has been performing cost modeling with Dr. Stahl since 2009 and is currently an operations research analyst with the Missile Defense Agency on Redstone Arsenal.