Webb Space Telescope primary mirror development: summary and lessons learned

Abstract. The primary mirror is central to the success of the Webb Space Telescope and the product of 100s of engineers and technologists who invented technologies and processes for its manufacture and test. We summarize the Webb mirror technology development program, explain how the technology was demonstrated to be TRL-6 (including the importance of an Engineering Development Unit), and list some of the author’s personal lessons learned.


Introduction
][3] From the beginning, mirror technology was identified as a critical capability.The summer study of 1996 determined that achieving the desired science objectives required a never before demonstrated space telescope capability: one with an 8-m primary mirror (providing 50 m 2 of collecting aperture) that is diffraction limited at 2 μm and operates at temperatures below 70 K. 4,5 Furthermore, because of launch vehicle limitations, two very significant architectural constraints were placed upon the telescope: segmentation and mass.Each of these directly resulted in specific technology capability requirements.First, because the launch vehicle fairing payload dynamic envelope diameter is ∼4.5 m, the only way to launch an 8 m class mirror is to segment it, fold it, and deploy it on orbit.Second, because of launch vehicle mass limits, the primary mirror allocation was only 1000 kg-resulting in a maximum areal density of 20 kg∕m 2 . 6inally, a cost goal of $500 M was levied on the Optical Telescope Assembly (OTA)-yielding an area cost of 50 M∕m 2 .Also, a production goal of 1 m 2 of glass per month was defined. 7n assessment of the pre-1996 state-of-the-art (as demonstrated by existing space, ground, and laboratory test bed telescopes) indicated that the necessary mirror technology was at a technology readiness level (TRL) of 3 (see Table 1).The largest space telescope was Hubble.Its 2.4m glass primary mirror has an areal density of 180 kg∕m 2 and operates at 300 K. Additionally, its primary mirror assembly has an areal density of 240 kg∕m 2 , and its OTA has an areal density of 420 kg∕m 2 .All values were significantly higher than what NGST required.Ground telescopes, such as Keck, demonstrated 10-m class semiactively controlled segmented mirrors.But as ground telescopes, they were exceedingly massive (2000 kg∕m 2 ) and thermally unsuitable.Test beds, such as the ITEK Advanced Large Optical Telescope (ALOT) and the Kodak Advanced Optical System Demonstrator (AOSD), demonstrated a proof of concept for a 4-m class pseudospace-qualifiable actively controlled segmented telescope in a laboratory environment; 8,9 the US Air Force Large Active Mirror Project (LAMP) demonstrated a 4-m actively controlled segmented primary mirror operating in a vacuum environment. 10But again, these test beds were 2× to 6× too massive for Webb (50 to 150 kg∕m 2 ) and only operated at ambient temperatures.The largest cryogenic mirror under development was the 0.85-m diameter Infrared Telescope Technology Testbed (ITTT) Beryllium primary mirror, which would eventually fly in the Spitzer Space Telescope in 2003.Additionally, the cost per square meter of the primary mirror for both Hubble and Spitzer was ∼$10 M∕m 2 (FY 2010), and the production rate for Hubble had been ∼1 year∕m 2 of polished glass, whereas Spitzer produced 1 m 2 in 4 months.
Finally, because one cannot make what cannot be measured, the Webb mirror technology development program required the invention and development of new optical metrology technologies.
This paper reviews the Webb mirror technology development program, explains how the technology was demonstrated to be TRL-6 [including the importance of an Engineering Development Unit (EDU)], and lists some of this author's personal lessons learned.This paper summarizes (by merging three papers) [11][12][13] 15 years of work by many people and organizations.

Mirror Technology Development
Based on the 1996 assessment and architectural concept studies performed by Lockheed-Martin, TRW (now Northrop), and NASA Goddard Space Flight Center (GSFC), it was concluded that NGST was feasible-provided that a well-planned, aggressive technology development effort was implemented early in the development phase. 4Thus a systematic mirror technology development program was initiated to invent mirror systems that could meet the NGST requirements; reduce the cost, schedule, mass, and risk of such mirror systems; and demonstrate a TRL of 6.An excess of $40 M was invested in mirror technology development from 1998 to 2004.As the lead for NGST Mirror Technology Development, Marshall Space Flight Center (MSFC) managed the investment and provided the study's principal investigator.The investment occurred through a series of related contracts: Subscale Beryllium Mirror Demonstrator (SBMD, $1.5 M), NGST Mirror System Demonstrator (NMSD, $15 M), and Advanced Mirror System Demonstrator (AMSD, $26 M), as well as several small technology studies and Small Business Innovative Research contracts.Additional mirror technology developments were conducted under the TRW (now Northrop) and Lockheed Pre-Phase-A Architecture Study Contracts.
The mirror technology development program was explicitly designed to be broad, follow a sequential or spiral development path, and employ phased down-select competition to produce TRL-6 mirrors.Specific technology areas investigated included substrate material (glass, beryllium, silicon carbide, nickel, etc.; mechanical, thermal, and optical material properties; and ability to manufacture large enough substrates; etc.); mirror design (open back, closed back, arched, thin face sheet; launch loads; etc.); architecture (passive, active, rigid, semirigid, etc.); fabrication process (substrate fabrication, grind and polish, and coating); metrology (vibration insensitivity, cryogenic characterization, etc.); and performance (cryogenic, thermal, mechanical, launch loads, etc.). 14,15Full and subscale mirror systems and their constituent components (i.e.., flexures, coatings, and actuators) were fabricated and cryogenically tested.Significant investments were made in facilities, equipment, procedures, and expertise.Also, to improve the ability of models to accurately predict on-orbit performance, an extensive program was conducted to characterize the cryogenic properties [i.e., coefficient of thermal expansion (CTE) and CTE uniformity, dynamic dampening, stiffness, and tensile strength] of various mirror and structure materials as well as their susceptibility to micrometeoroid impacts.

Subscale Beryllium Mirror Demonstrator
The SBMD project produced a 0.53-m diameter beryllium mirror with a 20-m radius of curvature (ROC) mounted on a solid Be support structure built by Ball Aerospace (BATC) (see Fig. 1). 16It was cryogenically tested multiple times at MSFC and provided invaluable experience and learning. 17For example, SBMD had cryogenic quilting (cryo-quilting), but the mechanical model did not predict any cryo-quilting.After several design iterations, how to properly model the cryo-quilting was learned. 18Using this knowledge, new rules were defined for how to design lightweight beryllium mirrors without cryo-quilting.These new design rules were successfully proven on AMSD.Additionally, SBMD taught valuable lessons on how to design cryogenic interfaces that do not distort the mirror surface shape.SBMD was also used to certify that the Webb gold coating, uncorrectable surface figure error, and creep were at TRL-6 (see Table 7).However, great caution is advised whenever extrapolating technical performance results from small mirrors to large mirrors.Given its size and design, SBMD was significantly stiffer than either AMSD or the Webb flight mirrors.SBMD was also important as the first use of O-30 beryllium for a cryogenic mirror.In 1996, the state-of-the-art cryogenic mirror was the 85-cm Spitzer Telescope made of I-70 beryllium 19 with a diffraction limited performance of 5 μm.But I-70 Be was not a good choice for the NGST primary mirror.Because it was produced using a mechanical pulverization process, its powder had irregular grain shapes.This irregularity limited how densely the powder could be packed into a hot isostatic pressure (HIP) can-which limited the maximum size mirror that could be made.Also the irregular grain shapes resulted in large CTE inhomogeneity.The solution was O-30 Be developed by Brush Wellman for the Air Force in the late 1980s.Because O-30 Be is a spherical powder material, it has a high packing density (thus allowing for hot isostatic pressuring of larger billets), and its CTE distribution is very uniform (which results in smaller cryo-distortion and higher cryo-stability).Also because O-30 Be has a lower oxide content than I-70 Be, it can achieve a smoother polished surface (i.e., less scatter).The ability to HIP a meter class billet was demonstrated in the late 1990s via the VLT secondary mirror.By 1999, Brush-Wellman had full production capability sufficient for the NGST program.This author's personal lessons learned from SBMD include the following.
• Do not trust models to validate performance-test to validate performance.Not on SBMD, nor on any subsequent study, did an apriori model correctly predict a mirror's thermal performance (i.e., cooling rate or cryo-deformation).Models were only able to replicate test data after the fact.• Validate models on the smallest possible test article before scaling up, and iterate until the model matches the data within the allocated error budget uncertainty.• The importance of repetition and learning.The first time SBMD was cryo-tested, the entire process took 3 months, but after several iterations, the MSFC X-Ray and Cryogenic Facility (XRCF) Team could do a test in a month.By the end of Webb, six mirrors were being cryo-testing at a time.• The importance of subscale demonstrators.SBMD offered the team invaluable early experience on a relatively low-cost but relevant subscale system-including the opportunity to understand the impact of design parameters on cryo-performance.

NGST Mirror System Demonstrator
The NMSD project was the most technically aggressive study.It sought to explore the limits of light weighting and successfully showed what does not work.NMSD clearly demonstrated the important roles that CTE and mechanical stiffness play in the ability to design and manufacture a stable mirror system.NGST produced two 1.6-m hexagonal shaped spherical mirrors with a 20 m ROC and areal density <15 kg∕m 2 .The two mirrors were manufactured by Composite Optics Inc. (COI) and the University of Arizona.The COI mirror was a thin glass sheet bonded to a rigid graphite composite structure.The Arizona mirror was a thin glass sheet attached to a graphite composite structure via 166 actuators.Both NMSD mirrors took significantly longer to make and achieved significantly lower cryo-performance than expected.The causes for these results were assessed to be CTE mismatch and inhomogeneity; too low of areal density (i.e., too low stiffness); and overly complex designs.This author's personal lessons learned from NMSD include the following.
• Avoid mirror systems with multiple CTE materials, even if it appears that the CTEs of the various materials will match at a specific temperature.CTE homogeneity is critical for cryogenic mirrors (or mirrors that need a stable shape as a function of temperature).CTE inhomogeneity produces cryo-wavefront error, and CTE mismatch between different component materials can produce a large error.Because of CTE mismatch, the COI mirror exhibited a very large cryo-deformation and quilting.• Stiffness is more important than areal density.Although SBMD and NMSD had the same "assembly" areal density requirement (<15 kg∕m 2 ), the NMSD systems were more than 10× less stiff (and the Arizona glass face sheet was many orders of magnitude less stiff) than SBMD.The reader is reminded that stiffness increases linearly with thickness and decreases quadratically with diameter.This stiffness difference resulted in profound effects.First, standard fabrication processes, handling procedures, and optician intuitions that are perfectly appropriate for conventional mirrors are not applicable to extremely low stiffness mirrors.In fact, Arizona broke their first face sheet.Second, because of the Arizona mirror's low stiffness, it was impossible to controllably adjust the actuators to figure the mirror.The simple act of stepping onto the test platform would change the shape of the mirror.• Large mirrors are harder to make than small mirrors and scale-up incrementally.Although SMBD had been successful, NMSD's factor of 3× scale-up (from 0.53 m diameter to 1.6 m) with the same areal density was a bridge too far.It may be better to scale-up in steps of 2×.For AMSD, the scale-up was ∼2.25×.
• Avoid complexity.Complexity adds cost and schedule risk.It is much more difficult to mass produce 166 actuators than it is to build a single prototype, and one should expect up to a 30% initial failure rate.

Advanced Mirror System Demonstrator
The AMSD study was designed to explore the most likely NGST mirror technologies at an appropriate scale.Its success formed a basis for estimating Webb ambient and cryogenic performance, manufacturability, schedule, cost, and risk.Given the importance of large lightweight mirrors to many government missions, AMSD was a joint NASA and Department of Defense program.Although some mission requirements were divergent, the pooling of resources provided greater funding to explore the technology landscape more widely and deeply.
A critical element of the AMSD program was competition.Competition between ideas and vendors resulted in a rapid TRL advance of modern, large-aperture lightweight cryogenic space mirrors.AMSD followed a phased down-select approach.Phase 1 awarded contracts to five different vendors to study and develop designs for a total of eight different mirror architectures.1][22][23][24][25] All of these mirrors were 1.3 to 1.4-m point to point-just the size needed to produce a segmented primary mirror 6 to 8 m in diameter-and had an areal density of ∼15 kg∕m 2 .
1][22][23] A key element of the BATC approach is that the mirrors required cryo-null polishing to remove cool down cryodistortions.
Goodrich proposed two high-authority mirror concepts consisting of a face sheet (one concept was shallow-ribbed glass and the other was silicon carbide) supported on an array of displacement actuators.The displacement actuators would be used to correct for cool down distortion. 24The face sheets would be fabricated via stress polishing on a mandrel.Early in its design phase, because of projected cost and schedule overruns, NASA terminated the SiC concept.
Kodak (now L3-Harris) fabricated a semirigid mirror system that utilized a closed-back allglass cellular-core mirror along with a few force actuators to correct low-order mirror distortions that occur during cool down to cryogenic temperatures (see Fig. 3). 9,25,26The Kodak approach also assumed cryo-null polishing to remove both correctable and uncorrectable cryodeformations.
After Northrop Grumman (NGC) was selected as the Webb prime contractor in 2002, the Goodrich effort was terminated due to incompatibility with the NGC Webb architecture.This allowed the remaining funds to be focused on the BATC and Kodak mirrors.Both mirrors were successfully cryo-tested, and their cryogenic performance was characterized.Findings of the cryo-testing included the following: a properly designed beryllium mirror substrate will not have cryo-quilting; a properly designed mirror mount will not introduce low-order cryo-deformation; and cryo-deformation is the result of CTE nonuniformity in the mirror substrate.This author's personal lessons learned from AMSD phase 2 include the following.
• Plan for the unplanned: increase cost and schedule estimates by 50%.Maybe the most important lesson that this author learned was to calibrate my intuition regarding cost and schedule.In this author's opinion, everyone's proposed phase 2 cost and schedule seemed reasonable, except for Kodak's, which seemed unreasonably conservative.Well, everyone-including Kodak-overran both cost and schedule.But Kodak's overrun was the smallest.And maybe they would not have overrun at all if they had not broken their first mirror (a black swan event).So lesson learned, avoid your own optimism, and avoid being misled by other's optimistic thinking or deliberate false pretenses.To mitigate this risk, add 50% to any cost or schedule estimate.• Again, stiffness is more important than areal density.Standard fabrication processes must be revised for low-stiffness mirrors.Kodak broke their first mirror.The root cause was found to be using an inappropriate torque stress margin.And Goodrich, similar to Arizona before them, fractured their glass face sheet.• Intuition about how things work at ambient does not scale to cryogenic temperatures.
Goodrich wisely made a subscale pathfinder glass mirror and learned an important lesson.
Although their design worked fine at ambient, its cryo-performance was significantly degraded because of mismatches between its constituent materials modulus and CTE changes as a function of temperature.• CTE homogeneity is critical.The Kodak ULE ® mirror exhibited a significant and unexpected cryo-deformation.At first, it was believed to be mount stress.But, after removing the mount and testing the mirror "hanging" on a single point-it had the same cryo-deformation.The "agreed upon" root cause was a CTE "wood-grain" effect.ULE ® is a laminar material.Although its bulk CTE is near zero, the CTE of each layer is not zero.It was determined that the mirror's aspheric departure cut through multiple CTE layers, giving the front face-sheet a "wood-grain" CTE texture.Goodrich also had a CTE mismatch problem between their glass facesheet and stress polishing blocking body that resulted in a midspatial frequency error.• Plan for the unexpected statistical outlier-and again, do not rely on models.Space environment models for SE-L2 predicted the existence of small high velocity micrometeoroids.
To assess their potential affect, ULE ® and Be samples, as well as layers of sunshade material, were impacted with glass microspheres using the University of Auburn hypervelocity gas gun.On ULE ® , the affect was small fractures.On Be, the affect was an impact crater with localized melting/resolidification spalling.On the sunshade, the initial particle impact produced a spray of particles that penetrated subsequent layers.In all cases, the affects were deemed acceptable because the predicted probability of a large particle impact was once per 100 years.But the reality of space is different, and Webb has encountered larger, more energetic micrometeoroids than the models predicted-approximately one per month.
It is the assessment of this author that AMSD (and the broader NGST mirror technology development effort) was successful because of the following.
• AMSD had very clear specification and performance metrics that were traceable to the potential flight mission.Although these specifications eventually proved to be inappropriate for the flight mission, they focused technology development and enabled apples to apples comparisons.• The compliance of each competing mirror system was independently verified by the government team.• The entire technology development program was executed by a single organization and principal investigator.• The government team consisted of the best and brightest from multiple agencies and organizations.
• The competing contractors were treated as full members of the team.
• The government team had full insight into each contractor's efforts.
• Competition motivated the contracting teams to innovate technology solutions to achieve the required performance and programmatic objectives and, in my personal assessment, at a lower cost and faster completion than if there had not been competition.

Metrology Technology Development
In 1999, NGST had a problem.The SBMD mirror had been delivered and was being tested using an Adaptive Optical Associates Shack-Hartmann wavefront sensor-which did not have sufficient resolution and reproducibility to certify specification compliance.A method did not exist that could certify the technology development mirror's prescription (ROC and conic constant) and surface figure error at their cryogenic operating temperature inside the XRCF cryo-vacuum chamber (see Fig. 4).As the adage goes-you cannot make what you cannot measure.Because the mirror's radii of curvatures were long, their center of curvature was located outside of the vacuum chamber, and the mirrors were tested through a vacuum window.Because of this separation, the interferometer and mirrors had a relative piston motion of 4 μm-too large and too fast for a commercial temporal phase-shifting interferometer.
In November 1999, NASA hired this author as a principal investigator for the NGST Mirror Technology Development Program, in part, because of his then-current relevant experience refurbishing, aligning, and operating the 4-m 7-segment actively controlled LAMP mirror in a vacuum environment.(LAMP was part of the strategic defense initiate), but more for his  It may be self-serving to state, but the most important lesson to be learned is that there is no substitute for direct relevant experience, and if you do not have that person in your organization, you must find and hire them.

PhaseCAM
Testing long ROC mirrors and overcoming the limitations of atmospheric turbulence and mechanical motion had long been an interest of this author.One solution to this problem was phase-measuring interferometry (PMI) at 10.6 μm. 27The infrared wavelength was insensitive to temperature induced index variations in the atmosphere and small amplitude mechanical vibrations.But infrared interferometers lack visible wavelength sensitivity.Another solution (developed at Breault Research Organization) was high-speed PMI with a 340-Hz Reticon CCD camera. 28If the atmospheric turbulence and mechanical motion are sufficiently slow, they can be sampled and removed via averaging-as long as the sampling interval was longer than the atmosphere's random-walk correlation time.
In 1987, Keck contracted with BRO to use this high-speed PMI system to test their primary mirror segments at ITEK.Because of their mirror's long focal length and off-axis near-parabolic optical prescription, the total optical test air-path for each segment was 48 m and required five reflective bounces-twice off mirror segment and three times off the autocollimation flat (ACF).This air path was too long for ITEK's LUPI (laser unequal path interferometer) or a commercial Fizeau temporal phase-shifting interferometer, and there was too much mechanical motion.The high-speed camera system helped, but the ultimate solution was a pseudo common-path test setup that this author had seen in Norm Cole's optical shop.The mechanical motions were mitigated by rotating the LUPI reference flat 45 deg to send the reference beam along the same path as the test beam (bouncing off the ACF and a flat attached to the segment test stand).Atmospheric turbulence was mitigated by averaging 64 measurements taken on a multiminute cadence to avoid atmospheric turbulence "random-walk" correlation. 28In the process, another problem was discovered, a problem that would be critical for absolutely characterizing the LIGO reference flats and for testing the Webb segments.Because the CCD camera frame capture was not instantaneous, laser frequency drift introduced a phase shift error from the start of the frame readout to the end.Now for serendipity, before joining NASA, while visiting MetroLaser on another matter, this author saw on a table in a back lab a breadboard setup of a "real-time" interferometer producing a phase map of a flame plume.Its potential was immediately obvious, and Bernie Seery, Pre-Phase-A Study Manager, agreed to fund a risk reduction experiment.Newly incorporated 4D Vision Technology was given a $60 K contract to build an interferometer to NASA's specification.They delivered the first ever PhaseCAM (Fig 5) in just 6 months and it worked great.Its resolution was 512 × 512; its repeatability was 1.2 nm rms; and over a 20-m air path, its measurement uncertainty was 5 nm rms.It was put into use immediately, and Webb could not have been made without the 4D PhaseCAM technology.

Absolute Distance Meter
Regarding measuring the ROC, a common technique is to use a distance measuring interferometer on a lens bench to measure the radius.But this technique would not work for the NGST mirrors because it requires a displacement measurement from "cats-eye" to the center of curvature.During a NASA fact-finding trip to SSG Walthan MA, a Leica theodolite was observed with an interesting distance measuring technology, but its accuracy was only AE5 mm, and the NGST specification was AE0.1 mm.So a development effort with Leica was funded and resulted in the absolute distance meter (ADM).The ADM was used to measure and set the ROC on all development and flight mirrors.(And maybe its funding led to Leica's commercial DISTO handheld laser distance measuring devices.)

Primary Mirror Design Iteration
In 2002, the Optical Telescope Element (OTE) aperture diameter was reduced from 8 to 6 m.This decision was made primarily for cost reasons but also, based on lessons learned from AMSD, to increase the Primary Mirror Segment Assembly (PMSA)'s areal density from 15 to 26 kg∕m 2to better survive launch and to improve manufacturability.
This architecture change initiated a design iteration that significantly improved the primary mirror architecture.The original 8-m primary mirror had 36 segments with rigid body actuation.When the aperture was reduced to 25 m 2 (i.e., 6 m), there was a trade between having 36 smaller segments (still with rigid body actuation) or having 18 larger segments with hexapod actuation.This author remembers Lee Feinberg (Webb OTA manager) deciding on the larger segments with hexapods based on "part count."But the real value of the hexapod came during fabrication of PMSAs at Tinsley.Because of uncertainty in locating each PMSA in the parent prescription space, there was always some residual astigmatic surface error.Having the hexapods and an edge gap allocation allowed for PMSAs to be adjusted in the parent space to minimize wavefront astigmatism-both during testing and on-orbit.

Webb Primary Mirror Selection
AMSD phase-3 funded two competing studies to design candidate flight primary mirrors, perform production planning, and generate cost/schedule proposals.The materials evaluated were O-30 Beryllium and ULE ® glass.The Beryllium team consisted of Ball Aerospace, Brush-Wellman, AXSYS, Tinsley, and ATK-COI.The ULE ® glass team consisted of Kodak, Corning, and ATK-COI.
A Mirror Recommendation Board (MRB) was established to evaluate the competing proposals.The MRB consisted of a balanced membership with representatives from NGC, Ball, Kodak, and NASA.The MRB also included consultants with extensive experience in technical and programmatic issues associated with optical design, analysis, manufacturing, and testing.The MRB defined evaluation criteria and key discriminators.Subcommittees were formed to review vendor data in the areas of technical performance, cost, schedule, facility, and staffing.The MRB met 3 times in the spring and summer of 2003.The final selection was briefed at the OTE Optical Readiness (OOR) review on September 9, 2003.
It was the OOR Review Panel's assessment that AMSD successfully raised both mirror technologies to TRL 5.5, reduced technical (weight and performance) and programmatic (schedule and cost) risks by fabricating full-scale mirror systems, and validated their thermal wavefront performance under flight-like operational conditions. 11,29he Ball beryllium mirror was selected for flight.Beryllium was rated as the highest performing, lowest technical risk solution.Its cited strengths included superior cryogenic CTE and thermal conductivity; significant margins on thermal performance, stiffness, and mass; and its excellent potential science performance.Specific concerns included managing surface stress to achieve convergence to the required final surface figure and manufacturing schedule.A key selection discriminator was the thermal stability of the beryllium mirror over the 30 to 50 K operating range.
Although the MRB found that ULE ® glass has significant programmatic advantages, this strength was offset by concern regarding the uncertainty about how ULE ® CTE variability impacts the thermal performance of lightweight cryogenic mirrors.Suitability of ULE ® would not have been fully proven until completion of the EDU in 2005.Sixteen of eighteen MRB members scored Beryllium higher than ULE ® . 29ext, AMSD-3 initiated the manufacture of an EDU, which was used for vibration and acoustic testing needed to achieve TRL-6.
A personal lesson learned for this author was how important open competition and the MRB/ OOR process were for building a clear unimpeachable consensus decision as to the best primary mirror architecture to take into the flight program.Another lesson was a reinforcement of the importance of competition for reducing cost.When each team presented their flight mirror proposal, each offered contract incentives (i.e., cost sharing via infrastructure investment) that exceeded the $3 M increment cost of redesigning a second mirror under phase 3.

Performance Subcommittee
Vendor primary mirror design concepts were evaluated for their impact on the Webb OTE level 2 performance requirements.Technical performance criteria were divided into two categories: mandatory and secondary.Mandatory criteria were defined as mirror performance factors that influence the ability of the OTE to meet Webb level 2 requirements.Secondary criteria were other factors that influenced OTE performance but were not directly traceable to level 2 requirements.Each mirror concept was characterized as to its ability to exceed, meet, or not meet mandatory evaluation criteria (see Table 2).
The Performance Subcommittee assessed that meeting key Webb Level 2 requirements would be very challenging but that the beryllium mirror provided significant performance advantages for Webb.
Regarding the science mission timeline, both mirrors were assumed to be able to satisfy the 10-year requirement.The only difference between them was that, if the end-of-life temperature of the OTE is 4 K different than the beginning of life temperature, then the ULE ® mirror's wavefront error (WFE) would degrade ∼8 nm rms.Regarding launch survival, it was assumed that both mirrors could be designed for the required vibroacoustic environment.
Regarding the collecting area, Webb requires each mirror segment to be polished to within 5 mm of the physical aperture.Both mirror teams had some difficulty with this on AMSD, but the ULE ® team had slightly more difficulty.
Compliance with the optical performance parameters was derived based on AMSD ambient and cryogenic test results.To facilitate a head-to-head comparison, a specific protocol was defined to ensure that all data were analyzed identically.This protocol defined in the data flow how to correct for CGH distortion; remove low-order aberrations; compensate for gravity sag; mask, clip and threshold the data; remove test setup induced misalignment aberration; and how much real cryo-deformation aberration would be allowed to be removed based on a limited ability to compensate for such errors by moving mirror segments on-orbit via a hexapod mount.
Based on AMSD results, neither mirror team predicted an OTE WFE of <117 nm rms.The Be team predicted 118 nm rms and the ULE ® team predicted 119 nm rms.However, both vendor teams subsequently figured their AMSD mirrors to a surface quality sufficient to meet this requirement.
A fundamental difference between the Be and ULE ® mirrors were their cryogenic distortion, i.e., mirror shape change from ambient to 30 K. AMSD showed that the ULE ® mirror experienced a larger cryo-distortion than the Be mirror.The total ambient to cryogenic figure change for the Be mirror was 171 nm rms and for the ULE ® mirror was 398 nm rms.As shown in Fig. 6, after performing a simulated hexapod adjustment to remove alignment aberrations, the Be mirror change was 77 nm rms, and the ULE ® mirror change was 188 nm rms.After removing 36 Zernike coefficients, the high spatial frequency residual error is 26 AE 2 nm rms for Be and 47 AE 9 nm rms for ULE ® .The ULE ® mirror exhibited obvious print through from its core structure.Although it is possible to remove cryo-deformation via cryo-null figuring, the larger magnitude of the ULE ® deformation, as well as its print through, posed more of a risk than that of the Be mirror.Also because this deformation was only recently identified as a result of the AMSD project, there was additional risk that the magnitude and sign of this cryogenic deformation could vary from mirror to mirror.
Another significant difference between the two mirrors was their thermal stability and sensitivity to a thermal operating set point.This sensitivity impacts encircled energy (EE) stability and on-orbit performance as a function of time.Over the anticipated operational temperature range of 30 to 55 K, the Be AMSD mirror experienced a 7 nm rms total surface figure change, whereas the ULE ® mirror's figure changed by 40 nm rms.Note that the Be change was at or below the measurement sensitivity.As shown in Fig. 7, after removing alignment aberrations, the ULE ® change dropped to 21 nm rms.After removing the first 36 Zernikes, the figure error for Be was 1.6 nm rms and for ULE ® was 4.6 nm rms.Using the measured AMSD optical performance as the basis, the Performance Subcommittee derived the OTE optical performance as measured by the Strehl ratio, point spread function (PSF), EE, and EE stability.Both mirrors predict an excellent Strehl ratio with a substantial margin (Be > 93.9% and ULE ® > 92.8%).The predicted EE for Be was >74%, and the predicted EE for ULE ® was >72%.However, because of the thermal stability of Be, it has a better EE stability over the entire thermal operating range and potential thermal gradient conditions.This was important because EE stability was a financial bonus performance incentive parameter in the prime contract.
Secondary evaluation criteria for each mirror concept were characterized as to its risk (high, moderate, or low) of impacting OTE performance (see Table 3).
Regarding design issues, all Beryllium design features for Webb (except for the segment size) were equal to or lower risk than what was demonstrated on AMSD.The Be Webb design had fewer pockets than AMSD with thicker ribs, a thicker face sheet, larger corner radii, and larger fillet radii.The ULE ® mirror had several parameters in its design that were higher risk than what was demonstrated on AMSD.In addition to a larger segment size, it would have a thinner front and back plate thickness, larger core depth, and a thinner edge ring.Mass was explicitly excluded as a selection factor, but at the time of the evaluation process, the Be mirror design was 32 kg over budget, and ULE ® would have been 176 kg over budget.
Finally, a new process was proposed for inspecting raw ULE ® glass before it was fabricated into mirrors.The purpose of this process was to mitigate the mechanism thought to explain the thermal deformation effect observed on AMSD.The subcommittee found both the mechanism and new process credible but unproven.Stahl: Webb space telescope primary mirror development: summary and lessons learned The Performance Subcommittee recommended Beryllium for Webb based on several factors.It was expected to meet all Webb level 2 requirements.AMSD successfully demonstrated most of the critical technology issues needed to scale up to Webb, and Webb design improvements would make the segments more producible with lower risk.The anticipated cryogenic deformation was within the range of what could be cryo-null figured.Beryllium's excellent thermal properties provided a stable mirror performance over the entire Webb operating temperature range.The subcommittee's findings on ULE ® were that its AMSD cryogenic behavior was not predictable, the mechanism for that behavior was unproven, and its thermal sensitivity could pose a risk to on-orbit optical performance.However, it was also the assessment of the subcommittee that these findings apply only to cryogenic operation and that all AMSD data support the fact that ULE ® is an excellent mirror material for ambient applications.

Manufacturing Process and Test Plan Subcommittee
Each vendor provided detailed schedules for an EDU, 18 flight segments, and 2 spare blanks with 100s to 1000s of elements.Each schedule included critical resource allocations and basis of estimates.AMSD was assumed as the basis for all operations, and detailed traceability matrices were provided to justify all Webb processes.Vendors were instructed to provide detailed justification for any process durations different from AMSD.
From the schedules, the subcommittee selected four critical milestones for assessment: EDU vibe test, EDU completion, first segment, and last segment.EDU structural and acoustic testing were selected to ensure that this critical milestone is accomplished before the nonadvocate review (NAR).EDU completion was a key selection for ensuring that all production steps were fully demonstrated before they were needed on flight hardware.The first primary mirror segment was selected because it was required for the OTE Pathfinder risk reduction activity.The last segment was selected because it defined the critical path to launch.
The manufacturing process and test plans were assessed for adequacy in terms of detail and thoroughness of understanding and adequacy of required equipment.The schedules were assessed for contingency, robustness, and flexibility.Slack to completion was evaluated for each of the four identified milestones.Available workarounds were considered, and a critical path chain analysis was performed.Each schedule was assessed for credibility and risk.The adequacy of justification for the differences between Webb and AMSD was assessed.Risk and mitigation plans were evaluated using Webb project criteria/guidelines.
Using each vendor's schedule, a comparative probabilistic schedule risk assessment was performed using Risk+© software.The assessment was based on schedule pessimistic and optimistic durations.Pessimistic assessment was based on interrogating the traceability matrix.Optimistic assessment was based on the vendor's identified process improvements.The assessment indicated Beryllium had 7% to 15% more schedule risk than ULE ® (see Fig. 8).The manufacturing process and test plan subcommittee had three findings.First, Beryllium had fewer new or modified steps from AMSD in its proposal than ULE ® .Second, the ULE ® schedule had more slack in the identified program milestones, and its processing flow was appreciably more immune to large perturbations without impacting the Webb critical path.Third, both vendor's schedules were optimistic and represented risk to the program.The probabilistic risk assessment indicated that Beryllium represented a 7% to 15% greater schedule risk than ULE ® .

Facility and Staffing
The facility and staffing subcommittee assessed that there were no major challenges for the ULE ® team and that the biggest challenge for the Be team would be setting up the polishing plant.The biggest equipment challenge for the Be team would be getting the first CNC machine on-line, whereas the biggest challenge for the ULE ® team would be getting small tool machines on-line.

Cost Subcommittee
Both vendors provided detailed cost proposals.After adjusting both proposals based on an indepth review of their basis of estimates, the cost subcommittee found little difference in either proposal's risk or content.Both vendors proposed a recurring cost that was 35% lower than one might predict using a simple AMSD extrapolation.Additionally, both vendors proposed substantial internal investment.It was the assessment of the subcommittee that much if not all of the cost of the AMSD mirror technology development program was recovered by these cost savings.Finally, based on the relatively greater schedule risk of 7% to 15% for Be, it was assessed that Be had a larger risk of cost overrun than ULE ® .However, this risk was offset by the fact that 45% of the Be proposal was a firm fixed price.

Science Subcommittee
The science subcommittee assessed that Beryllium would be the highest performing, lowest technical risk solution.It had superior cryogenic CTE and thermal conductivity-thus providing significant optical performance margin in the event of thermal gradients and bulk temperature set point uncertainty.It also was more forgiving of differences between the thermal conditions for on-orbit and ground testing.The Beryllium mirror would provide, with margin, a PSF that meets both the EE and EE stability requirements.

Engineering Development Unit
All flight programs have EDUs, but in the case of Webb, the PMSA EDU was critical.Based on lessons learned from AMSD, the Webb flight PMSA design was modified to improve the producibility, performance, launch survival, and risk (see Table 4).Because of these design changes, the fabrication process developed on AMSD needed to be modified and requalified.
Ideally and in accordance with the National Research Council report on controlling NASA space mission cost growth, 30 it would have been nice to demonstrate TRL-6 compliance on a single-mirror system before entering phase C/D, but that did not happen.AMSD ran out of both time and money, so TRL-6 was demonstrated piecewise.Furthermore, because the flight mirrors were long lead items on the critical path, it was necessary to start their production in phase B. But as well meaning as this decision was, there was a problem.Because of the length of the mirror fabrication process, too much time had passed.By the time that the OOR review panel had selected the flight mirror configuration, it had been several years since Brush-Wellman had manufactured a mirror blank, AXSYS had machined a substrate, or Tinsley had performed rough grinding.This is a problem because of the forgetting curve.Just as there is a learning curve (which reduces the cost and schedule of similar items by up to 30%), there is also a forgetting curve.(First formulated in 1880, Ebbinghaus determined that humans forget information exponentially with time.Subsequent studies confirm that, without repetition, humans forget 90% of their training within 1 month. 31This author has heard it said that organizations forget half of their corporate knowledge every 6 months.)Consequently, because of changes to the Webb flight PMSA design and forgetting, the entire fabrication process had to be relearned and revalidated on the EDU.Because of this relearning, the EDU underwent a fabrication process that was not only different from AMSD but also different from the subsequent flight mirrors.In fact, the flight fabrication process did not become truly reproducible until flight mirror #3.One example is that Tinsley's polishing compound supplier changed their compound's formula.
Additionally, just as it is important to avoid large gaps in a process to prevent forgetting, it is also important to avoid getting the subsequent flight mirrors too close to the EDU.Otherwise, it is impossible to fully implement the lessons learned from the EDU with the flight mirrors.A recommendation for any future ground or space telescope that uses multiple primary mirror segments is to process more than one EDU or manufacture the flight spares first.The purpose is to ensure that no process is ever performed for the first time on a flight mirror and that flight production begins once the process is stable.If the spares meet all requirement specifications, they can always be promoted to flight status.Some personal lessons learned from the EDU are as follows.
• Polishing a 1.5-m class mirror to within 5 mm of its physical edge is very difficult.It is possible to misregister the edge by 25 to 50 mm for a number of reasons, including the combination of image distortion when testing a mirror at center of curvature with a CGH; the effect of retrace errors when light from a rolled edge travels 16 m back to the center of curvature; or the effect of Fresnel diffraction from out of focus edges coherently summing with the surface wavefront.Extensive fiducialization is important to knowing where to small-tool polish on the mirror.• Plan for forgetting.Manufacture the EDU and flight spares before making flight units.
Because of the "forgetting" curve, fabrication and testing processes need to be relearned before making flight articles.Also during the flight production, situations in which personnel forget the process steps arose, and it was necessary to stop work for a day and retrain everyone on how to follow the process.• There is no substitute for government insight/oversight experience.Contractor personal come and go; it is the responsibility of the government insight/oversight team to ensure compliance with all specifications.To do this, the team needs to consist of persons with direct relevant experience.One example was understanding how Fresnel diffraction impacts the ability to correctly measure a mirror's edge and the consequences of CGH pupil distortion.

TRL-6 Certification
A central requirement of the mirror technology development program was to mature the TRL for mirror technology critical to Webb from the pre-1996 TRL-3 level to a level of TRL-6 for review by a technical NAR panel.Assessment of TRL-6 by the TNAR had to occur before the Webb OTA could undergo its critical design audit.This gate was achieved on January 31, 2007.The process used to certify that the Webb mirror technology was at TRL-6 was systematic and rigorous. 12,13It was accomplished by defining a set of critical technology capabilities (which flowed directly from the level 1 science requirements) that had to be demonstrated under relevant flight conditions and then demonstrate that compliance.Demonstration of compliance was accomplished piecewise using SBMD, AMSD, flight mirrors, and test coupons.

Requirements Flowdown
Technology requirements for the PMSA were derived from level 1 science requirements through level 2 mission requirements and level 3 observatory requirements (see Table 5).Level 1 science requirements were defined in the Webb Program Plan. 32Level 2 mission requirements were defined in the Webb Mission Requirements Document. 33Level 3 observatory requirements and specific mirror technology component requirements were derived during the phase 2 NGST Observatory Contract and refined after the Prime Contractor (and Implementation Team) was selected.Complete PMSA requirements are defined in the Equipment Specification for Webb PMSA. 34 comparison of these PMSA requirements with the pre-Webb state-of-the-art for space telescopes, as defined by Hubble and Spitzer (see Table 6), clearly showed that they were truly well beyond the state-of-the-art.Thus these capabilities were TRL-6 technologies that needed to be demonstrated.
Although there were literally 100's of engineering specifications necessary to manufacture a Webb PMSA, only a select few were considered technology requiring demonstration: gold coating cryo-survivability, figure thermal stability, areal density, figure launch distortion, primary mirror optical area, surface figure error (including ROC, hexapod, creep and polishing error), and cryogenic performance.Note that this list is not in a priority order, but in the order of their flow down from the level 1 science requirements developed in Tables 5 and 6.The balance of this section details the system engineering logic of how each mirror technology requirement flows from its originating level 1 science requirement.
Although the observatory operating temperature was listed as a key technology, it was really an existence principle.It is the one requirement that pervades all other requirements.To achieve the level 1 science requirement of providing a thermal environment that permits the science instruments to have zodiacal light background limited imaging performance over the wavelength range from 1.7 to 10 μm, the observatory must limit its thermal emissions by operating at a cryogenic temperature of <50 K.This directly drives the need to place the telescope at L2, which requires an EELV launch vehicle that demands low areal density mirror segments.This requirement also directly drove all operational thermal requirements, including performance, survival, and stability.Thermal modeling indicated that some of PMSAs might be as cold as 28 K.
Gold coating cryogenic survivability was a relatively minor TRL-6 technology.Level 1 science requirements specified a spectral range of 0.6 to 27 μm.This, in combination with sensitivity, flowed into a level 2 optical transmission requirement, which directly flowed into a PMSA reflectivity requirement.Uncoated polished Beryllium cannot achieve the required reflectivity over the required spectral range.Overcoats of gold, silver, and aluminum were considered.Gold was the best candidate material.It provides excellent reflectivity in the near-and mid-infrared and acceptable performance in the visible.Silver does provide better performance in the visible, but it requires a protective layer to avoid oxidation problems.Aluminum, although common for ground-based visible telescopes, does not have acceptable infrared performance.Gold is a common coating material and thus was not itself a TRL-6 technology.But the cryogenic survival of a gold coating applied to a large O-30 Beryllium mirror had never been demonstrated.
The PMSA surface figure thermal stability was possibly the most important TRL-6 technology and was a key factor in selecting Beryllium as the primary mirror material. 11,33Level 1 specified that science observations must be able to occur at any position in the celestial sphere.This placed a stability requirement on the EE as the observatory slews, which in practice was a constraint on how much the PSF shape could change due to thermal gradients introduced into the telescope as a function of angle to the sun.At the PMSA level, EE thermal stability is directly determined by the thermal stability of the surface figure shape.Although dozens of engineering issues can contribute to this stability (such as material CTE uniformity and structural design, including actuator athermalization bracket design and bimetallic effects), it was the system level PMSA performance that is the TRL-6 technology.Thus a specific PMSA design implementation must be demonstrated to have cryogenic figure stability of <0.3 nm rms per K, which manifested itself as a maximum surface figure change of 7.5 nm rms from 30 to 55 K.
PMSA areal density was one of the two key technologies identified as requiring significant development effort.The level 1 science requirement of operating the observatory at L2 flowed down to a level 2 requirement that the observatory must be launched via a heavy lift rocket (such as an Areianespace Ariane 5).This placed a mass constraint of 6159 kg on the observatory.The original primary mirror allocation of this mass was 1000 kg, and given that the original telescope collecting area was to be 50 m 2 , this placed an areal density requirement on the primary mirror of 20 kg∕m 2 .To provide margin, a technology goal of 15 kg∕m 2 was defined.It was this goal that drove the entire mirror technology development program.As the observatory architecture evolved and mass maturity of different observatory elements improved, the PMSA areal density specification was raised to 26.5 kg∕m 2 .
The PMSA diameter was the second key technology identified as requiring significant development effort.Originally, an 8 m class primary mirror was required to achieve the desired observatory sensitivity.Given that the observatory needed to operate at L2, that the only way to get to L2 was to be launched on a heavy lift rocket, and that the maximum available shroud diameter was only 4.5 m, it was clear that a segmented and deployed architecture was required.Competing design solutions required segments with diameters ranging from 1 to 3 m.Groundbased observatories (Keck, Hobby-Eberly) and test beds (LAMP, ALOT, and AOSD) had demonstrated the ability to produce segmented telescopes, but their areal densities were too high (70 to 2000 kg∕m 2 ).Thus a primary focus of the mirror technology development effort was on how to manufacture 1 to 3 m class mirror systems with the required areal density.A key task was to design and demonstrate a substrate that could be manufactured, safely handled, optically finished including ground testing, and integrated into a system that would survive launch-all with an areal density <20 kg∕m 2 .A second issue was the ability to manufacture the substrate blank.Pre-Webb, all large mirrors were glass, which, although acceptable for ambient operation, were less than ideal for a cryogenic telescope.The largest cryo-mirror was the ITTT 0.85-m I-70 Be mirror.Hence, the AMSD program was tasked with demonstrating the ability to manufacture a 1.5-m class O-30 Beryllium mirror blank-as well as the entire mirror system.The Webb PMSA diameter of 1.5-m, which is slightly larger than what was demonstrated on AMSD, was derived from a combination of the level 1 science requirement to have a minimum of 25 m 2 of unobscured optical collecting area and the choice of an 18 segment architecture.
PMSA cryogenic surface figure, creep, launch distortion, and adjustability requirements were derived from performance metrics directly traceable to the level 1 science requirement that the observatory shall be diffraction limited at 2 μm.To achieve the level 1 requirement, the telescope was required to have a residual WFE of <131 nm rms after fixing correctable on-orbit figure errors.To "fix" correctable errors, each PMSA has the ability (at temperatures < 50 K) to change its ROC and adjust its rigid body position.Detailed error budgeting by Ball Aerospace partitioned the residual WFE between multiple sources, including uncorrectable residual PMSA surface figure error; errors in the ability to adjust all PMSA's to a common ROC; errors in the ability to phase all PMSA's into a common primary mirror by correcting PMSA rigid body errors; creep of a PMSA figure as a function of time; and figure change experienced by a PMSA as a function of the launch environment.The result of this process was that each PMSA had to be able to adjust its ROC with a resolution of ≤10 nm PV sag, and each PMSA had to be able to adjust is piston position with a resolution of ≤10 nm.Uncorrectable PMSA cryogenic surface figure error-i.e., errors that cannot be corrected by ROC adjustment or sliding the PMSA in the "parent" space with the hexapod-had to be ≤23.7 nm rms at delivery to the OTE integration and test (I&T) process.Also, from the time that a PMSA is delivered for I&T through its end of life, the uncorrected surface figure error from material creep had to be ≤1.8 nm rms.Furthermore, the PMSA uncorrectable surface figure distortion due to launch had to be ≤2.9 nm rms.
An interesting detail of these requirements was the role of material stress/strain and precision elastic limit (PEL) on PMSA design and their connection with figure creep, launch deformation, surface figure error, and areal density.To meet the creep and launch figure change requirements, it was critical that the PMSA substrate had sufficient stiffness to avoid introducing excessive stress/strain into the mirror during optical fabrication.It is the release of this stress/strain from the mirror with time or exposure to the launch environment (vibration and acoustic) that causes undesired figure change.PMSA stiffness is also important for in-process optical testing and I&T; a mirror must have sufficiently small gravity sag that it can be accurately measured in one-g (i.e., on the Earth) while being manufactured for optimized performance in zero-g.So although AMSD demonstrated that an areal density <20 kg∕m 2 was achievable, a specification of 26.5 kg∕m 2 was necessary to produce a PMSA with sufficient stiffness to meet the other requirements.

Requirements Verification
To certify compliance with TRL-6, specific success criteria were established for each critical technology and then confirmed by test.Table 7 lists the critical requirements, their success criteria, and how each was confirmed by test.
All PMSA technologies necessary to meet the Webb level 1 requirements were demonstrated to be at TRL-6 via a piecewise methodology.As desirable and recommendable as it might be, TRL-6 was not demonstrated on a single-mirror system.Rather, SBMD, AMSD, flight mirrors, and test coupons were used to mature specific technologies and demonstrate their performance in a relevant environment.For example, SBMD demonstrated gold coating performance at 28 K and cryo-null figuring.Although AMSD was designed to explore fabrication limits associated with areal density and size, it could not certify everything.AMSD did produce a complete mirror system (with a design that is traceable to flight) and tested its performance in a relevant environment from 30 to 50 K.But AMSD was not designed to (nor was it ever intended to) meet Webb launch loads.To survive launch, Webb flight PMSAs were redesigned to have significantly more areal density than AMSD (which made a Webb PMSA easier to fabricate).Additionally, Webb's flight PMSA design was modified based on lessons learned from AMSD.Therefore, it was necessary to use an actual Webb flight PMSA for vibration and acoustic testing.
The PMSA-110 and PMSA-530 ability of a gold coating to survive the 28 K requirement was verified with SBMD.TRL-6 was demonstrated by performance testing at 30 K and survival testing to 28 K a gold coating deposited on the SBMD mirror.Because cryogenic adhesion of gold on O-30 beryllium was the ability being tested and not the ability to deposit gold coatings on to large mirrors, it was determined that repeating the test with a gold coated AMSD mirror was unnecessary.The deposited gold coating introduced no discernible cryogenic surface figure distortion into SBMD.The uncoated SBMD's 30 K surface figure was 52.8 nm rms, and its coated 30 K surface figure was 53.9 nm rms 35 (Fig. 9).
The PMSA-530 requirement that a PMSA could operate over a 28 to 50 K temperature range was verified with the AMSD mirror system and the Webb flight actuators.TRL-6 was demonstrated by testing the AMSD beryllium mirror system multiple times over operational temperatures from 28 to 50 K to characterize its cryogenic performance.Cryogenic figure stability was characterized.The cryogenic figure and ROC change were demonstrated, and the cryogenic ROC adjustability was demonstrated.TRL-6 was further demonstrated by testing the cryogenic performance of the Webb flight actuators.
The PMSA-170 requirement that a PMSA maintains a surface figure stability of <0.3 nm rms for a 1 K temperature change (7.5 nm rms over a 30 to 55 K thermal range) was verified with AMSD.TRL-6 was demonstrated by measuring the surface shape of the AMSD beryllium mirror system as a function of temperature.The cryogenic surface figure was measured at multiple temperatures and was found to change linearly with temperature.The total surface figure change from 30 to 55 K was 7.0 nm rms or 0.28 nm rms per 1 K temperature change 11,28 (Fig. 10).
PMSA-410 and PMSA-70 derived requirement that a PMSA could be manufactured with an areal density <26.5 kg∕m 2 was verified with AMSD and confirmed with Webb flight segments.TRL-6 was demonstrated by calculating the areal density of the AMSD beryllium mirror and an assembled PMSA (Fig. 11) from measurements of their respective masses and physical dimensions.The achieved areal density for the PMSA was 25.8 kg∕m 2 .AMSD actually demonstrated the feasibility of manufacturing a mirror system with an areal density of 15.6 kg∕m 2 .This was achieved by CNC machining a beryllium mirror substrate with exceptionally thin ribs and facesheet while controlling the introduction of residual stress.Residual stress is very important.It can adversely affect the ability to polish a beryllium mirror to the required surface figure and keep that shape because of long-term figure creep.The higher PMSA areal density requirement (allowed by design maturity, incorporating lessons learned from AMSD and validated with improved modeling) improves manufacturability and reduces risk.The PMSA-70 requirement that a PMSA could be manufactured with a polished surface area of larger than 1.46 m 2 was verified via a combination of SBMD, AMSD, and EDU.TRL-6 was verified by three specific demonstrations of fact.First, the Webb flight program successfully manufacturing and machined a 1.315 m flat to flat beryllium substrate.Although this may seem trivial now, before the mirror technology development program, there was great uncertainty as to whether or not the manufacture of beryllium substrates of that size was even feasible.Second, AMSD demonstrated the ability to fabricate a 1.2 m flat to flat polished beryllium mirror with a mechanical design and aspheric prescription traceable to Webb.Until it was surpassed by Webb, AMSD was the largest diameter beryllium mirror ever fabricated.Third, SBMD demonstrated the ability to use small tool polishing on a lightweight mirror substrate to within 5 mm of a straight edge.
The PMSA-150 requirement that a PMSA could be polished with an uncorrectable surface figure error of <23.7 nm rms was verified with SBMD and AMSD.TRL-6 was confirmed by verifying two key abilities: (1) the ability to polish a large-aperture low-areal-density aspheric O-30 beryllium mirror to the required specification and (2) the ability to cryo-null figure an O-30 beryllium mirror to have the required figure specification at temperatures <50 K.The ability to polish a meter-class highly aspheric lightweight O-30 beryllium mirror was demonstrated on AMSD.AMSD was polished to have an uncorrectable surface figure error of 19.2 nm rms over 97.1% of its aperture (Fig. 12).Achieving a <20 nm rms surface figure was actually the last major task of the AMSD program, and its accomplishment represented a never before demonstrated capability for meter-class lightweight beryllium mirrors.Furthermore, because AMSD had a 10 m ROC, it was a more difficult prescription to polish than Webb segments with their  16 m ROC.The <20 nm rms uncorrectable surface figure was achieved via a small tool computer controlled optical surfacing (CCOS) technology at Tinsley Laboratories in Richmond, CA.Critical to this accomplishment was highly spatial sampled data and precision fiducial registration knowledge.The ability to cryo-null figure such a mirror to yield the required surface figure error at cryogenic temperatures was demonstrated on SBMD.SBMD exhibited a cryo-deformation of ∼90 nm rms.This shape change consisted of a low-order mount induced error and a highorder quilting error associated with the substrate rib structure.After two cryo-cycles proved that the deformation was stable and repeatable, i.e., that the O-30 beryllium mirror had no apparent creep induced figure change associated with residual stress in the mirror, SBMD was cryo-null figured.The predicted final cryogenic surface figure was 14.4 nm rms.The actual final cryogenic surface error was 18.8 nm rms 35 (Fig. 13).
Based upon the SBMD success of cryo-null figuring via small tool CCOS technology of both low-order mount induced as well as high-order rib structure quilting, it was determined unnecessary to cryo-null figure AMSD.
The PMSA-370 requirement that a PMSA could be positioned in space with 6 degrees of freedom (DOF) with <10 nm step resolution was verified with AMSD and PMSA components.TRL-6 was demonstrated by test of the PMSA actuator performance at 30 K and analysis of PMSA hexapod motion at 30 K. The Webb cryogenic hexapod mechanism, with its six cryogenic actuators, controlled the 6 DOF position of a mirror segment relative to the Webb telescope backing structure.A seventh actuator was used to deflect the center of the mirror, changing the ROC for that segment.Although the use of a hexapod was not new technology, the actuator step size resolution required at cryogenic temperature was.To meet the hexapod motion resolution  and accuracy requirements, the Webb actuators had to be independently capable of <10 nm step size resolution at <50 K.This level of motion resolution was achieved when the Webb actuators were operated in their "fine" mode.Webb actuators are dual stage with coarse and fine operating modes.The Webb actuators were developed by BATC, initially under IRAD funding, and then via AMSD, to meet specific mass, stiffness, and performance requirements.These actuators were used for both PMSA hexapod and ROC adjustments.
The key component of the actuator is a cryogenic capable geared stepper motor, which was derived from the gear motor flown on the Spitzer Space Telescope and operated at 4.5 K. TRL-6 capability was demonstrated by characterizing the cryogenic performance from 25 to 35 K of over 24 actuators: 2 actuators via Ball IRAD, 4 actuators via AMSD, 36 and 18 Webb engineering unit actuators.All actuators met the resolution requirement with the Webb engineering unit actuators showing a resolution of 7 nm (Fig. 14).Extensive testing of the actuators through a variety of fine-stage step increments verified that the actuator performs single steps, without backlash, to an accuracy of 0.6 nm rms.Finally, flight actuators were installed into a flight hexapod system and exercised at ambient temperature to show basic functionality.
The PMSA-1560 requirement that a PMSA cryogenic ROC sag could be adjusted by <10 nm peak-to-valley (pv) was verified with AMSD.TRL-6 was demonstrated by test, analysis, and corollary.PMSA mirrors were designed to adjust their ROC at cryogenic temperatures by expanding or contracting a linear actuator.The actuator, attached to the back center of the mirror, reacts its force via six struts that attach to each mirror corner through a flexured joint.A similar design was implemented on AMSD except that the actuator reacted its force against spreader bars (Fig. 11).Once at 30 K, the AMSD actuator was commanded to execute "coarse-steps" until an ROC sag change was detected.A move of 35 coarse steps resulted in an ROC sag change of 38 nm pv.By analysis, a single AMSD "coarse-step" should result in a sag change of ∼1.1 nm pv.And a single "fine-step" motion (which is 4.5 times smaller than a coarse step) should result in an ROC sag change of ∼0.24 nm pv. 28Because of the difference between where the actuator force reacts against the mirror substrate, the distance between those reaction points, and the intrinsic stiffness between AMSD and Webb, a Webb PMSA experiences an ROC sag change that is ∼110% larger per linear motion than AMSD experiences.Thus the minimum Webb coarse step is ∼1.2 nm pv, and the minimum fine step is ∼0.27 nm pv.
The PMSA-195 requirement that a PMSA could be designed such that its surface figure changes by <1.8 nm rms because of creep was verified with SBMD, AMSD, and Webb flight segments.TRL-6 was demonstrated by test and analysis.Funded via AMSD, Draper Laboratory measured the creep properties of O-30 beryllium. 37Significant creep was measured for samples stressed to 4 and 6 ksi.Neglible creep was measured for samples stressed to 2 ksi or below (Fig. 15).Analysis indicates that 2 ksi of stress will creep 1.8 parts per million over 10 years at room temperature. 38Further analysis indicates that a PMSA with a surface stress of 2 ksi will see a total figure change of <1.8 nm rms during its room temperature life prior to launch and that no figure change due to creep is expected on orbit at cryogenic temperatures.A rule was established that all beryllium components of the Webb PMSA must be designed, processed, and handled in such a way that no component had a residual stress of >2 ksi.Additionally, extensive tests were performed under AMSD III to quantify exactly how much stress was introduced into a Be mirror during the machining process at AXSYS and grinding/polishing process at Tinsley.These processes were controlled to limit the residual stress in the final mirrors to <2 ksi.Furthermore, all Be components were stress relieved throughout the fabrication process to prevent the accumulation of stress.
The PMSA-180 requirement that a PMSA could survive launch with <2.9 nm rms surface figure distortion was verified with a Webb flight segment.TRL6 was demonstrated by test.An unpolished Webb mirror segment B1 was assembled into a flight configuration PMSA and exposed to design limit loads with sine burst, random vibration, and acoustic testing.Its surface figure change as a function of each loading test was measured using a phase measuring electronic speckle pattern interferometer.The design limit load accelerations for every component within the PMSA were exceeded in each of these tests.Two acoustic tests were performed.The first test hard mounted the PMSA to a concrete wall.The second test suspended the PMSA for a "freefree" test (Fig. 16).The two different boundary conditions provide valuable information for finalizing the PMSA test environment.Neither test resulted in any measurable change to the PMSA surface figure.
The tests showed that the surface figure was repeatable to within the noise floor of the metrology system, 14 nm rms (Figs.17-19).Astigmatism and power figure terms were removed from the total surface change measurement because they can be compensated for on-orbit by adjusting the PMSA ROC or physical location via the cryogenic hexapod with the ROC actuator.The measurement of a surface figure error change that was smaller than the measurement noise floor was consistent with the pretest prediction.Nonlinear plastic material finite-element analysis predicted a surface figure deformation of only 1.6 nm rms.Although not affecting a determination of demonstrating TRL-6, there were two special circumstances associated with this test.First, the random vibration and acoustic levels were notched to maintain safe exposure levels on the PMSA, and a minor inconsistency was discovered with the design limit loads.A new test environment was defined and a minimal PMSA redesign performed to meet the new test environment.Second, although all flight PMSAs will be thermally cycled to 25 K before launch, for reasons of expedience and convenience, the B1     PMSA was only thermally cycled to 150 K.This was determined acceptable because the 150 K temperature subjected the mirror to ∼88% of the beryllium cryo-strain and over 70% of adhesive mount strain.Additionally, an extensive qualification program was conducted for the bonded joints.Test samples were cycled 3 times between 15 and 383 K and subjected to static pull testing.These samples saw only a 12% reduction in ultimate strength following thermal cycling and still maintained a margin of safety of 7.4.This testing coupled with the 150 K that the B1 segment saw was more than sufficient to assure that the TRL6 vibration testing demonstrated the true robustness of the PMSA.TRL6 was achieved by demonstrating the technology to design a lightweight beryllium mirror to design limit loads, testing it to those loads, and showing surface figure stability after exposure to the design limit load, thus assuring that lightweight beryllium mirror technology could meet the Webb launch distortion requirements.

Conclusions and Lessons Learned
One reason for the Webb Space Telescope's on-orbit performance 39 is the success of the NGST Mirror Technology Development Program.The Webb mirror material selection process and technology development program is a model for future NASA missions.AMSD presented two mature technologies to the Webb program for consideration.The competition between these two technologies advanced the TRL of both, resulted in better defined proposal plans, and significantly reduced the total program cost.
Based on the 1996 technology assessment, NASA initiated a systematic mirror technology development program to invent mirror systems that could meet the NGST requirements, reduce the cost and risk of such mirror systems, and demonstrate a TRL of 6. TRL-6 was achieved in 2007 by the combination of the mirror technology development effort and testing of flight mirrors.In the opinion of this author, this achievement was made possible by four specific technical developments and the aperture diameter descope.The four technical advances are the development of O-30 Beryllium by the Air Force with its greatly improved CTE uniformity (compared with I-70 Be used on Spitzer); improvements to computer-controlled polishing at Tinsley; the NASA funded development of the 4D PhaseCam and Leica ADM; and the AMSD program.AMSD was the key to achieving TRL-6.Its success formed a basis for estimating Webb ambient and cryogenic performance, manufacturability, schedule, cost, and risk.The aperture descope from 8 to 6 m enabled stiffer, high-areal density mirrors to survive launch and the addition of hexapods actuation for astigmatism compensation.
Programmatic factors that contributed to the NGST Mirror Technology Development success included unified civil servant technical management, well defined specifications and performance metrics, and competition between ideas and vendors.In total, at least 12 different architecture designs were funded, and the selected beryllium architecture went through five design iterations before flight.AMSD's competitive phased down-select process successfully advanced TRL for large-aperture lightweight cryogenic space mirrors from less than TRL-3 to TRL-5.5 in 4 years (1999 to 2003) with a $26 M investment.Although the effort consisted of multiple contracts, the entire effort was executed by a single civil servant technical/managerial team.The team also provided independent assessment of each contract's accomplishments via cryo-testing mirrors in MSFC's X-Ray and cryogenic test facility.This single-team eliminated the risk of stove-pipes or company proprietary compartmentalization and provided technical continuity through the entire technology development effort and into the flight project.Also, although the initial mirror specifications proved to be wrong-because a primary mirror assembly of 20 kg∕m 2 areal density could not survive launch-these specifications did provide a well-defined set of metrics for assessing the technology development.When it was determined that mirrors made to the initial specification could not survive launch nor could they achieve the desired cost goal, there was technical justification for increasing their areal density and reducing the telescope collecting area from 50 to 25 m 2 .Finally, the competed phase down-select process motivated contractors to meet their schedules and control costs.
Although the actual value cannot be quantified, AMSD certainly paid for itself in cost savings to Webb.Lessons were learned (i.e., mistakes to be avoided), and both vendors demonstrated process efficiencies that did not exist at the start of AMSD.Efficiencies promised to reduce flight mirror fabrication cost by an amount greater than the entire $26 M cost of AMSD (phases 1 to 3).Additionally, both vendors offered contract incentives (i.e., cost sharing via infrastructure investment) during the final downs select process that exceeded the $3 M increment cost of taking a second mirror into phase 3. It is the opinion of this author that these improvements would not have been developed as rapidly or cost effectively without competition.Furthermore, it is the observation of this author that the entire "feel" of Webb mirror fabrication changed once the down select was made and competition was eliminated.I am not saying the feel changed for better or worse, it was just different.There was a change in perceived urgency, need to innovate and reduce cost, transparency, etc.
Therefore, for any potential future segmented telescope (either ground or space), this author highly recommends having two competing fabrication vendors operating in a leader/follower model and giving the lead vendor most of the work and the follower the rest.Thus, if one vendor is having trouble, work can be shifted to the other vendor.Although speculative, I believe that the cost savings from competition will more than offset the infrastructure setup costs.Of course, this recommendation only applies to projects that have modularization and are making many duplicate sub-systems.But a couple of potential thought experiments are to fund five instruments but only four get to fly or design the mission to use a COTS spacecraft.
A programmatic lesson learned from the EDU, AMSD, and NMSD is to plan for the unplanned.Take the most pessimistic schedule and add an additional 50%.Both NMSD mirrors took significantly longer to make and achieved significantly lower performance than expected.On AMSD, because of unplanned activities, the fabrication process was 60% longer than its initial prediction.During the OOR process, the vendor team thoroughly analyzed the AMSD schedule, identified all of the unplanned activities, and detailed how the lessons learned from these unplanned activities had been fully incorporated into the Webb flight mirror production schedule.The government team reviewed this analysis and predicted that the EDU would take 75% longer to complete than the vendor schedule.In actuality, the EDU took roughly 150% longer than the original vendor schedule-2× longer than the government team prediction.
Throughout the process of writing this paper, this author has tried to remember and capture lessons learned during the mirror technology development effort as follows.
• Start with very clear specifications and performance metrics.
• Examine a wide solution trade space-do not limit your trade space too early.
• Use a competitive down-select process to rapidly and cost effectively develop technology.
• Place the effort under a single Government Principal Investigator and Insight/ Oversight Team.• Use a single Government Team to certify compliance with performance metrics.
• Do not trust models to validate performance-validate performance by testing at a relevant scale in a relevant environment.Then iterate until the model matches the data within the allocated error budget uncertainty.• It is nearly impossible to have sufficient "as-built" information to model a mirror's performance to optical specifications.For example, CTE homogeneity is critical for achieving stable thermal performance, but it is nearly impossible to achieve a high resolution 3D asbuilt CTE map.• Plan for failure and statistically improbable events.Mirrors break, bend, or fracture; mechanisms fail; micrometeoroids happen.• Technology development costs more and takes longer than what anyone estimates-maybe as much as 2× more and longer.• Stiffness is more important than areal density.
• CTE homogeneity and uniform properties are critical for stable thermal performance.
• Avoid complexity; it is expensive and risky.The simplest solution is always the best solution.
• Make the mirror as large as possible.Polishing edges is hard.Mechanisms are complex and have had infant mortality up to 30% • Large mirrors are harder to make than small mirrors.Demonstrate technology and processes on the smallest relevant mirror and then scale up by factors of 2×.
• You cannot manufacture something that you cannot test, and you cannot be certain that you are testing it right unless you have an independent confirming test.• Things do not behave the same at 30 K as they do at 300 K and-without experience-your intuition about how they will behave is probably wrong.• Iterate the design, and then iterate again.
• Full-scale pathfinders and EDUs are extremely valuable.If possible, make the flight spares before starting flight mirror production.• Manage the transition to production to maximize learning and minimize forgetting.
• Transparently include all stakeholders and consider alternatives to gain a consensus decision.• Most importantly, there is no substitute for relevant experience.

Disclosures
This author declares no potential conflicts of interests with respect to the research, authorship, or publication of this article.
H. Philip Stahl is a senior optical physicist at NASA MSFC and is a leading authority in optical systems engineering and metrology.He matures technology for large space telescopes; was a PI for SMBD, NMSD, and AMSD; was responsible for Webb Telescope mirror fabrication and certification; and developed the "Stahl" telescope cost model.He is a recipient of NASA's DSM; a fellow of SPIE and OSA; and the SPIE 2014 President.He earned his PhD in 1985, his MS degree in optical science from the University of Arizona in 1983, and his BA degree in physics/mathematics from Wittenberg University in 1979.
previous relevant experience developing the test setup for the Keck segments.Although unknown at the time, the solution for how to test the technology development and flight Webb mirrors (the PhaseCAM and Advanced Distance Meter) was a combination of this relevant prior experience and serendipity.

A
related risk was the change in the ROC as a function of temperature.The AMSD Be mirror predicted a radius change of −13 mm, and a change of −13.06 mm was measured.Its radius change sensitivity over the operating range is only −0.1 ppm∕K.The ULE ® mirror predicted a radius change of þ1.4 mm and measured a change of −4.3 mm.Over the operating range, the radius sensitivity was about −1 ppm∕K.Consequently, it was concluded that a ULE ® is more susceptible to uncertainty in operating temperature and thermal gradients than a Be mirror.

Fig. 17
Fig. 17Figure change from exposure to three axis sine burst testing to design limit loads.
Fig. 17Figure change from exposure to three axis sine burst testing to design limit loads.

Fig. 18
Fig. 18 Figure change from exposure to three axis sine burst testing to design limit loads and first acoustic test.

Fig. 19
Fig. 19Figure change from exposure to second acoustic test.
Fig. 19Figure change from exposure to second acoustic test.

Table 1
Webb optical system requirements versus 1996 state-of-the-art.

Table 2
Mandatory performance criteria.

Table 3
Secondary performance criteria.

Table 4
Webb primary mirror segment assembly design changes from AMSD.

Table 6
Webb mirror technology versus state-of-the-art.

Table 7
Mirror technology success criteria.