Use of close range photogrammetry in James Webb Space Telescope alignment testing under cryogenic conditions

Abstract. The James Webb Space Telescope (JWST) cryogenic testing required measurement systems that both obtain a very high degree of accuracy and can function in that environment. Close-range photogrammetry was identified as meeting those criteria. Extensive modeling prior to installation verified that the design would meet the desired accuracy goals. Extensive validation work was done to ensure that the actual as-built system met accuracy and repeatability requirements. The simulated image data predicted the uncertainty in measurement to be within specification and this prediction was borne out experimentally. Uncertainty at all levels of a measurement volume that comprised 8  ×  8  m was verified experimentally to be <0.1  mm with a repeatability of <0.03  mm at the primary mirror (PM) level, achieving measurement accuracies on par with a laser tracker or radar system. During a 3-month Optical Telescope Element and Integrated Science Instrument Module thermal vacuum test performed in chamber A at Johnson Space Center, the photogrammetric system provided key data to allow for verification of actuator ranges needed for on-orbit alignment of JWST, alignment of the PM and secondary mirror (SM), and positional information needed for thermal model verification.


Introduction
Close range photogrammetry has successfully been used on large hardware systems such as the microwave anisotropy probe 1 for dimensional surveys of large numbers of points. Photogrammetry was also the approach selected at Goddard Space Flight Center (GSFC) for the James Webb Space Telescope (JWST) Integrated Science Instrument Module (ISIM) testing at cryogenic temperatures. 2 Typical uncertainty requirements for such work are AE0.100 mm, 2σ, per axis, per target point. For the JWST OTIS testing, the requirements were even more stringent at the primary mirror (PM) level and required accurate measurement over a cylindrical volume of 8 × 8 m in a cryogenic vacuum environment that comprised the space between the secondary mirror (SM) and PM. Given the demonstrated ability of photogrammetry (PG) to operate in such an environment to similar tolerances, it was chosen for use as the Cryo-Position Metrology (CPM) system for the Optical Telescope Element and ISIM (OTIS) thermal vacuum (T/V) tests to be performed in chamber A at Johnson Space Center (JSC). For the purposes of this paper, "PG system" and "CPM" are used interchangeably.
Prior to the construction and installation of the CPM, the expected accuracy of the as-designed system needed to be validated. At first, a simple mockup in a large clean room was used to demonstrate feasibility, but this required a fairly large effort and was limited by spatial constraints and the inability to model structures as designs were updated. Due to the size and complexity of the OTIS structure, the creation of a computer model was chosen rather than a mockup for greater flexibility in examining various testing configurations and to provide a test *Address all correspondence to Sharon R. Lunt, E-mail: sharon.lunt@l3harris.com bed that can be used for the life of the program. This paper summarizes work that was done to assess the expected accuracy of measurements for the OTIS configuration before any hardware was installed in the chamber. Predictions were then compared to subsequent verification measurements of the operational CPM performed during initial chamber tests of ground support equipment (GSE) to be used in the full OTIS tests. These tests are designated as chamber commissioning test (CCT), OGSE1 and OGSE2 (optical GSE), and thermal pathfinder (TPF) in this paper. The predicted performance met the requirements of the error budget and the PG system was successfully utilized in several optical tests utilizing the JWST pathfinder 3 before being used on the flight hardware.
Initial testing of the actual performance of the PG system after assembly in the chamber did uncover system performance variability that exceeded requirements. After additional testing and extensive analysis, the team developed operational and analytical tools to address these issues. Final testing then demonstrated that the PG system did indeed meet all performance requirements at cryogenic temperatures. This would not have been possible without the pretest computer models and extensive JWST pathfinder testing at under cryogenic conditions.

Value of the PG System for JWST
Proper alignment of telescope mirrors to each other is essential to meet image quality requirements. The JWST is the largest segmented space telescope every built. With an actuated primary and SM, the system is designed to align itself on orbit. Ensuring the system has the range required was a primary goal of the OTIS cryogenic test. 4 With a PM diameter of 6.5 m, a spacing to the SM of 8 m, an operational temperature of 40 K, and alignment tolerances approaching an absolute accuracy of 0.100 mm this is an exceptionally challenging problem. The only conceivable technology to meet those parameters was a photogrammetry system.
The requirements for the PG system were as follows.
• Globally align the PM to the tertiary mirror via PG to the tolerances found in Table 12. This provides the starting point to phase the mirror optically and ensures proper alignment of the fore optics (PM and SM) to the aft optics (tertiary mirror and science instruments). 5,6 This iterative process verifies that the PM segments will have adequate actuator range on orbit and ensures that the flight alignment starts from the proper location.
• Align the SM to the PM via PG to the tolerances found in Table 14, thereby ensuring its actuator range.
• Verify that the thermal distortion of the SM with respect to the PM [alignment distortion (AD)] is within required range.

CPM Hardware Description
For thermal/vacuum tests of the JWST observatory, a set of four cameras on rotating windmill booms contained in pressure and thermal tight enclosures are used inside the helium shroud. 7 As the windmills rotate, the camera system records multiple images of special photogrammetry targets placed on the OTE and throughout the chamber. Figure 1 provides an overview of the JWST OTIS flight hardware and optical GSE (OGSE) in the JSC chamber A. The OTIS photogrammetric survey was done primarily during conditions of a 30-K vacuum environment, although some testing was done at ambient temperature and during cool down/warm up. The successful application of photogrammetry requires support equipment that includes scale bars, coded targets to aid automated processing, and metrology target point locations that define the object of interest. The scale bars were used for the correct calculation of scale to apply to a photogrammetric bundle. This is of critical importance to the overall accuracy of a measurement survey. The code targets are used to aid in defining the position and orientation of the camera during bundle adjustment.

Scale Bars
To scale a photogrammetric measurement, at least one known distance must be present in the imagery. For the purposes of the JWST PG system, multiple scale fixtures (i.e., scale bars) are incorporated into the object space that contain target pairs of known distances. The distances between target pairs on the bar were characterized and can be used to scale the measurement survey. Multiple distances allow a higher order of scale calculation accuracy and the redundant measurements allow for the ability to find scale errors. Thermal sensors were placed on each bar to accurately calculate scale bar length at each point in the temperature cycle where a measurement was performed.
The modeled setup for this PG application included 20 invar scale bars positioned around the PM from posts attached to the Hardpoints and Offloader Support Subsystem (HOSS) structure. In addition, as part of the gravity references located on the posts, there were five vertical scale bars for vertical (Z) scale. Each scale bar contains one thermal diode and a set of four target pairs for redundant scale. Figure 2 shows the distribution and an example invar scale bar.
Based on results from JWST pathfinder chamber tests, vertical scale bars were also added near the SM affixed to the chamber walls. The bars provided a vertical scale reading that had a lower bundle adjustment uncertainty in some pointing schemes than the vertical bars on the HOSS and thus provided a check that the scale factors were consistent throughout the bundle. Inconsistent scale would be a sign of distortion of the bundle.

Coded Targets
Coded targets are a special type of target that the Video, Self-Calibration, Triangulation, and Resection Software (V-STARS) can recognize, automatically decode, and use to automatically  calculate the position and orientation of the camera to aid bundle adjustment. Each code is made up of a unique pattern of dots. When planning a photogrammetry, project coded targets must be well-distributed throughout the object space so that each captured image contains a minimum of four coded targets. Coded targets were distributed on the chamber wall and on sleeves that went around the telescope rod supports. In addition, the telescope rod sleeves provide PG targets in what would otherwise be empty space so that there were sufficient points in each image to stitch the entire bundle together. Initially, the points on the telescope rods were planned to be used as scale bars, but measurements of expected coefficient of moisture expansion and coefficient of thermal expansion (CTE) indicated that the materials properties were too variable to ensure reliable scale bar accuracy. An illustration of the telescope rod sleeve is shown in Fig. 3.

INCA Camera System
The photogrammetric camera system selected for the JWST photogrammetric application is the third generation Intelligent Camera (INCA3) system provided by Geodetic Systems, Inc. (GSI). In addition to its robust design and vetted operational accuracy, this system was chosen because  it could operate in an automated capacity. Due to the demands of the cryogenic vacuum tests at JSC, remote operation was an important capability. A compact but powerful computer accompanied the camera system to do image processing and bundle adjustment. Figure 4 shows the INCA3 photogrammetric camera system and identifies some of the advanced features. Table 1 outlines the manufacturer's specifications for the INCA3 camera system.

Coordinate System
The results in this paper are reported relative to a coordinate system that is defined with respect to the Aft Optics Subsystem (AOS). The origin is the center of the base of the AOS on the PM and the axis is oriented as shown in Fig. 1 where +M1 is toward the top of the chamber, and +M2/ +M3 are in the plane of the PM.

Data Processing
All PG data analysis was completed using the V-STARS software supplied by GSI to create a bundle adjustment. Bundle adjustment optimally determines the solution of 3-D coordinates of the targeted objects, as well as camera parameters, in an automated operation using imagery taken at multiple points of view. The bundle adjustment eliminates systematic error and minimizes residual random error through a least squares process that accounts for the position and orientation of the cameras when each image was captured and distortions from the camera lens. The 3-D coordinates from the output of the bundle adjustment process are then aligned to the desired coordinate system to provide the final measured position of all targets. Coordinate system alignment was performed with respect to AOS, which is at the center of the PM using Monte Carlo methods incorporated in the software described in Ref. 8.
For most PG data collection runs, no bad or weak point filtering was applied other than that done automatically by V-STARS as removal of all identified weak points resulted in the elimination of many desired targets, though there was a rejection limit set for the minimum number of rays (images) required per point. A test sequence was done where bad and weak points were manually removed and the resulting uncertainty estimates of the remaining points were determined to be equivalent. The proper selection of scale bars was found to be a critical factor in absolute accuracy and is discussed in more detail in Sec. 4.3.2 under "Scale factors."

Modeled Imagery Generation
The size of the flight hardware and chamber created a measurement volume of 16 × 8 × 8 m, which made testing the expected performance of the system prior to construction challenging. The time and expense of modifying physical mockups indicated the need for a more cost-effective approach. Simulated imagery was used to model the expected performance of the CPM prior to installation. The imagery was constructed using the Digital Image and Remote Sensing Image Generation (DIRSIG) version 4.5 software developed by Rochester Institute of Technology DIRSIG Laboratory and has been used extensively throughout the remote sensing community for more than 20 years. 9 DIRSIG performs end-to-end radiometric calculations from scene to entrance aperture to generate synthetic radiometrically accurate images.
To determine the distance and angular fall-off of the reflective material, extensive testing of actual targets at various sizes, angles, and distances was done with the INCA3 camera. From this testing, a model was created and verified to have the correct angular fall-off and intensity profile.
By importing the CAD model of the OTIS configuration as well as the material properties of 3M retroreflective material used for the photogrammetry target material, a realistic, albeit virtual, model of the structures such as the PM and SM was constructed. This model helped to identify hardware occlusions and refine target locations in the configuration shown in Fig. 1. The camera positioning and movement was incorporated into the definition of each modeled image. Image processing routines were employed after the computer model generation to simulate sensor response and the point spread function of the INCA camera system. Images were converted from 64-bit radiance to 8-bit digital counts to match the output format of the INCA 3 camera system. The images were then imported into V-STARS for processing to determine the predicted PG measures of each point. An example of a DIRSIG generated and postprocessed image is shown in Fig. 5.

Hardware Configuration, Target Placement, and Data Flow Testing
Prior to the completion of the CPM, the modeled imagery was used to determine optimal camera pointing schemes to ensure that all objects were adequately sampled and to assess the expected accuracy of the system. Two camera pointing schemes were developed-one that samples the whole chamber (OTIS_ps) and one that provides for more accurate determination of objects at or below the PM plane by not recording images of the Auto-Collimating Flat (ACF) at the top of the chamber (LOWCH_ps). Both pointing schemes were used successfully without modification during CCT and OGSE1 testing. A small modification was made to the lower chamber pointing scheme (LOWCH_ps) during OGSE2 to provide better imaging of the SM targets. The DIRSIG model provided the ability to assess the impact of changes made in the number of targets used and answer other configuration questions such as the utility of having vertical scale bars on the telescope rod sleeves if the deficiencies in materials properties could be overcome. For example, originally 120 targets were proposed for the PM, but that number was eventually reduced to 30 for the final OTIS design. V-STARS analysis using modeled imagery of the two cases indicated that the same level of accuracy would be expected. In the case of the vertical scale bars on the telescope rods, the model showed that they were poorly seen and would be expected to have a very large uncertainty and so would not provide reliable scale values.
The existence of a full set of DIRSIG model imagery also provided the ability to exercise the analysis software to ensure that there were no issues with the number of images required, the spacing and placement of coded targets, etc. proved quite useful prior to actual chamber testing. It allowed very quick turnaround of the initial chamber test data without having to work out logistics, fitting parameters, etc. at that time of the first data collects. The necessity of multiple coded target sizes in the chamber to accommodate the large variation in distance, and hence intensity, between the closest and furthest sets required modification to V-STARS. Using the modeled image data, the new capability was tested out prior to actual chamber measurements. Having the data available from modeled images also allowed the determination of the pros and cons of various coordinate system alignment fitting schemes in terms of number of objects used so the best accuracy could be ensured.

Prediction of PG Accuracy
One key advantage of using a model versus a physical mockup is the ability to compare against absolute truth. There are several ways to estimate the expected accuracy of the CPM system based on the DIRSIG modeled imagery. Overall numbers can be calculated based on the image residuals or RMS of the bundle adjustment uncertainty. More detailed analysis of each object can be obtained by looking at point by point comparison of the V-STARS calculated bundle values versus the reference values. A more extensive description of all these methods can be found in Ref. 10, some key results are provided below.
To determine values and statistics for residuals, bundle adjustment uncertainty and absolute position four consecutive PG runs were obtained under both ambient and cryogenic conditions during the CCT. Comparison of model results to observed performance indicated that there was a larger value for image residuals than seen under experimental conditions. Image residuals reflect the variability in the value of PG measurement of an individual target point in all the images that contain that point. Image residuals were ∼0.73 μm for the DIRSIG modeled imagery, whereas GSI predicted their camera would be ∼0.25 μm for the bundle adjustment. Image residuals are measured with respect to the dimensions of a pixel on the sensor, which is 10 μm (Table 1). For ISIM testing, measured residuals were 0.2527 μm. 11 For the commissioning tests, the bundle adjustment of imagery from the CPM gave an overall image residual of 0.18 μm which is similar in magnitude to that found for the ISIM testing and consistent with what was expected by the manufacturer. The contribution to image residuals due to model limitations of, for instance, the target center visibility at oblique angles, is the most likely cause of the over prediction.
The actual bundle adjustment uncertainties of a representative run from CCT are shown in Table 2 compared to model predictions. A few areas are similar (e.g., SM V2/V3), but in most cases, the modeled imagery bundle adjustment uncertainties were a factor of ∼3 to 4 times larger than measured in the chamber. The actual imagery versus simulated imagery predicted bundle adjustment uncertainties were quite consistent with the differences in the image residuals of about 3 to 4 times (0.75 versus 0.18 μm), so the increase in residual correlates directly with an increase in bundle adjustment uncertainty.
It should be noted that bundle adjustment uncertainty does not necessarily translate to increased absolute error, as the bundle adjustment uncertainty is a measurement of variability image to image of the point measurements and a large distribution does not necessarily imply a shifted mean value. The mean value is the absolute position in space.
To provide an independent characterization of the absolute accuracy of the CPM system, laser trackers (LTs) were utilized in CCT and laser radar (LR) systems were utilized in OGSE 1 preliminary chamber tests to provide calibrated 3-D coordinates for a subset of PG target mount locations. The CPM testing consisted of four consecutive PG runs with concurrent LT or LR measurements. Prior to installation in the chamber, the LR measurements of many targets were also taken in the clean room. The run-to-run repeatability and absolute accuracy with respect to an LT-or LR-based reference position were calculated for each run and compared across runs.
LT data were taken in the chamber. LR data were obtained both in the clean room and in the chamber. Due to the constraints of where the LR could be put in the chamber and the fall off of accuracy as a function of laser angle of incidence, some targets could only be measured in the clean room where the LR could be positioned more freely. Some pieces of hardware were not installed in the cleanroom and so were only measured in the chamber. The top of the SM targets could not be measured in the chamber or the cleanroom, and so the back of the PG target was measured instead, and the PG center was calculated based on design measurements of the target mount.
The error of PG measured versus LT and radar measured values are shown in Table 3. The measured errors include both the PG and independent metrology uncertainties. The average error of the model is consistent with the actual measurements within the uncertainty. The actual measurement uncertainty with respect to model values is at a level that is consistent for individual axes with a 3× change in image residuals roughly causing a 2× change in overall magnitude. Table 3 Actual error from CCT measurements versus predicted error using DIRSIG imagery (in parentheses) for selected objects. Values are 1 sigma. ACF numbers are from CCT data, SM, AOS, and PM from OGSE1 data. Data are from a single photogrammetry run.  The lack of availability of known reference points other than scale bars under cryogenic conditions prevented extensive characterization of the absolute accuracy under those conditions. The CTE of the invar used to construct the scale bars was measured experimentally and used to predict the length under cryo-conditions to provide an accurate reference.
The strut base points targets (SBT) on the HOSS do provide a reference to cross check that the scale predictions are within expected tolerances. Spatial analyzer software was used to fit measured changes in SBT positions for ambient and cryogenic runs from OGSE1 and OGSE2 preliminary chamber tests. For the SBT points, a repeatability of 50 μm was achieved for an object of 7-m diameter. A cross check of the change in distance from ambient to cryo-stable on a similarly sized stainless steel feature in the chamber also agreed within 50 μm to expected changes.
In summary, modeled performance using DIRSIG generated imagery was limited by the artificially high residuals from bundle adjustment, which did provide good guidance when that difference was taken in account as to the overall performance of the system. The use of modeled imagery also allowed for determination of camera pointing schemes that would best optimize the trade between number of images per object and the need to cover a large chamber space.

Repeatability
Although the use of modeled imagery prior to initiation of chamber testing provided many insights into the CPM, additional experimental data were needed to fully characterize the CPM prior to OTIS.
An important measure of PG system performance is the repeatability of measurement. The four runs taken during CCT and used to determine accuracy were also used to calculate an RMSE on a per point basis for each object under ambient conditions. During OGSE2 testing, five consecutive runs were taken under cryogenic conditions with real hardware to verify the performance. The repeatability was calculated by (1) where i is the i'th PG target position on the calibration fixture, σ i is the standard deviation calculated for a given number of PG runs for the i'th position, and N is the number of points. The results in Table 4 were used in the final error budget assessment instead of bundle adjustment uncertainties because these numbers better reflected the total measurement variability of the CPM. The full error budget includes not only uncertainty from the PG measuring and processing but also contributions from factors such as mount uncertainty, scale uncertainty, and Table 4 Values by hardware object for run-to-run repeatability of PG measurements. 2σ is calculated as per Eq. (1).

SM measurement cross checks
Due to the critical need of knowledge of the SM position in both cryo-stable PM-SM alignment and AD testing and the difficulty in getting the absolute position comparison during the commissioning process, additional data was collected and analysis was performed during subsequent OGSE1, OGSE2, and TPF tests on the repeatability and stability of the SM measurements. Two general areas were assessed-the ability to accurately determine the scale and the impacts of different camera pointing schemes.
Scale factors. One large driver of possible errors in SM position is the ability to correctly assess and to repeatedly calculate scale. As a cross check on the repeatability of the scale calculation, the distance from scale bar set to scale bar set across the PM was monitored (referred to as "Big Bar," see Fig. 6) starting in OGSE2 and continued through OTIS. Results showed that the 2σ variance was <0.06 mm on average across the ∼8-m distance from scale bar to scale bar under cryo-stable conditions for OGSE1 and <0.08 mm for OGSE2. Similar results could not be obtained for the TPF test as several scale bar points became unmeasurable (Fig. 7) as discussed below.
Of the scale bars remaining, the bundle adjustment uncertainty was markedly higher than during OGSE2 (Fig. 8). The increase in noise in the scale bars began during cool down. Root cause investigation traced the problem to a chamber leak causing O 2 and N 2 condensation on the coldest scale bar targets thus obscuring the retroreflective surface ( Table 5). The sensitivity of the bundle adjustment uncertainty calculation to condensation was used during OTIS testing to monitor for signs of this kind of issue. The only occurrence found was during a brief period during warm-up that had conditions that were conducive to previously condensed gases that had started evaporating off warm parts of the chamber (a "burp") recondensing on the much colder scale bar posts.
An experimental change during OTIS testing as compared to prior chamber tests was to add vertical scale bars attached to the chamber walls. These bars were attached about a meter below the SM level so that the bundle adjustment uncertainty was predicted to be at an acceptable level for both camera pointing schemes. Unfortunately, the position experimentally proved to give unreliable scale bar estimates in the LOWCH_ps case due to the low number of usable images, Fig. 6 Examples of scale bar to scale bar distances (Big Bar) used to monitor scale calculation consistency during cryo-stable testing.  Table 5 Average temperature for each post for the last cryostable run in TPF. Posts are ranked from least noisy to most noisy based on the average of the bundle adjustment uncertainty shown in Fig. 8   as approximated by the number of "rays" used in the PG solution, (Table 6) and so those bars were excluded for use in the pointing scheme. The OTIS_ps and use of bundles created from the combined images of both sets provided much more consistent estimates of vertical and horizontal scale because the number of rays was significantly increased.
Camera pointing scheme factors. The calculation of scale was very consistent run to run and between camera pointing schemes. However, testing indicated increasing discrepancy between the calculated SM position in M1 as more targets were installed near the PM. Table 7 illustrates the difference between the average calculated position of the SMA between the two pointing schemes. Although differences always existed, with the full flight hardware installed in OTIS, the difference in M1 became more pronounced than in previous tests and this prompted further analysis. Because the measurements from scale bar to scale bar (Big Bar) agreed, scale error between the pointing schemes was quickly ruled out as a cause of the discrepancy. During pre-OTIS testing, inconsistency in calculated SM position by the CPM system was also observed during initial AD testing. The purpose of the AD test is to characterize the change  in PM to SM M1 position as a function of telescope temperature. Because the variations on flight are so small (∼0.2 K), a large temperature delta is used for the ground test (20 K or more) so that the measured PM to SM distance changes are well above the measurement noise of the PG system. The experimental results showed large (>0.2 mm) deviation of measured results to expected values in the first change from 35 to 55 K in some but not all of the chamber tests. Measurement of the SM was hampered by two factors. The first is that SM targets were constructed as flat targets facing up and due to the configuration of the booms and requirements of pointing angles to see other critical objects, there were significantly lower number of images of the SMA targets than at the PM level. The second issue is target visibility. The SM is the only object besides the telescope rods that provide PG targets in the 16 m of space between the PM and the ACF. Examination of the predicted bundle adjustment uncertainty at the SM level in the pretesting DIRSIG simulation data indicated that the dearth of points in the bundle at this level creates a nonuniform distribution in visibility across the object at the SM versus the PM level (Fig. 9). The predicted uneven distribution in visibility was seen in actual OTIS_ps data taken during OTIS testing (Fig. 9). The average number of SM views is also roughly half the number of PM views for both camera pointing schemes. There is a strong correlation between visibility and bundle adjustment uncertainty up to a level of about 200 rays (Fig. 10) with an empirically observed floor for this case system of 0.012 mm for all points on the JWST hardware.
The LOWCH_ps case, with the increased number of rays for objects lower in the chamber, might be expected to improve the situation. Although there is a noticeable increase in the number of rays between the OTIS_ps and LOWCH_ps camera pointing schemes ( Table 6) that improves the overall bundle adjustment uncertainty of the points (Fig. 10), overall the same nonuniformity is present (Fig. 11).
In summary, chamber measurements indicated that there was a noticeable inconsistency in the solution of the position of the SM points as a function of pointing scheme, objects present in the chamber, and the presence of a thermal gradient during the start of AD testing. Investigation as to possible causes of these observations found no problems in the CPM in image collection, image quality, or bundle adjustment despite extensive cross checks. However, analysis of the modeled data indicated that there is a measurable nonuniformity of the visibility of the SM points and half the number of views, which was confirmed with test data. This nonuniform distribution and sparse point density is likely to have increased the sensitivity to how changes in uncertainty distributed in the lower chamber due to different camera pointing schemes or number of lower chamber PG targets in turn changes the optimum SM position required to give the minimum random error distribution during the photogrammetric calculation. More views of the SM were required.  The behavior during the initial phase of AD testing in OGSE 1 and OGSE 2 was also consistent with an increased sensitivity to changes in the lower chamber. Precise photogrammetry relies on the assumption that the targets are in a stable position throughout the duration of the image collection. The disagreement between the predicted and measured SM position during the AD testing in OGSE 1 and OGSE 2 occurred during a period in which the hardware temperature was increasing per the test plan. The suspended configuration as a whole expands during this change in temperature thereby resulting in motions of the scale bars with respect to each other, the flight hardware and all tie points used in the V-STARS bundle. The stable assumption was not met during these periods.
The calculated movement of horizontal scale bars was 3 to 5 μm per meter per hour based on the change in Big Bar lengths (total change of up to 40 μm of length of scale bars) during a 45-min data collection. The height of the hanging configuration changed by an estimated rate of 100 μm∕h based on the average change in height of two rail targets with respect to the AOS. Even with the larger noise of the rail points compared to the scale bars (estimated values are 2σ ¼ 72 μm for the rails versus 2σ ¼ 28 μm for Big Bar points), the change is considerably larger than the estimated horizontal expansion and so not only is the hardware changing, it is changing asymmetrically.
The team believes that this instability impacted the calculated position and variability of the SM. This could have been analyzed using the DIRSIG imaging analysis tools developed prior to hardware installation but was deemed too time-consuming. With the hypothesis that the changes in temperature were the cause, the test plan was modified to include thermal plateaus, in which the data were acquired. With this change for the flight hardware test, the agreement between measured and predicted data was within the predicted test uncertainty and the observed measurement variability was also reduced to an estimated 10 μm∕h in the vertical direction.
In an effort to mitigate the sensitivity to SM position calculations inherent in the use of a single camera pointing scheme, a "combined" bundle solution was created from all 2360 images from both camera pointing schemes for critical measurements, thus providing additional images of the SM. During active OTIS testing, because of the processing time needed to create the combined bundle, a numerical weighted average was created from a sequential OTIS_ps, LOWCH_ps, and OTIS_ps collection set. As shown in Table 8, the approximation of a combined bundle with an average gives a difference 0.059 2σ between the results-well within the measured error in absolute position of 0.086 (Table 3).
There was a marked increase in number of rays for the SM points with the use of the combined data (Table 9), though the visibility nonuniformity pattern was the same as it is a feature of both camera pointing schemes. The two camera schemes also collected images from different angular positions, which are known to help in bundle adjustment uncertainty. The bundle adjustment uncertainty for the SM points in the combined set approached the empirical limit, and so presumably a lower limit of sensitivity to camera pointing scheme and chamber configuration had also been reached. The combined bundles are considered to have the highest absolute accuracy because they consist of the greatest number of images and have the lowest bundle adjustment values. DIRSIG modeled data did correctly predict the visibility nonuniformity at the SM level, but overpredicted the actual number of rays and expected relative bundle adjustment uncertainty. A lesson learned is that in future systems more scrutiny should be paid to assessing sensitivity to camera and pointing scheme of areas with asymmetric visibility to ensure that the sensitivities are within acceptable limits and that the pointing schemes adequately sample the objects from a variety of angles.

OTIS Testing Results
The purpose of the final OTIS test was to ensure that flight hardware met all criteria necessary to ensure successful operation on orbit. The CPM provided data to allow for verification of alignment capabilities.

Repeatability
To verify that the CPM variability in object and scale was the same as observed in the previous chamber tests, Big Bar and run statistics were collected. The 2σ variability of PM-Global and SM center position for runs completed during the cryo-stable test period are given in Table 10. The average 2σ for Big Bar measurements were 0.028 mm.

Alignment of PM and SM
The CPM provided reliable measurement of the position of the PM and SM during alignment, which allowed for effective predictions of adjustments needed to obtain the desired positional accuracy. Representative runs taken from 8/5/17-8/24/17 over the period of PM phasing are shown in Table 11. Note the initial jump from unaligned to close-to-predicted on 8/6. The period from 8/6-8/24 covered various moves done to phase the mirrors based on optical information from the Center of Curvature Optical Assembly (CoCOA). Note the drift from ideal and realignment back to the model as the phasing progressed.
Detailed investigation of the cryo-shift behavior postchamber test indicated that there was a small but statistically meaningful movement of the AOS Source Plate Assembly (ASPA) Bridge with respect to the AOS baseplate consisting of a shear of 0.05 to 0.06 mm. The reference coordinate system was revalued to account for the actual position of the AOS and ASPA Bridge points. Small updates to the optical model were also made post-test. The results of the PM aligned position with respect to tolerances compared with the optical model used in-test and the final post-test revised optical model are show in Table 12.
Since the SM is a single unit, it required much less adjustment to be brought into alignment. The position at times is shown in Table 13. The SM was placed in a nominal start position on 8/21/17, and an initial alignment move was made 8/26. Examination of the large angular

Conclusions
To provide the required in situ measurements of JWST hardware during chamber testing for alignment, a close-range photogrammetry system was implemented as the CPM. The hardware was modeled extensively prior to construction to verify performance, define camera pointing schemes, and assess effects of various proposed target configuration. This was accomplished years before the actual test was completed. A computer generated DIRSIG model predicted a higher level of image residuals and measurement uncertainty then observed in the actual system but correctly predicted the error in absolute measurement. When the difference between model and actual image residuals was accounted for, the measurement uncertainty from the DIRSIG model was similar to that found in chamber T/V verification testing. Further DIRSIG model simulations may be improved by better camera characterization to better simulate noise from the cameras. One of the main learnings from the comparison of the actual system performance and the modeling work was that there was an underestimation of the number of rays and angular diversity required to ensure a stable photogrammetric solution of the SM points. Future systems should use a metric that adds more weight to the angular diversity in determining camera pointing scheme design. For objects with a sparse number of total points, such as the SM, it is also important that all points have a sufficient number of rays to unsure that bundle adjustment uncertainty is at a minimum and that a stable positional solution can be found. In terms of experimental design, the use of spherical PG targets or ones at a 45-deg angle on the SM would have helped mitigate the issue of lower angular diversity in the image collection and increased the number of  rays on the less well seen points. Without the capability of modeling this type of system, these types of trades would be very difficult to assess. The CPM was successfully used in four initial chamber tests and the final OTIS testing phase of JWST hardware. The extensive initial testing was essential to understanding the complex nature of the PG system performance. Testing on the JWST OTIS confirmed that final alignment of the PM global and SM global as well as the cryo-shift, alignment distortion due to thermal gradients, and ambient to ambient repeatability all met requirements. PG systems, properly designed, can achieve measurement accuracies that approach the performance of LT or radar systems and should be considered for configurations requiring accurate measurements of targets in a vacuum environment.
Conrad Wells received his BS/MS degrees in optics from the University of Rochester and his PhD in optical science from the University of Arizona. He worked for TRW, Hughes SBRC, Wyko, and KLA-Tencor, and he joined L3Harris Technologies in 2003. His experience includes interferometers, interferometric microscopes, weather satellites, optical testing, and space-based astronomical telescope integration and test. He served as the deputy optical test director for the cryogenic test of the James Webb Space Telescope at the Johnson Space Center and is currently supporting the LUVOIR and WFIRST programs as a senior optical systems engineer.
David Rhodes is a senior image scientist at L3Harris Technologies. He graduated with a degree in imaging science from the Chester F. Carlson Center for Imaging Science at the Rochester Institute of Technology and began working for L3Harris in 2012. His professional interests mostly apply to end-to-end modeling and simulation of imaging and optical systems as well as 3-D reconstruction techniques.
Andrew DiAntonio received his BS degree in imaging science from the Rochester Institute of Technology in 2012 and is expecting an MS degree in technical management, with a concentration in optics, from the University of Rochester in 2020. He has worked for L3Harris Technologies for the past eight years where he is currently a program manager.