One of the investigated solutions to the augmented reality (AR) problem has been the freeform prism combiner [1-8]. The freeform prism design is robust during use because it features fewer independently-moving elements. However, this design space also comes with fewer surfaces to optimize to meet optical performance. It is therefore very important to examine how surfaces are used. This paper adopts a careful, nodal-aberration-theory-based design approach, which examines how specific polynomial terms correct the aberrations of the system with minimal freeform surface complexity. Results clearly show that the choice of freeform prism geometry greatly affects the potential for aberration correction. In this design study, the freeform prism combiner, which will adopt alternatively one of two fundamentally different geometries, receives light from a micro-display and projects it into the eye, with the intention that auxiliary elements could be added if necessary for improving see-through performance. All systems were designed to operate over a 45° diagonal full field of view, an 8mm eyebox, and a 21mm focal length, for intended use with a micro-display with 10 μm pixels. All systems achieve >10% MTF at 50 lp/mm over all fields, evaluated over centered and decentered 3mm subpupils chosen at nine strategic locations within the eyebox. From prior published work of others, full-color systems are expected to suffer from chromatic aberrations . Consequently, to analyze geometry-induced aberrations, all designs in this specific investigation are purposely chosen to be monochromatic. Issues of chromatic aberrations and novel means of correction are treated in a parallel investigation by this team in a separate submission . Full specifications are in Table 1.
Specifications for Freeform Prism Combiner Systems
|Eye Clearance (mm)||>18.25|
|FOV (deg.)||26 × 41 (H × V)|
|Image Quality||MTF >10% @ 50 lp/mm|
|Optical Path Folding Direction||XZ Plane Symmetry|
|Number of Freeform Surfaces||3|
|Total Number of Freeform Coefficients||63|
|Number of Freeform Surfaces||2|
|Total Number of Freeform Coefficients||15|
|Diagonal Length of Active Display (mm)||21.8|
|Resolution in Pixels||1080 × 1920 (X × Y)|
|Pixel Pitch (μm)||10|
The first question in terms of designing the freeform prism is a choice of mathematical basis for describing the freeform surfaces. Descriptions range from XY polynomials to the broadly-used Zernike polynomials to application-specific characterizations such as the freeform Q-polynomials introduced by Greg Forbes [10-12]. In this paper, FRINGE Zernike polynomials, which follow the order of the Seidel aberrations, are used . This gives sag equation
where u = ρ/ρmax is the normalized radial coefficient, c is the curvature of the base sphere, and Cj is the coefficient of the jth FRINGE Zernike term. Without loss of generality, a spherical base surface was used in all cases, and degeneracy between the base surface and the freeform surface are managed in software .
Once the mathematical surface description is selected, the designer needs to understand how to use it. Because of increased complexity of both freeform surface descriptions and of freeform aberration theory, designers may lean more heavily on the optical design software to improve nominal performance. This can lead to systems that are overdesigned, difficult to manufacture and are very sensitive to small changes in parameters. Often, it is hard to undo such decisions: polynomial terms which increase the complexity of one surface can lead to increased complexity in other surfaces as compensation. This emphasizes the need for the careful understanding of how polynomials are used. Freeform aberration theory, including Nodal Aberration Theory, is used to understand the performance of the system [15, 16].
Lastly, it is helpful to estimate the complexity of the freeform surfaces. This paper focuses on the number of freeform surfaces in the design and the number of freeform coefficients on those surfaces, since more complex surfaces are more expensive to manufacture and test.
ANALYSIS AND CORRECTION OF INITIAL ABERRATIONS
Two different geometries for a freeform prism combiner are designed and analyzed. The purpose of the freeform prism combiner is to map a micro-display onto a distant point of the eye (chosen to be infinity in this case for benchmarking) to facilitate the creation of virtual images to the eyes of a user. Equivalently, the freeform prism can image the retina of the eye onto the micro-display, which enables the designer to treat this problem solving as a traditional imaging optical design problem. Both freeform prism combiners sit in front of the eye according to the diagram in Figure 1.
The first geometry of interest, which was first designed as early as 1997, leverages total internal refraction (TIR) to utilize the optical surface closest to the eye as both a refractive element and a reflective element . The strength of this geometry is the reduced volume, the size of the eye-box, and the few surfaces involved in the design. However, the TIR requirement also forces the prism to adopt a familiar wedge shape. It is worth noting that the relative orientation of the TIR surface and the fully-reflective surface is in fact not optimal for correcting astigmatism and coma with freeform surfaces, a phenomenon identified and analyzed by Bauer et al. while designing for a different application . Analysis of the impact of the geometry on performance is a key focus of this paper and is discussed in later sections.
The second geometry investigated adopts a different relative orientation of the two mirrors and does not leverage total internal refraction [8, 18]. To achieve this geometry, the total-internal-reflection is separated into a fully-refractive element in front of the eye and a fully-reflective element whose orientation can be selected based on aberration correction potential. The design following this geometry is referred to as the Reflective Prism. The Reflective Prism features a fourth surface, but requires a significant reduction in the number of freeform terms to correct the astigmatism and coma. XZ cross-sections of the final designs of both geometries can be seen in Figure 2.
Initial Limiting Aberrations: Astigmatism and Coma
The preliminary focus is an analysis of the aberrations for the all-spherical starting points two geometries, with the aim to show how geometry can affect the correction of astigmatism and coma. For the TIR Prism, aberration correction was also considered during the selection of powers, thicknesses, and tilts of the starting point. This clarifies the analysis of the design but does not change which aberrations limit the system. For the Reflective Prism, packaging constraints and specifications in Table 1 drove the selection of powers, thicknesses, and tilts, without any optimization for aberration correction. Full-field displays for the RMS wavefront error (RMS WFE), as well as the limiting Zernike aberrations, are shown in Figure 3a,b.
As can be seen in Figure 3a,b, both systems are initially limited by astigmatism. For the reflective prism, astigmatism is initially field-constant (FC) astigmatism. Because no aberration correction has yet taken place, 100 λ of FC astigmatism are present. For the TIR prism, there is 20 λ of astigmatism, but the shape is instead field-asymmetric, field-linear (FAFL) astigmatism, as well as 4 λ of FC coma.
With these aberrations understood, it is necessary to try to correct them. As will be shown, for the TIR prism, the FAFL astigmatism and FC coma will be difficult to correct. To demonstrate this point, the effects of introducing Zernike astigmatism and Zernike coma on each of the three surfaces of the TIR Prism separately, then all together, was investigated. Figure 4a shows full-field displays for the starting point astigmatism and coma, while Figure 4b shows the full-field displays for the astigmatism and coma after introducing the specified freeform terms to all three surfaces. While astigmatism has decreased in scale from 20 λ down to 14 λ from Figure 4a to Figure 4b, FAFL Astigmatism is nevertheless still present. Further, FC coma has gotten worse, increasing from 4 λ to 5.5 λ.
At this point, notice the relative orientation of the FAFL astigmatism versus the FC coma. The FAFL astigmatism has a characteristic “C” shape and the FC coma points to the right. As can be seen in Figure 5 later, this orientation is the opposite of the orientation of FAFL astigmatism and FC coma which emerges during design in the Reflective Prism.
Further, neither FAFL astigmatism nor FC coma are fully corrected in the TIR Prism above. In other words, introducing Zernike coma and astigmatism does not correct these limiting aberrations. As the design continues, FAFL astigmatism and FC coma remain present except as balanced by higher order aberrations and as reduced by optimization of the powers, thicknesses, and tilts. Powers, thicknesses, and tilts were initially optimized for aberration correction in the TIR Prism.
That said, the TIR Prism can still be improved beyond the performance above. Other designers have designed improved TIR Prisms, and, in this paper, a TIR Prism is shown later which achieves the specifications in Table 1. However, the difficulty in correcting the FAFL astigmatism and FC coma induced by the geometry necessitates more complicated surfaces with increased freeform terms to create a design which meets the specifications of Table 1. Further, the TIR Prism performance is significantly limited in achieving uniformity in performance over the field, which is highly challenging with this form.
The TIR geometry is compared to the Reflective Prism, in which FAFL astigmatism and FC coma can be very easily corrected. The initial limiting aberration of the Reflective Prism is FC astigmatism, which is corrected by introducing Zernike astigmatism onto just one surface. After trying multiple surfaces, Mirror 2 proves to be very effective. The aberrations for the reflective prism after introducing Zernike astigmatism to Mirror 2 can be seen in Figure 5. Astigmatism improves from 100 λ down to 20 λ. Now, the system is limited by FAFL astigmatism and FC coma. The FAFL astigmatism has a characteristic “C” shape, while the FC coma points to the left – this relative orientation is the opposite of that for the TIR Prism.
Revisiting the theory, introducing Zernike coma away from the stop will create both FAFL Astigmatism and FC Coma. After investigation, it is enough to add Zernike coma to Mirror 2. Figure 6 shows the astigmatism and coma full-field displays after coma is added to Mirror 2. The magnitude of both the astigmatism and the coma have been cut in half, and both FAFL astigmatism and FC coma have been simultaneously removed. The simplicity of this correction is the main advantage of the Reflective Prism geometry. The ease of correcting FAFL astigmatism and FC coma is the primary difference between the Reflective Prism versus the TIR Prism. Finally, as shown through analysis of this design, achieving uniformity comes natural with this new Reflective Prism geometry.
From here further on the design, nodal aberration theory continues to drive the design. A design is created that matches the specifications with minimal freeform surfaces. Full results for the reflective prism are analyzed in the next section.
ANALYSIS OF THE FINAL TIR DESIGN
The TIR Prism design was concluded after introducing freeform departures of up the 36th FRINGE Zernike term on all three freeform surfaces, leading to a total of 63 freeform and aspheres variables across all three surfaces. Performance was analyzed over an effective 3mm diameter pupil centered at nine different locations within the design eyebox. These locations are chosen to ensure that performance is analyzed up to the edge of the 8mm diameter eyebox. The locations of the nine positions are: (0,0), (1.75,0), (-1.75,0), (0, -1.75), (1.75, -1.75), (-1.75, -1.75), (2.5, 0), (-2.5,0), and (-2.5, - 2.5). Figure 7 shows a diagram of these locations, as well as an XZ cross-section of the final reflective prism. Because of XZ plane-symmetry, sampling just the lower half of the eyebox is sufficient for analyzing performance.
The TIR Prism achieves the specifications in Table 1. With the significant increase in freeform terms, we achieve approximately the same performance as the reflective prism. The final design is shown in Figure 7, below. Figure 8 shows the best-case and worst-case full-field displays for RMS WFE of the TIR Prism. The best-case performance is achieved at the centered pupil, while the worst-case performance is achieved at the fully vertical offset pupil centered at (0, -2.5). Performance meets the specifications in Table 1 at all fields, but varies significantly over the field.
The optical performance in terms of MTF at 50lp/mm across all fields meets the specifications in Table 1. Figures 9 and 10 show the full-field displays of both sagittal and tangential MTF in the best and worst cases. The best-case performance averages at 63%, which is well above specifications, and is uniform over the field. The worst-case performance, on the other hand, is highly non-uniform. Further, while performance exceeds 10% at all field-points, virtually no tolerance exists if this specification is to be held for an as-built TIR Prism.
ANALYSIS OF FINAL REFLECTIVE DESIGN
The Reflective Prism was completed after introducing a total of fifteen Zernike terms across only two surfaces (Mirror 1 and Mirror 2), including eight rotationally invariant aspheric terms which could be reduced by using a more complex base surface. This number of terms is significantly fewer terms than the TIR Prism. The same pupil positions from Figure 7 above were used to analyze the Reflective Prism. As will be shown, the Reflective Prism has better performance than the TIR Prism. Performance is greatly improved versus that of the TIR Prism. An XZ cross-section of the final Reflective Prism design is shown in Figure 11.
Best performance was achieved at the centered pupil and worst performance was achieved at the fully vertical offset pupil located at (0, -2.5). Figure 12 shows full-field displays of the RMS WFE at these pupil positions. Performance is uniform over the field, with an average RMS wavefront error of 0.044 λ and 0.057 λ. The best-case performance of the Reflective Prism is mildly better than that of the TIR Prism. However, the worst-case performance of the Reflective Prism is nearly three times better compared to that of the TIR Prism.
Continuing with the Reflective Prism, optical performance in terms of MTF exceeds the specifications in Table 1. Figures 13 and 14 show full-field displays of the best-case and worst-case MTF at 50lp/mm. For almost all field points in both best and worst cases, performance is >50% MTF at 50lp/mm. Further, with few exceptions, this performance is highly uniform over the field. While the minimum performance of 18% does not leave much tolerance for performance drop in an as-built design, this minimum is achieved only at the edge of the field for the worst-case pupil position, rather than throughout the field.
Two freeform prism combiner designs were investigated, which are shown to be suitable for use in AR/VR applications. Both designs were purposely chosen to be monochromatic at λ = 587 nm to isolate and compare monochromatic aberrations. Results show that both designs achieve >10% MTF at 50 lp/mm over a 45° diagonal field, with an 8mm eyebox, configured for a micro-display with 10μm pixels.
It is shown, however, that removing the TIR requirement leads to a reflective prism design with far greater potential for aberration correction. Most importantly, the Reflective Prism yields high uniformity in performance unlike the TIR Prism. By leveraging nodal aberration theory and the Reflective Prism geometry, uniform high performance over the field was achieved using only two freeform surfaces with base spheres plus a total of fifteen freeform polynomial terms. The performance was compared to performance of the TIR Prism geometry that results in more complex surfaces and worse performance especially when it comes to uniformity over the field, which is an important finding of this investigation. The reduction in freeform complexity means the Reflective Prism is easier to manufacture and test, which is a key driving factor in freeform design.
This Research was supported in part by the National Science Foundation I/UCRC Center for Freeform Optics (IIP-1338877 and IIP-1338898). We thank Aaron Bauer for insight learned from discussion of the interaction of geometry and aberrations. We thank Synopsys for the student license of CODEV.
K. Takahashi, “IMAGE DISPLAY APPARATUS COMPRISING AN INTERNALLY REFLECTING OCULAR OPTICAL SYSTEM”. United States of America Patent 5,699,194, 16 December 1997.Google Scholar
A. Okuyama and S. Yamazaki, “Optical system, and image observing apparatus and image pickup apparatus using it”. United States of America Patent 5,706,136, 6 January 1998.Google Scholar
T. Togino and J. Takahashi, “Head-mounted display apparatus comprising a rotationally asymmetric surface”. United States of America Patent 5,959,780, 28 September 1999.Google Scholar
D. Cheng, H. Hua and Y. Wang, “Optical see-through free-form head-mounted display”. United States of America Patent US 9.239,453 B2, 19 January 2016.Google Scholar
H. Hua and J. Kuhn, “High-resolution freeform eyepiece design with a large exit pupil”. United States of America Patent US 2018/0045949 A1, 15 February 2018.Google Scholar
Synopsys, Inc., “Zernike Polynomials,” in CODEV Lens System Setup Reference Manual, 2015, p. Appendix A.Google Scholar
N. Takaki and J. Rolland, “Mathematical Properties of Describing Freeform Optical Surfaces with Orthogonal Bases,” Proc. SPIE 10590, International Optical Design Conference 2017, 105900U, 27 November 2017.Google Scholar
A. Bauer, E. Schiesser and J. P. Rolland, “Starting geometry creation and design method for freeform optics,” Nature Communications, (in press).Google Scholar
A. Broemel, C. Liu, Y. Zhong, Y. Zhang and H. Gross, “Freeform surface descriptions. Part II: Application Benchmark,” Advanced Optical Technologies, vol. 6, no. 5, 2017.Google Scholar