With the increase of consumer expectation and the commercialization trend of head-mounted displays (HMD), the need for developing compact, high-resolution optical see-through HMDs for augmented reality (AR) application has become more urgent than ever before. A typically optical see-through HMD system mainly consists a microdisplay, an eyepiece, and an optical combiner. The microdisplay renders a digital image, the eyepiece magnifies the small digital image on the microdisplay to create a virtual image appearing at a comfortable distance from the viewer, and the combiner combines the view of the virtual image with that of a real-world scene. Among these components, the combiner plays a critical role in determining the overall performance of an AR display. There are different kinds of optical combiners, such as a flat beamsplitter , a curved or freeform surface with beam splitting coating [2-3], holographic waveguide [4-5, 11] or geometrical waveguides [6-10, 13]. A freeform combiner can achieve large FOV and great image quality, but its size is typically bulky and it is heavy to wear. The holographic waveguide has been used in some commercial products such as Microsoft Hololens . However, its diffractive nature s makes it sensitive to incident angle as well as wavelength, and thus the FOV is limited and the severe chromatic aberration affects the image quality.
The geometrical waveguide is an alternative way of constructing the optical combiner. Several different types of geometrical waveguides have been discussed before, like using partially reflecting mirrors [7-8, 12] or surface structure mirrors [9-10, 13]. Compared with other alternative methods, the geometrical waveguide has several advantages. First, the chromatic aberration will not be dominant compared with holographic waveguide, which is due the nature of diffractive optical elements. Meanwhile, waveguide combiners can be ultra-thin with glasses-like thickness. While freeform eyepieces are typically at least tens of millimeters thick, geometrical waveguides can be within 2-5mm thick, and can be particularly light-weighted. Last but not least, compared with a single freeform combiner manufacture that will cost tens of thousands of dollars, the cost of a geometrical waveguide, especially the designs using microstructure mirrors can be quite low. Its substrate material is also easy to get and manufactured, which can be plastic like PMMA. However, designing an optimal geometrical waveguide with high image quality confronts several major challenges, one of which is to minimize stray lights caused by multiple reflections and maximize image uniformity and contrast.
In this paper, we present the design of an AR system consisting of a geometrical waveguide optical combiner with microstructure mirrors and a freeform collimator, in which we develop optimization methods to address the key challenge of achieving high out-coupling uniformity and efficiency. In Section 2, we introduce the basic geometry of the geometrical waveguide. Section 3 focuses on the waveguide design and optimization of achieving optimal waveguide out-coupling uniformity and efficiency. To make the image collimating optics compact, a freeform collimator with only a single plastic piece is designed and discussed in Section 4. Finally, the system assembly and 3D CAD model are proposed in Section 5. The final AR system has a 30° FOV diagonally with an angular resolution of 1.21 arcmin. It is compact, light-weighted and potentially low-cost.
BASIC SYSTEM CONFIGURATION AND GEOMETRICAL CALCULATION
Overview of the system
Figure 1 shows the optical design layout of an augmented reality (AR) system with an ultra-thin geometrical waveguide as the optical combiner. It has a glasses-like form factor and mainly consists three parts: the micro-display, image-collimating system and the geometrical waveguide. The micro-display is used to generate the virtual image seen by the viewers. It can be self-emissive type micro-display like OLEDs, or reflective type liquid crystal on silicon (LCoS) microdisplay with a light engine. In our design, we used an LCoS mircodisplay since it has brighter illumination, higher pixel resolution, as well as larger contrast ratio compared with the self-emissive microdisplays. Nevertheless, it can also be replaced by OLEDs to reduce the complexity of illumination as well as the system volume. The collimating system is a monolithic freeform prism consisting of multiple freeform surfaces. It is used to magnify the image on the microdisplay and collimate the image such that the magnified virtual image appearing at optical infinity, which ensures that the light from the microdisplay is efficiently coupled into the waveguide with parallel incidence.
The waveguide contains three parts based on their functionality: an in-coupling wedge, a guide substrate and an out-coupling microstructure mirrors. The in-coupling wedge, which is located at the far end of the waveguide, is to couple the collimated light from a microdisplay into the waveguide. By appropriately choosing the wedge angle, the incident angle of all fields can be larger than the critical angle when the light hits the bottom surface of the substrate, so that the collimated bundles can be propagated within the waveguide substrate by total internal reflection (TIR) until they finally reach the out-coupling zone. The out-coupling zone consists of an array of micro-mirrors with a width of several hundred microns to reflect the collimated light from the microdisplay. The micro-mirror array is angled with respect to the substrate, so that when the ray bundles reach the micro-mirrors, the rays will deviate from the TIR condition and transmit through the other side of the waveguide and finally reach the eye pupil of the viewer, as shown in Fig. 1.
The array of microstructure mirrors is arranged periodically with small flat transparent surfaces in between each tapered micro-mirrors, as shown in the inset in Fig.1. The collimated image is coupled into the eye pupil by the micro-mirrors, while the real-world scene can be seen through the flat transparent surfaces. Thus the virtual image and the see-through scene can be combined with this half-reflection-half-transmission out-coupling structure. Since the micro-mirrors are too close to the eyeball and their size are too small compared with the size of eye pupil, the viewer is unable to focus at the mirror and see the occlusion artifact caused by the mirrors.
System geometry: in-coupling wedge
Since the out-coupling microstructure mirrors are directly engraved on the waveguide substrate, the geometry of the waveguide should be well designed in order to appropriately guide the rays in and out of the waveguide. As discussed in the previous sub-section, an in-coupler wedge should be adopted to couple the rays from the collimator into waveguide, so that collimated ray bundles can be propagate within the substrate. In our design, a right angle wedge in-coupler is designed to couple the rays into waveguide, which is shown in Fig. 2. The entrance pupil of the waveguide in the YOZ plane is set to be at the front surface of the in-coupling wedge. The wedge angle β is set to be the complement angle of the central field angle of incidence to ensure the central field is normally incident upon the wedge surface. To optimize the central field coupling efficiency, it is preferable to have a zero gap between the bundle cross section on the substrate surface at each reflection, as shown in Fig. 2(a). To enlarge the area of cross section on the substrate surface and eliminate the gap, the height of the in-coupler t1 should satisfy:
where t is the thickness of the waveguide substrate. Under this condition, the central field out-coupling efficiency is independent with the length of the ray path in the waveguide. However, the other fields, due to the coupling-in angle mismatch, will inevitably have cross section gap ΔW, as shown in Fig. 2(a) and 2(b). With appropriate inter-coupler distance (the distance between the centers of the in-coupler and out-coupler) and the out-coupler arrangement, the efficiency degradation of the off-axis fields can be refined and compensated.
System geometry: waveguide substrate
The refractive index and dimensions of the waveguide substrate can affect the coupling efficiency. Higher refractive index materials tend to be favorable due to the smaller angle of refraction within the substrate and the larger critical angle. However, higher refractive index materials may not be easy to obtain and process. Therefore, we selected PMMA as the substrate material. Meanwhile, the waveguide thickness is one of the key factors of the configuration and will be treated as an optimization variable (Section 3).
System geometry: out-coupler
The size, angle and location of the out-coupler are important factors to the out-coupling efficiency and uniformity. First, the out-coupler should be large enough to couple all the fields into the eye pupil. It also cannot be too large, since large reflective area will cause significant degradation of the out-coupling efficiency, especially for the fields that propagate deep inside the waveguide. In this case, the width of the out-coupler, d, is
where ERF is the eye relief, t is the waveguide thickness, ω and ω′ are the emerging and refractive angles of the full field. In our design, ERF is set to be 23 mm to accommodate most of eyeglasses . Secondly, to ensure that the central field is normally incident onto the eyebox, the slanted angle of the micro-structure mirror (μ, as shown in Fig. 1) should also satisfy the relation
Finally, the inter-coupler distance is also important, for it determines the optical path difference (OPD) of rays propagating in the waveguide.
Other geometry parameters, like the arrangement of the microstructure mirror (e.g. width and displacement of the micromirror) are also important factors to the system performance. Details of how to select the appropriate values for the variables in the arrangement are discussed in Section3.
Stray light analysis
Besides tracing the normal ray path, tracing source of stray light is essential for the design of waveguide combiners, since the stray light will also depredate the image contrast ratio. The stray light mainly comes from an unexpected extra reflection before the rays entered into the out-coupling region, as shown in Fig. 3(b). Rays that hit and are reflected by the substrate upper surface before they are is reflected by the slanted micro-structure mirrors become stray light. This extra reflection makes the reflective angle changes and makes the rays TIR again rather than being coupled out like the normal rays. The ray paths of the stray light are shown in Fig. 3(c). A small portion of the rays which should be coupled out from the left side field (as the blue field in Fig. 3(a)) was accidentally reflected twice at the top surfaces and some of them are out-coupled to the exit pupil after the next reflection. The stray light outcoupling angle in the substrate ω′s can be calculated by
Where α is the incident angle of the central field. If β = 60°, then α = μ = 30°, ω′s = ω′, which means the stray light and the normal ray are mirror symmetry with respective to the central field. Notices that the stray light will only affect the right side fields reflected between Region2 and Region3 shown in Fig. 3(a), since the twice reflection on top boundary will only happen when
WAVEGUIDE DESIGN PROCESS
Based on the analysis of waveguide configuration in Section 2, we can classify the waveguide parameters into two groups: one is called waveguide parameters, which is the geometrical dimensions and angles in the scope of the waveguide, such as the in-coupler wedge angle β, the waveguide thickness t, the inter-coupler distance H, the substrate refractive index n, etc. The dimension of the waveguide and ray path angles can be fully determined by these waveguide parameters. Another group is called out-coupler parameters, which are the parameters describing the micro-structure mirror arrangement and size, such as the placement of the mirror, the micro-mirror width, the displacement distribution center, etc. Both types of parameters will influence the out-coupling efficiency and uniformity. However, optimizing the waveguide configuration by considering all the parameters together will make the potential solution pool too large and make it hard to find an optimal solution. In this case, we considered the two groups of variables separately; meanwhile using both global and local optimization to narrow down the solution set to find the optimum configuration.
Global search of waveguide parameters
Three geometrical parameters determine the basic configuration of the waveguide: in-coupler wedge angle β, the waveguide thickness t, as well as inter-coupler distance H, which are shown in Fig. 4. The in-coupler wedge angle β determines the TIR angle of the ray bundles when propagating in the waveguide. The value of β should allow all the fields incident upon the substrate with angles larger than the critical angle of the substrate. In our design, the refractive index of PMMA equals to 1.4936 and its critical angle θc=42°. Since the half field of view (HFOV) is ω=13°, the maximum field angle in the substrate is ω’=8.66°. This means the in-coupler slanted angle β must be greater than 50.66° in order to allow all the fields to be TIR in the substrate. In our design, the searching range of the wedge angle β is set to between 56° to 64° with a searching step of 0.8°. The waveguide thickness t also determines the ray path and propagation distance in the waveguide. It cannot be too bulky to wear, thus it is set to be varied from 1.6mm to 3.2mm with the searching step of 0.2mm. The inter-coupler distance H determines the length of the waveguide, and location where ray bundles are coupled out. It should be long enough to leave the space for the clearance of the eyeglasses frame as well as the space for see-through FOV, but also cannot be too long as well for the system volume consideration. The range between 24mm to 32mm with step of 1mm is setup in our optimization algorithm. The searching ranges and steps of the waveguide parameters are concluded in Table 1.
The searching range and step length of the waveguide parameters.
|Variable name||Searching Range||Searching interval|
|slanted angle β||56° - 64°||0.8°|
|waveguide thickness t||1.6mm - 3.2mm||0.2mm|
|central distance H||24mm - 32mm||1mm|
Local optimization of out-coupler parameters
After the global search of the waveguide geometrical parameters, the best 50 configurations with minimum merit function values (defined in Section 3.3) are selected to be the solution pool. Local optimization is then adopted to those configurations to further searching for the best arrangement of the out-coupler. Several out-coupler parameters are selected to be variables during the local optimization: the micro-mirror width c, the periodic offset of the micro-mirror, and the displacement of the micro-mirrors. We investigated both the uniform displacement and nonlinear displacement micromirror structures and found that the non-linear displacement distribution shows significant advantage in terms of the field uniformity. That is because the fields propagating longer in the waveguide tend to have more power losses during the propagation before reaching their out-coupled locations. The micro-structure mirrors distribution further away from the in-coupler side should be much denser than the closer side, to avoid redundant ray losses before they reached their out-coupling positions and ensure higher coupling proportion of the right side fields. The micro-mirror displacement x is defined as a 3rd order polynomial which is expressed as
Where i is the mirror count. A is the offsets of the central, B is the linear term of the displacement, C and D are the non-linear displacement term. Both the linear and nonlinear parameters are set as variables in the optimization process. The variable ranges and initial values are concluded in Table 2.
The searching ranges and initial values of the out-coupler parameters.
|Variable name||Upper and lower bound||Initial value|
|Micro-mirror width c||0.02mm – 2mm||0.3 – 1.4mm, search step 0.3mm|
|X Offset (A)||Out-coupler half width d/2 ± c||Out-coupler half width d/2|
|Linear displacement B||0 - 4||5·c|
|Non-linear displacement C||0 - 4||1|
|Non-linear displacement D||0 - 4||1|
Performance evaluation and merit function
Since our goal is to get a virtual image coupled out with high uniformity and high image contrast across the eyebox, four factors should be considered to evaluate the performance of different configurations: (1) the coupling uniformity across the FOV, (2) the overall image out-coupling efficiency, (3) the uniformity across the eyebox and (4) stray light. Since in this design, the eyebox has a comparable size with the eye pupil which is a circular area with 4mm diameter, pupil uniformity can be treated as a minor factor. Also, small portion of stray light with moderate contrast influence can be acceptable. In this case, the factor (3) and (4) can be set as the threshold factors to narrow down the solution pool, whereas the other two factors define the merit function. The merit function can then be defined as
Where S1-S4 are mathematical expression of the four influence factors; W1 and W2 are the weighting factor of S1 and S2. By choosing different ratios of the weighting factor, the weighting between the coupling uniformity and efficiency can be varied.
To model the waveguide configuration, we use LightTools® to setup the basic waveguide geometries. A 3-D structure is created on the top surface of the substrate to model the micro-structure mirrors, as shown in Fig. 5. It is set as an array of right triangular holes with polynomial displacement. For the initial global searching step, seven collimated surface sources are overlapped at the in-coupling surface with different field angles. The sampling angles of incidence are set to be the fraction of 0, ±0.3, ±0.6 and ±1 of the full field angle. Ten surface receivers with 2mm radius are set to be located at the exit pupil of the waveguide, which is about 23mm away from the center of the out-coupler. Seven of them are used to receive the normal out-coupling rays from each field, whereas the remainingthree receivers are used to receive stray light from the negative angle fields. The global search of the waveguide geometrical parameters is assigned by using LightTools-MATLAB API. During the global search, the variable values shown in Table 1 are assigned in LightTools by MATLAB connection, and the simulation results will be exported back to MATLAB for the performance evaluations.
The local optimization of the out-coupler parameters is performed with the LightTools built in optimizer with user-defined merit function in LightTools. The initial values and the limits of the variables shown in Table 2 are preset by LigthtTools-MATLAB API before each optimization loop. The starting points are selected based on 50 optimal results of global searching of the waveguide geometrical parameters. For each configuration, four different initial micro-mirror widths are preset before local optimization. More fields are sampled later for further improving the field uniformity.
The final result is selected based on the filtering condition and merit function defined in Section 3.3. Several configurations are selected based on different weighting factor ratio after local optimization. Those configurations are further evaluated by their overall performances. Table 3.1 and 3.2 shows geometrical and out-coupler parameters of the optimal waveguide configuration.
Geometrical parameters of the optimal waveguide configuration
|Geometrical Parameters||slanted angle β||waveguide thickness t||central distance H||In-coupler height t1||Micro-mirror slanted angle|
Out-coupler parameters of the optimal waveguide configuration
|Out-coupler Parameters||Micro-mirror width c||Coupler zone width||X Offset (A)||Linear displacement B||Non-linear displacement C||Non-linear displacement D|
Figure 5 shows the geometry of the optimal waveguide and the Monte Carlo ray tracing results. Notices that we only simulate the ray path in YOZ plane to simplify the raytracing model and save raytracing time, since the other field plane will have similar efficiency distribution in a one-dimensional waveguide. The width of micro-mirrors is 0.92mm with more intensive distribution at the far end of the waveguide. The out-coupling efficiency and stray light ratio of each field received by the receiver at the eye position are shown in Table 4.1. The average out-coupling efficiency is 32.44%, and the peak to valley variation is 12.88%. The average stray light is 0.8%. The residual field non-uniformity can be further balanced by digital compensation.
The efficiency and stray light ratio of each field at the eye position.
|Field -1||Field -0.6||Field -0.3||Field 0||Field +0.3||Field +0.6||Field +1||Stray light 0.3||Stray light 0.6||Stray light 1|
The statistical result of the merit function.
|Average out-coupling efficiency||0.3244|
|P-V out-coupling efficiency||0.1288|
|RMS out-coupling efficiency||0.0445|
|Average Stray light percentage||0.8%|
DESIGN OF FREEFORM COLLIMATOR
The image collimator, which is located between the microdisplay and waveguide in-coupler wedge, is used to project the mircodisplay image to the infinity and couple the collimated ray bundles into the waveguide. To obtain high image resolution, a compact, high image quality collimator is needed. Several key features and requirements of the collimator increase the design difficulties. First of all, to couple the image into waveguide without light losses, the collimator and waveguide pupil should match. Since the one-dimensional waveguide only guide light in one direction, the XOZ and YOZ exit pupil of the collimator should be located at different location. In order to guide all the image fields into the exit pupil of the waveguide, the exit pupil in YOZ plane of the collimator should be located at the waveguide in-coupler wedge (the entrance pupil of waveguide), whereas the collimator exit pupil in XOZ plane should be located at the system’s exit pupil plane. Secondly, Since LCoS needs an external light engine for its illumination, long object distance is required to leave sufficient space for inserting the PBS and light engine. Thirdly, owing to the reflective nature of LCoS displays, it is highly desirable to achieve telecentric at the object space to get uniform illumination modulation.
Figure 6 shows a side view of the waveguide in XOY plane at the end cross section. Since the microstructure mirrors will not change the reflective angle in x-direction, the distance between the XOZ and YOZ pupil plane ERFx can be easily calculated by
Where the number of TIR time r = 9, in-coupler size e1 = 3.0808mm, waveguide thickness t = 2.2mm, ERF = 23mm, ω is the field angle in XOZ plane.
We choose to use HED-5216 color LCoS microdisplay by Holoeye as the virtual image generator. This LCoS is 0.55’’ in diagonal with the resolution of 1280*768 (SXGA) and the pixel pitch of 0.96 micron. Its response time is fast enough to handle the three time-sequential color channels up to 60 Hz RGB frame rate. To make the overall system compact, the configuration of a freeform prism is employed as the image collimator, which is shown in Fig. 7. The entrance pupil diameter of the collimator is set to be 3.6818mm to match with the waveguide entrance pupil size. The representative wavelengths are set to be 0.47, 0.55 and 0.61μm according to the dominant wavelengths of the RGB LEDs and is weighted as 2:3:1 based on the relative luminance response of the human visual system. The collimator are set up and traced back from the exit pupil to the micro-display in Code V. Since the prism is YOZ plane symmetry, only +X fields are sampled for the design. 25 fields with different X and ±Y field angles are sampled. Five zoom configurations are used to model the x-direction pupil shift at the YOZ pupil plane. The collimator contains three surfaces as shown in Fig. 7(c). Surface 1 and 2 are described as 8th and 6th order of XY-polynomial, whereas S3 are described as 8th order anamorphic aspheric surface to contribute different powers in two orthogonal directions. The ray emitted from the microdisplay will firstly refracted by surface 3, then reflected by surface 1’ and 2, and transmitted through surface 1 and finally reach the exit pupil location. Surface 1 and 1’ are the same surface, so TIR condition must be satisfied at 1’ surface in order to get the rays reflected. Surface 2 are a coated reflective surface. The system is constrained to be image space telecentric, and the image clearance is set to be larger than 10mm.
Figure 8 and 9 shows the polychromatic modulation transfer function (MTF) and the distortion grid of the freeform collimator. The MTF values are all above 0.25 at the cut-off frequency of 52 cycles/mm over the entire FOV. The system distortion is well corrected, less than 5% over the field. The residual distortion can be corrected by image processing to pre-warp the original image.
Figures 10 and 11 shows the 3D assembly model of the microstructure mirrors AR system. The waveguide has the dimension of 18mm (W) *2.2mm (H) * 32mm (L) and the freeform collimator has the dimension of 21mm (W) * 11mm (H) * 24mm (L). The overall system is 70mm * 32.5mm * 24mm.
In this paper, a geometrical waveguide and a freeform prism are used to develop an ultra-thin, super-compact, light-weighted and high image performance AR system. The geometrical waveguide using microstructure mirrors as the out-coupler is employed as the optical combiner. Both global and local optimization are used to search for the best micromirror construction with optimal coupling uniformity and efficiency. Stray light is also analyzed in this paper. A freeform prism is designed to make the image projection engine super compact while maintaining high image quality. A simple mechanical model of the prototype is also proposed. The system has FOV of 30° with angular resolution 1.21 arcminutes.
Heinrich, M. J., and Olsson, M. I., “Wearable display device,” U.S. Patent Application, No. 29/412, 365 (2012).Google Scholar
Mukawa, H., Akutsu, K., Matsumura, I., Nakano, S., Yoshida, T., Kuwahara, M., and Ogawa, M., “8.4: distinguished paper: a full color eyewear display using holographic planar waveguides,” SID Symposium Digest of Technical Papers, 39(1), 89–92 (2008).Google Scholar
Amitai, Y., “P-27: A Two-Dimensional Aperture Expander for Ultra-Compact, High-Performance Head-Worn Displays,” SID Symposium Digest of Technical Papers, 36(11), 360–363 (2005).Google Scholar
Amitai, Y., “P-21: Extremely Compact High-Performance HMDs Based on Substrate-Guided Optical Element,” In SID Symposium Digest of Technical Papers, 35(1), 310–313 (2004).Google Scholar
Pascal, B., Guilhem, D., and Khaled, S., “Optical guide and ocular vision optical system,” U.S. Patent, No. 8,433,172. (2013).Google Scholar
Sarayeddline, K., Mirza, K., Benoit, P., and Hugel, X., “Monolithic light guide optics enabling new user experience for see-through AR glasses,” Photonics Applications for Aviation, Aerospace, Commercial, and Harsh Environments V, 9202, 92020E (2014).Google Scholar
Kress, B. C., and Cummings, W. J., “11-1: Invited Paper: Towards the Ultimate Mixed Reality Experience: HoloLens Display Architecture Choices,” In SID Symposium Digest of Technical Papers, 48(1), 127–131 (2017).Google Scholar
Mirza, K., and Sarayeddine, K., “Key challenges to affordable see through wearable displays: the missing link for mobilearmass deployment,” Internal Technical Paper-OPTINVENT SA, (2012)Google Scholar
Rolland, J. P., and Hua, H., “Head-mounted display systems,” Encyclopedia of optical engineering, 1–13 (2005).Google Scholar