Front-lit LCOS for AR displays

Abstract. Augmented reality (AR) displays are recognized as an important human–machine interface for next-generation computing platforms. They consist of a micro projector and a waveguide combiner. Researchers are looking for a compact and powerful projector to power up the waveguide combiner. The projector itself is comprised of microdisplays [liquid-crystal-on silicon (LCOS), MicroLED, laser beam scanning, and organic light-emitting diode] that generate the virtual image and a collimating optics module that collimates the image toward the waveguide’s entrance aperture. However, the microdisplays mentioned above suffer from different problems, such as being too bulky, having too low brightness, reliabilities issue, or too high power consumption. In particular, projectors using LCOS are too bulky and heavy to fit into an AR glasses form factor. Therefore, we proposed a new illumination system called front-lit for LCOS. In the front-lit system, the conventional polarization beam splitter (PBS) in the LCOS projector is replaced by a polarized waveguide plate with a micro mirror array. The polarized waveguide functions like a PBS. It reflects the s-polarized beam and transmits to the p-polarized beam. Moreover, the polarized waveguide input beam is shaped to a profile with two intensity peaks from a Lambertian profile by our coupling lens, which is located between the LED and waveguide. Having this profile, the polarized waveguide efficiency is improved, and a larger display illumination area is obtained. From our real measurement results, it is proven that the front-lit LCOS significantly reduces the size and weight of the illumination system. It weighs less than 1 g, and the size of the full module is as small as 0.47cc. The color filter front-lit LCOS and color sequential LCOS deliver 30,000  nits  /  175  mW and 100,000  nits  /  300  mW, respectively. The color sequential front-lit LCOS also provides high image quality with a 500:1 contrast ratio and 140% sRGB color gamut.


Introduction
Augmented reality (AR) is widely recognized as the next-generation computing platform.It may replace smart phones and computers as the most important human and machine interface.In AR, information is presented to viewers with virtual objects such as graphics or captions that immerse them in real environments without interfering with their vision.Clearly, the most essential component of AR is the near-eye display (NED), which is worn by the viewer and is used to combine real and virtual imageries together.Although AR NEDs offers a replacement for smartphones and computer monitors and provide a next-level visual experience to viewers, designs for AR NEDs involve trade-offs between several different metrics, including resolution, eye box, form factor, field of view (FOV), eye relief, brightness, full color, battery life, etc.All of the mentioned metrics are actually correlated.Therefore, the greatest challenge in AR NEDs is not in optimizing any individual metric, but instead simultaneously providing a wide FOV, high resolution, wide eye box, and slim form factor device.
A small form factor AR NED is usually composed of three main parts: (1) a transparent waveguide plate, (2) a collimating optics module, and (3) a micro display unit.
The collimating optics module acts as a bridge between the transparent waveguide and the micro displays.The transparent waveguide is capable of in-coupling the virtual image into the waveguide, transmitting light waves with a total internal reflection (TIR), and outcoupling the virtual images into the user's eyes.Based on the coupling components used, waveguides can be categorized into two main types: reflective and diffractive.
In reflective waveguiding AR NEDs such as Epson's Moverio, the molded plastic substrate acts as the waveguiding for the virtual image, and one single semi-reflective mirror is placed in front of the human eye to reflect the virtual image to the viewer.To increase the eye-box and keep the AR NEDs as compact as ordinary glasses at the same time, multiple semi-reflective reflectors can be deployed.Several reflective waveguides with a one-dimensional (1D) exit pupil expander (EPE) have been reported. 1Amitai 2 proposed a reflective waveguiding two-dimensional (2D) EPE as well.
The diffractive waveguiding AR solution makes use of the diffractive optical element. 3,4To mitigate the chromatic aberration caused by the diffractive grating, diffractive waveguiding has to combine two or three layers of waveguiding structures targeting red, green, and blue lights. 5ach of these technologies requires different micro display characteristics.For example, a single semi-reflective mirror and 1D EPE waveguide require a larger display panel and a large emission cone for large FOVAR NEDs.On the other hand, AR NEDs with a 2D EPE waveguide require a very high brightness and compact display panel.
In this paper, we propose a color-filter front-lit liquid-crystal-on silicon (CFFLCOS) 6 to meet the requirements of a single semi-reflective mirror and 1D EPE waveguide AR NEDs.In the second section, we demonstrate our new development of color sequential front-lit LCOS (CSFLLCOS).The CSFLLCOS is <0.47 cc with a resolution of 1024 × 1024.It is capable of delivering over 100,000 nits for 2D EPE waveguide AR NEDs.

Front-lit LCOS
In a front-lit device, a 0.7 mm thick polarized waveguide replaces the conventional polarization beam splitter (PBS) cube.Therefore, the full display system can be as small as <0.5 cc. Figure 1 shows the comparison of a conventional 2.5cc illumination system of the projector (right red box) and a front-lit LCOS device (left yellow box).The conventional illumination projector is composed of a PBS, LED, illumination optics, and color filter LCOS (CFLCOS).The color filter front-lit LCOS (CFFLLCOS yellow box) is shown on the left side.The volume of CFFLLCOS is less than 0.25cc, including the front-lit and color filter LCOS.Clearly, a compact and light weight CFFLLCOS is more favorable to AR glasses designs.

Color Filter Front-lit LCOS
The structure of the color filter front-lit LCOS is shown in Fig. 2. It consists of six parts: (1) LED light source, (2) coupling lens, (3) input polarizer [(i.e., wire grid film (WGF)], (4) polarized waveguide, (5) LCOS panel (i.e., color filter LCOS or color sequential LCOS), and ( 6) analyzer (i.e., absorptive polarizer).The white light, beam S, emitted from the LED is collected by the coupling lens.The Lambertian light beam S from the LED is shaped to a profile with two intensity peaks by the coupling lens.We call the two intensity peaks beam A and beam A 0 .Both beam A and beam A 0 are polarized by the WGF, which is located between the polarized waveguide and coupling lens.The reflected polarized ray, which is bounced back by the WGF, is recycled in the coupling lens system with a suitable conversion surface.Beam A-s and beam A-s 0 are then injected into the waveguide plate.The A-s beam hits the micro mirror array inside the polarized waveguide without any further optical bouncing within the waveguide.It is then steered toward the color filter LCOS CFLCOS by the micro mirror array.The A 0 -s beam propagates across the polarized waveguide by one additional TIR.Finally, beam A 0 -s hits the micro mirror array, where it is located further away from the WGF, and is reflected to the CFLCOS similar to beam A-s.Because beam A 0 -s propagates further away from the LED, it allows front-lit to illuminate a larger display area.Both A-s and A 0 -s are converted into p-polarized light (B-p and B 0 -p) by the liquid crystal layer in the CFLCOS.
The CFLCOS is equipped with a pixelized red (R), green (G), and blue (B) color filter and mixed twisted nematic (MTN) liquid crystal configuration. 7,8The white light input source is  converted into the RGB color image by the pixelized red, green, and blue color filter and the 2.5 μm thick liquid crystal layer.The reflectance RðλÞ of the MTN liquid crystal mode is optimized by selecting the proper twist angle φ, retardation Γ, and beta angle β. 9,10 The average reflectance R avg of the CFLCOS is defined as where λ is the wavelength, λ ∈ ½400 nm: : : 700 nm, λ2 ¼ 700 nm, λ1 ¼ 400 nm, and yðλÞ is the color matching function.Figure 3 shows the reflectance of CFLCOS.Because two-thirds of the input light is absorbed by the red, green, and blue color filters, the average reflectance of the CFLCOS is about R avg ¼ 14% for a CFLCOS with a 7.5 μm RGB pixel.
The reflected beams B-p and B 0 -p then pass through the polarized waveguide because the micro mirror array is transparent to p-polarized light.The analyzer on top of the polarized waveguide acts as a clean up polarizer, which improves the contrast ratio of the device.Such a system enables the observer (Fig. 2) to see a color image as the usual direct view display [Fig.1(b)].
Owing to the high efficiency of white LEDs, the brightness of CFFLLCOS can be as high as 30,000 nits under 175 mW LED power.Figure 4(a) shows the brightness versus LED power.Figure 4(b) shows the corresponding emission cone of the CFFLLCOS.The full wave half maximum (FWHM) of the emission cone is AE15 deg.Furthermore, by adjusting the optical element in the coupling lens, the emission cone of the CFFLLCOS can be adjusted to cope with different AR-NEDs waveguide requirements.For example, we demonstrate an asymmetric emission cone with FWHM (vertical AE30 deg, horizontal AE12 deg).It is particularly useful for a large FOV, i.e., 35 deg, 1D EPE waveguiding AR NEDs.

Color Sequential Front-lit LCOS
As shown in previous section, the color filter front-lit LCOS is compact (i.e., pixel size is 7.5 μm) and bright (30,000 nits) enough to power the AR NEDs using a single semi-reflective mirror or 1D EPE waveguide.However, the 2D EPE waveguide AR NEDs require microdisplays with an even higher brightness (i.e., >10;0000 nits) and smaller pixel size (i.e., <5 μm).The optical performance of color filter front-lit LCOS has difficulty meeting these requirements.
It is found that the pixel size of the color sequential LCOS (CSLCOS) can be as small as 4.25 μm.Furthermore, the reflectance of the CSLCOS is about R avg > 70%, which is about six times higher than the reflectance of the CFLCOS (Fig. 5).Therefore, once the CSLCOS is equipped with front-lit, it will become an ideal image source for 2D EPE waveguide AR NEDs.To prevent unfavorable color mixing of the color sequential display, the response time of the color sequential LCOS has to be shorter than the frame time of 2.7 ms (i.e., 1 s/360 fps).In Fig. 6, we demonstrate the gray to gray response time of the thin cell gap of the 1.6 μm analog color sequential LCOS (CSLCOS).All gray to gray transitions shown in Fig. 6 are defined as graylevel 1 to graylevel 2. It can be found that, having the thin cell gap color sequential LCOS, all of the gray to gray response times are shorter than 2.5 ms.
Figure 7 shows the structure of CSFLLCOS.The working principle is similar to CFFLLCOS as discussed in previous section.Nevertheless, the white LED and CFLCOS are replaced by an RGB mini LED light bar and CSLCOS, respectively.The RGB mini LED light bar is composed of three red, four green, and four blue mini LED chips.A CSFLLCOS is built successfully.The real working sample is shown in Fig. 8(a).A clear full 1024 × 1024 color sequential display image can be captured from that working CSFLLCOS, as shown in Fig. 8(b).outermost dimension of the full module is shown in Fig. 8(c), with length L ¼ 13.7 mm, width W ¼ 9 mm, and total thickness being less than H ¼ 3.8 mm.The overall volume of the module is about 0.47cc.Because the width of the device is less than 9 mm,  Owing to color sequential lighting mechanism and high contrast device, the color gamut is over 140% of sRGB standard.
The brightness of the color sequential front-lit LCOS can be adjusted by the RGB LED power.Figure 9(a) shows the brightness to RGB LED power curve.The maximum brightness is about 150,000 nits with an LED power of 500 mW.The white balance can be maintained at 6800K in full brightness adjustment range (15,000 nits to 150,000 nits).The emission cone of the color sequential front-lit LCOS is shown in Fig. 9(b).The FWHM of the emission cone is > AE 12 deg.

Conclusions
The main parameters of CSFLLCOS, CFFLLCOS, and conventional LCOS projector (illumination part) are summarized in Table 1.The PBS in the conventional LCOS projector is replaced by the polarized waveguide in the front-lit device, so the module size of front-lit device is much smaller than the conventional LCOS projector.The CFFLLCOS is suitable for AR NEDs using a single semi-reflective mirror or 1D EPE waveguide.Moreover, the color sequential front-lit LCOS, which can deliver very high brightness >100;000 nits, meets the display module requirements of 2D EPE waveguide-based AR-NEDs.

Discussion and Future Works
There are several front-lit design considerations that are important in real applications.Because the micro-mirror array is located between the LCOS image plane and the collimating optics, the image quality is very sensitive to the presence of artifacts in the micro-mirror array.To avoid such issues, the micro-mirror array is positioned far away (1 to 2 mm) from the image plane (liquid crystal layer) of the LCOS.Therefore, the artifacts on the micro mirror array will not form the image by the collimating optics.On the other hand, the steering properties of the micro mirror array must adjust to having good transparency to p-polarized light.It is also crucial to avoid ghosting images.Furthermore, the illumination cone on the LCOS panel is controlled by the coupling lens.The coupling lens has to compensate for the illumination cone angle variation across the LCOS.Currently, the nine points uniformity of the front-lit device is about 70% (Table 1).The polarization recycling will be further optimized in the future, and it is expected that the optical efficiency of future devices can be 1.5 times higher.

Fig. 1
Fig. 1 (a) Size comparison between CFFLLCOS (left yellow box) and conventional CFLCOS projector (right red box) and (b) a real image of color filter front-lit LCOS.

Fig. 3
Fig. 3 Reflectance of CFLCOS with full white image.

Fig. 6
Fig. 6 Gray-to-gray response time of an analog CSLCOS.

Fig. 8
Fig. 8 (a) The color sequential front-lit LCOS with resolution 1024 × 1024.(b) A real display image captured from the working sample.(c) The dimensions of the display module are L ¼ 13.7 mm, W ¼ 9 mm, and H ¼ 3.8 mm.

Fig. 9
Fig. 9 (a) The brightness emission of the color sequential front-lit LCOS under different RGB LED power levels.The white balance stays at 6800K.(b) The emission cone of the color sequential front-lit LCOS.

Table 1
Main features of CSFLLCOS, CFFLLCOS, and conventional LCOS projector.