During the history of display fields, there have been a desire to implement ideal three-dimensional (3D) displays. The ideal 3D displays may provide full of physiological cues including binocular disparity, motion parallax, and focus cues. Several technologies have been introduced and studied for the ideal 3D displays: light field near-eye stereoscopes,1–3 super multi-view displays,4 holographic displays,5–7 and varifocal displays.8–10 Although these approaches could support most physiological cues, it is still challenging to satisfy commercial demands in resolution, depth of field, form factor, eye-box, field of view, and frame rate.
According to the previous works related to 3D displays, we have observed trade-off problem among several terms that determine display performance. Light field displays2, 11–13 usually suffer from trade-off between the spatial and angular resolution, which limits depth of field. In holographic display,5–7 it is hard to achieve large field of view and enough eye-box, simultaneously. Varifocal displays8–10 also have trade-off problem among frame rate, spatial resolution, depth of field, and accuracy of focus cues.
Here, we propose shape scanning displays that practically achieve great display performance in those terms by combining tunable lens and fast spatially adjustable backlight (FSAB). Shape scanning displays may have extremely large depth of field (10cminfinity) without loss of frame rate or resolution, and enough eye-box (7.5mm) with moderate field of view (30°). According to the optical design of the FSAB, shape scanning display may support up to 120 planes within the addressable depth of field.
In summary, we introduce a novel 3D display technology called shape scanning displays that present superior performance in resolution, depth of field, and focus cue reproduction. It could be efficient solution for vergence-accommodation conflict as providing accurate focus cues. In addition, shape scanning displays have a lot of potential to be applied for various field in 3D displays including head-up displays, tabletop displays, as well as head-mounted displays. Due to the versatility of shape scanning displays, it is feasible to apply this methodology for advanced applications such as vision correction14 and high dynamic range displays.15, 16
Light Field Displays
Light field displays have a long history from glasses-free 3D displays to near-eye tensor displays. Light field displays aim to express volumetric shape of a real object by reconstruction of light field from the object. There have been several approaches to reconstruct the light field: parallax barrier, integral imaging,17 multi-view,18 and tensor displays.11, 12 Although these approaches have different optical structure and working principle, they share a similar trade-off problem between spatial and angular resolution.
In order to reconstruct light field considered as 4-dimensional variable, light field displays should divide display pixels according to the corresponding angular direction. Thus, spatial resolution is sacrificed for angular resolution, which becomes a barrier for commercialization of light field displays. Tensor displays could alleviate this trade-off problem by compression of light field that show clear tendency according to the viewing direction. However, tensor displays still suffer from the trade-off problem because actual amount of light field information is not increased.
Holographic displays reconstruct the wavefront from a volumetric object rather than light field. Modulating intensity or phase profile of incident coherent light source, holographic displays may generate a desired wavefront. Recent researches applied holographic displays for near-eye displays6, 19 and head-up displays.5, 20 Compared to light field displays, holographic displays have advantages in depth of field and occlusion effect. However, holographic displays usually have limited viewing window or eye-box.
As viewing window of holographic displays is determined by the size of spatial light modulator, the most feasible application of holographic displays is near-eye displays where viewing window is referred to as field of view. Field of view of holographic displays could be enlarged by using a floating lens. In this case, however, eye-box is sacrificed for enlargement of field of view. Therefore, holographic displays suffer from another trade-off problem between field of view and eye-box. Note that accommodation cue could be obscure if eye-box is too small to provide clear focus blur effect.
Instead of light field or wave reconstruction, varifocal displays aim to form physical layers for depth expression. In varifocal displays, volumetric objects are reconstructed via synthesis of layer images on different depths. When each layer image is optimally rendered, varifocal displays are known to be able to provide continuous accommodation cues. In order to implement and synthesize multi-layer images, we may stack multiple display layers by using a beam combiner,10, 21 or employ focus-tunable lens and temporal multiplexing.22, 23
In varifocal displays, the number of layers is the most important term to determine display performance. However, it is challenging to enlarge the number of layers without sacrifice in form factor or frame rate. If using multiple beam combiners, it is hard to design compact optical system. When employing focus-tunable lens and temporal multiplexing, the number of layers involves the decrease in the frame rate because state-of-the-art display panel usually support up to 240Hz. Thus, 4-6 layers are considered as maximized specifications for varifocal displays.
Shape Scanning Displays
The principle of shape scanning displays is close to the varifocal displays. Shape scanning displays employ focus-tunable lens to generate multiple tomographic layers. The core idea of shape scanning displays is to implement FSAB, which could be operated at much faster frame rate compared to ordinary display panels. Combining FSAB with display panels, we can update display images at 120 times faster frame rate so that 120 tomographic layers are reconstructed. Thus, shape scanning resolve the trade-off problem between the number of layers and frame rate in varifocal displays.
Shape scanning displays consist of a tunable lens, a display panel (e.g. liquid crystal panel), and a FSAB. We employ a tunable lens (focal length 50mm-130mm) that is separated by 50 mm from the display panel. The tunable lens is operated by triangle wave at 60Hz so that it sweeps the focal range within single frame. Within the single frame, the backlight is spatially modulated about 8-120 times while the display panel shows static 2D images (no need to be faster than 60Hz). Figure 1 demonstrates intuitive schematic diagram of shape scanning displays.
The FSAB independently illuminates pixels of the display panel so that each pixel is floated on the desired depth. If a pixel should be floated on the infinity, the FSAB illuminates the pixel only when focal length of the tunable lens is set to 50mm. We may implement FSAB using a LED array backlight (1920Hz) or a digital micromirror device (17kHz). The LED array backlight has advantages in the form factor and cost while the digital micromirror device may support high resolution binary images at the faster frame rate. Figure 2 shows the implemented prototypes of shape scanning displays.
In order to synchronize focus-tunable lens and FSAB, we used Data Acquisition (DAQ) board and LabView. Two reference clock signal are generated by DAQ board: 60Hz for focus-tunable lens and 6000Hz for FSAB. By synchronizing these two clock signals, we could implement shape scanning displays. Note that this specification could be varying according to the response time of FSAB. When a LED array backlight is used as FSAB, 480Hz signal is generated because of the latency caused by the LED array backlight. Figure 3 illustrates the reference clock signals for synchronization of focus-tunable lens and FSAB.
Figure 4A is optical simulation results using Zemax, which demonstrates display performance of shape scanning displays. As shown in results, shape scanning displays may support wide depth of field between 10.5D and 0.0D. Note that Commercial focus-tunable lens from Optotune (EL10-30-TC-VIS-12D) is used. According to the results, eye-box and diagonal field of view are estimated as 7.5mm and 30°. The size of display panel is 18mm×18mm. Note that the eye-box and field of view could be enlarged by using alternative focus-tunable lens that has larger aperture.
Figure 4B is experimental results that demonstrate accommodation cues provided by the prototype. In the prototype, the depth of field is between 5.5D and 0.0D, which is narrower than optical design due to the camera specifications. The binary sequence of tomographic layers is also described in the figure. This binary sequence is displayed by FSAB and synchronized with the focus-tunable lens as shown in Fig. 3. In this experiment, we used a 3D content introduced by Butler et al.24 As shown in the figure, a 2D projected image and corresponding depth map are used for generating tomographic layers. The resolution of the content is set to 450×450.
In this study, we propose a new type of 3D displays that we call shape scanning displays. Our system support wide field of view between 0.0D and 10.5D, quasi-continuous focus cues (120 tomographic layers), preserved resolution (450×450) of display panels, full frame (60Hz), and moderate diagonal field of view (30°) within eye-box of 7.5mm. Proposed system shows superior display performance among feasible approaches for 3D displays including light field displays, holographic displays, and varifocal displays. Shape scanning displays has a large potential to be applied for various field such as vision correction, high dynamic displays, and augmented reality. We believe shape scanning displays could open a new opportunity for industry in 3D displays.
This work was supported by Institute for Information & Communications Technology Promotion(IITP) grant funded by the Korea government(MSIT) (No. 2017-0-00787, Development of vision assistant HMD and contents for the legally blind and low visions).
Maimone, A. and Fuchs, H., “Computational augmented reality eyeglasses,” in [Proc. ISMAR], 29–38 (2013).Google Scholar
Park, J.-H., “Recent progress in computer-generated holography for three-dimensional scenes,” Journal of Information Display 18(1), 1–12 (2017).Google Scholar
Lee, S., Cho, J., Lee, B., Jo, Y., Jang, C., Kim, D., and Lee, B., “Foveated retinal optimization for see-through near-eye multi-layer displays,” IEEE Access (2017).Google Scholar
Butler, D. J., Wulff, J., Stanley, G. B., and Black, M. J., “A naturalistic open source movie for optical flow evaluation,” in [European Conf. on Computer Vision (ECCV)], A. Fitzgibbon et al. (Eds.), ed., Part IV, LNCS 7577, 611–625, Springer-Verlag (Oct. 2012).Google Scholar