21 May 2018 Shape scanning displays: tomographic decomposition of 3D scenes
Author Affiliations +
Abstract
Although there have been a desire to implement ideal three-dimensional (3D) displays, it is still challenging to satisfy commercial demands in resolution, depth of field, form factor, eye-box, field of view, and frame rate. Here, we propose shape scanning displays that may have extremely large depth of field (10cm-infinity) without loss of frame rate or resolution, and enough eye-box (7.5mm) with moderate field of view (30°). Furthermore, our prototype provides quasi-continuous focus cues as well as motion parallax by reconstruction of 120 tomographic layers. Shape scanning displays consist of a tunable lens, a display panel, and a spatially adjustable backlight. The synchronization of the tunable lens and spatially adjustable backlight could provide additional dimension of depth information. In summary, we introduce a novel 3D display technology called shape scanning displays that present superior performance in resolution, depth of field, and focus cue reproduction. This approach has a lot of potential to be applied for various field in 3D displays including head-up displays, tabletop displays, as well as head-mounted displays. It could be efficient solution for vergence-accommodation conflict as providing accurate focus cues.

1.

INTRODUCTION

During the history of display fields, there have been a desire to implement ideal three-dimensional (3D) displays. The ideal 3D displays may provide full of physiological cues including binocular disparity, motion parallax, and focus cues. Several technologies have been introduced and studied for the ideal 3D displays: light field near-eye stereoscopes,13 super multi-view displays,4 holographic displays,57 and varifocal displays.810 Although these approaches could support most physiological cues, it is still challenging to satisfy commercial demands in resolution, depth of field, form factor, eye-box, field of view, and frame rate.

According to the previous works related to 3D displays, we have observed trade-off problem among several terms that determine display performance. Light field displays2, 1113 usually suffer from trade-off between the spatial and angular resolution, which limits depth of field. In holographic display,57 it is hard to achieve large field of view and enough eye-box, simultaneously. Varifocal displays810 also have trade-off problem among frame rate, spatial resolution, depth of field, and accuracy of focus cues.

Here, we propose shape scanning displays that practically achieve great display performance in those terms by combining tunable lens and fast spatially adjustable backlight (FSAB). Shape scanning displays may have extremely large depth of field (10cminfinity) without loss of frame rate or resolution, and enough eye-box (7.5mm) with moderate field of view (30°). According to the optical design of the FSAB, shape scanning display may support up to 120 planes within the addressable depth of field.

In summary, we introduce a novel 3D display technology called shape scanning displays that present superior performance in resolution, depth of field, and focus cue reproduction. It could be efficient solution for vergence-accommodation conflict as providing accurate focus cues. In addition, shape scanning displays have a lot of potential to be applied for various field in 3D displays including head-up displays, tabletop displays, as well as head-mounted displays. Due to the versatility of shape scanning displays, it is feasible to apply this methodology for advanced applications such as vision correction14 and high dynamic range displays.15, 16

2.

RELATED WORK

2.1

Light Field Displays

Light field displays have a long history from glasses-free 3D displays to near-eye tensor displays. Light field displays aim to express volumetric shape of a real object by reconstruction of light field from the object. There have been several approaches to reconstruct the light field: parallax barrier, integral imaging,17 multi-view,18 and tensor displays.11, 12 Although these approaches have different optical structure and working principle, they share a similar trade-off problem between spatial and angular resolution.

In order to reconstruct light field considered as 4-dimensional variable, light field displays should divide display pixels according to the corresponding angular direction. Thus, spatial resolution is sacrificed for angular resolution, which becomes a barrier for commercialization of light field displays. Tensor displays could alleviate this trade-off problem by compression of light field that show clear tendency according to the viewing direction. However, tensor displays still suffer from the trade-off problem because actual amount of light field information is not increased.

2.2

Holographic Displays

Holographic displays reconstruct the wavefront from a volumetric object rather than light field. Modulating intensity or phase profile of incident coherent light source, holographic displays may generate a desired wavefront. Recent researches applied holographic displays for near-eye displays6, 19 and head-up displays.5, 20 Compared to light field displays, holographic displays have advantages in depth of field and occlusion effect. However, holographic displays usually have limited viewing window or eye-box.

As viewing window of holographic displays is determined by the size of spatial light modulator, the most feasible application of holographic displays is near-eye displays where viewing window is referred to as field of view. Field of view of holographic displays could be enlarged by using a floating lens. In this case, however, eye-box is sacrificed for enlargement of field of view. Therefore, holographic displays suffer from another trade-off problem between field of view and eye-box. Note that accommodation cue could be obscure if eye-box is too small to provide clear focus blur effect.

2.3

Varifocal Displays

Instead of light field or wave reconstruction, varifocal displays aim to form physical layers for depth expression. In varifocal displays, volumetric objects are reconstructed via synthesis of layer images on different depths. When each layer image is optimally rendered, varifocal displays are known to be able to provide continuous accommodation cues. In order to implement and synthesize multi-layer images, we may stack multiple display layers by using a beam combiner,10, 21 or employ focus-tunable lens and temporal multiplexing.22, 23

In varifocal displays, the number of layers is the most important term to determine display performance. However, it is challenging to enlarge the number of layers without sacrifice in form factor or frame rate. If using multiple beam combiners, it is hard to design compact optical system. When employing focus-tunable lens and temporal multiplexing, the number of layers involves the decrease in the frame rate because state-of-the-art display panel usually support up to 240Hz. Thus, 4-6 layers are considered as maximized specifications for varifocal displays.

2.4

Shape Scanning Displays

The principle of shape scanning displays is close to the varifocal displays. Shape scanning displays employ focus-tunable lens to generate multiple tomographic layers. The core idea of shape scanning displays is to implement FSAB, which could be operated at much faster frame rate compared to ordinary display panels. Combining FSAB with display panels, we can update display images at 120 times faster frame rate so that 120 tomographic layers are reconstructed. Thus, shape scanning resolve the trade-off problem between the number of layers and frame rate in varifocal displays.

3.

IMPLEMENTATION

Shape scanning displays consist of a tunable lens, a display panel (e.g. liquid crystal panel), and a FSAB. We employ a tunable lens (focal length 50mm-130mm) that is separated by 50 mm from the display panel. The tunable lens is operated by triangle wave at 60Hz so that it sweeps the focal range within single frame. Within the single frame, the backlight is spatially modulated about 8-120 times while the display panel shows static 2D images (no need to be faster than 60Hz). Figure 1 demonstrates intuitive schematic diagram of shape scanning displays.

Figure 1.

Schematic diagram to describe principle of shape scanning displays. Focus-tunable lens is synchronized with the DMD projection module. When the focus-tunable lens forms an image of LCD panel at depth of z1, partial image (A) on the LCD panel is illuminated by the DMD so A is floated at z1. In the same way, the other image (B) could be floated at depth of z2. Thus, shape scanning displays could reconstruct arbitrary volumetric objects.

00089_PSISDG10676_1067617_page_3_1.jpg

The FSAB independently illuminates pixels of the display panel so that each pixel is floated on the desired depth. If a pixel should be floated on the infinity, the FSAB illuminates the pixel only when focal length of the tunable lens is set to 50mm. We may implement FSAB using a LED array backlight (1920Hz) or a digital micromirror device (17kHz). The LED array backlight has advantages in the form factor and cost while the digital micromirror device may support high resolution binary images at the faster frame rate. Figure 2 shows the implemented prototypes of shape scanning displays.

Figure 2.

Photographs of shape scanning displays. On the left hand side, DMD-based prototype is presented. On the right hand side, LED array-based prototype is introduced. As shown in the photographs, LED array-based prototype has small form factor.

00089_PSISDG10676_1067617_page_3_2.jpg

In order to synchronize focus-tunable lens and FSAB, we used Data Acquisition (DAQ) board and LabView. Two reference clock signal are generated by DAQ board: 60Hz for focus-tunable lens and 6000Hz for FSAB. By synchronizing these two clock signals, we could implement shape scanning displays. Note that this specification could be varying according to the response time of FSAB. When a LED array backlight is used as FSAB, 480Hz signal is generated because of the latency caused by the LED array backlight. Figure 3 illustrates the reference clock signals for synchronization of focus-tunable lens and FSAB.

Figure 3.

Synchronization of focus-tunable lens and FSAB. 120 tomographic layers are displayed by FSAB as shown in the figure.

00089_PSISDG10676_1067617_page_4_1.jpg

4.

RESULTS

Figure 4A is optical simulation results using Zemax, which demonstrates display performance of shape scanning displays. As shown in results, shape scanning displays may support wide depth of field between 10.5D and 0.0D. Note that Commercial focus-tunable lens from Optotune (EL10-30-TC-VIS-12D) is used. According to the results, eye-box and diagonal field of view are estimated as 7.5mm and 30°. The size of display panel is 18mm×18mm. Note that the eye-box and field of view could be enlarged by using alternative focus-tunable lens that has larger aperture.

Figure 4.

A) Zemax simulation results to demonstrate optical performance of shape scanning displays. B) Experimental results to show the accommodation cues provided by an implemented prototype.

00089_PSISDG10676_1067617_page_5_1.jpg

Figure 4B is experimental results that demonstrate accommodation cues provided by the prototype. In the prototype, the depth of field is between 5.5D and 0.0D, which is narrower than optical design due to the camera specifications. The binary sequence of tomographic layers is also described in the figure. This binary sequence is displayed by FSAB and synchronized with the focus-tunable lens as shown in Fig. 3. In this experiment, we used a 3D content introduced by Butler et al.24 As shown in the figure, a 2D projected image and corresponding depth map are used for generating tomographic layers. The resolution of the content is set to 450×450.

5.

CONCLUSION

In this study, we propose a new type of 3D displays that we call shape scanning displays. Our system support wide field of view between 0.0D and 10.5D, quasi-continuous focus cues (120 tomographic layers), preserved resolution (450×450) of display panels, full frame (60Hz), and moderate diagonal field of view (30°) within eye-box of 7.5mm. Proposed system shows superior display performance among feasible approaches for 3D displays including light field displays, holographic displays, and varifocal displays. Shape scanning displays has a large potential to be applied for various field such as vision correction, high dynamic displays, and augmented reality. We believe shape scanning displays could open a new opportunity for industry in 3D displays.

ACKNOWLEDGMENTS

This work was supported by Institute for Information & Communications Technology Promotion(IITP) grant funded by the Korea government(MSIT) (No. 2017-0-00787, Development of vision assistant HMD and contents for the legally blind and low visions).

REFERENCES

[1] 

Maimone, A. and Fuchs, H., “Computational augmented reality eyeglasses,” in [Proc. ISMAR], 29–38 (2013).Google Scholar

[2] 

Huang, F.-C., Chen, K., and Wetzstein, G., “The light field stereoscope: immersive computer graphics via factored near-eye light field displays with focus cues,” ACM Trans. Graph. 34(4), 60 (2015). 10.1145/2809654Google Scholar

[3] 

Moon, S., Lee, C.-K., Lee, D., Jang, C., and Lee, B., “Layered display with accommodation cue using scattering polarizers,” IEEE Journal of Selected Topics in Signal Processing 11(7), 1223–1231 (2017). 10.1109/JSTSP.2017.2738614Google Scholar

[4] 

Moon, S., Park, S.-G., Lee, C.-K., Cho, J., Lee, S., and Lee, B., “Computational multi-projection display,” Opt. Express 24(8), 9025–9037 (2016). 10.1364/OE.24.009025Google Scholar

[5] 

Li, G., Lee, D., Jeong, Y., Cho, J., and Lee, B., “Holographic display for see-through augmented reality using mirror-lens holographic optical element,” Opt. Lett. 41(11), 2486–2489 (2016). 10.1364/OL.41.002486Google Scholar

[6] 

Maimone, A., Georgiou, A., and Kollin, J. S., “Holographic near-eye displays for virtual and augmented reality,” ACM Trans. Graph. 36(4), 85 (2017). 10.1145/3072959Google Scholar

[7] 

Park, J.-H., “Recent progress in computer-generated holography for three-dimensional scenes,” Journal of Information Display 18(1), 1–12 (2017).Google Scholar

[8] 

Narain, R., Albert, R. A., Bulbul, A., Ward, G. J., Banks, M. S., and O’Brien, J. F., “Optimal presentation of imagery with focus cues on multi-plane displays,” ACM Trans. Graph. 34(4), 59 (2015). 10.1145/2809654Google Scholar

[9] 

Lee, C.-K., Moon, S., Lee, S., Yoo, D., Hong, J.-Y., and Lee, B., “Compact three-dimensional head-mounted display system with savart plate,” Opt. Express 24(17), 19531–19544 (2016a). 10.1364/OE.24.019531Google Scholar

[10] 

Lee, S., Cho, J., Lee, B., Jo, Y., Jang, C., Kim, D., and Lee, B., “Foveated retinal optimization for see-through near-eye multi-layer displays,” IEEE Access (2017).Google Scholar

[11] 

Lanman, D., Hirsch, M., Kim, Y., and Raskar, R., “Content-adaptive parallax barriers: optimizing dual-layer 3d displays using low-rank light field factorization,” ACM Trans. Graph. 29(6), 163 (2010). 10.1145/1882261Google Scholar

[12] 

Wetzstein, G., Lanman, D., Hirsch, M., and Raskar, R., “Tensor displays: Compressive light field synthesis using multilayer displays with directional backlighting,” ACM Trans. Graph. 31, 80:1–80:11 (July 2012). 10.1145/2185520Google Scholar

[13] 

Lee, S., Jang, C., Moon, S., Cho, J., and Lee, B., “Additive light field displays: realization of augmented reality with holographic optical elements,” ACM Trans. Graph. 35(4), 60 (2016b). 10.1145/2897824Google Scholar

[14] 

Huang, F.-C., Wetzstein, G., Barsky, B. A., and Raskar, R., “Eyeglasses-free display: towards correcting visual aberrations with computational light field displays,” ACM Transactions on Graphics (TOG) 33(4), 59 (2014). 10.1145/2601097Google Scholar

[15] 

Gotoda, H., “A multilayer liquid crystal display for autostereoscopic 3d viewing,” in [Stereoscopic Displays and Applications XXI], 7524, 75240P, International Society for Optics and Photonics (2010). 10.1117/12.840286Google Scholar

[16] 

Wetzstein, G., Lanman, D., Heidrich, W., and Raskar, R., “Layered 3d: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 95 (2011). 10.1145/2010324Google Scholar

[17] 

Park, J.-H., Hong, K., and Lee, B., “Recent progress in three-dimensional information processing based on integral imaging,” Applied Optics 48(34), H77–H94 (2009). 10.1364/AO.48.000H77Google Scholar

[18] 

Lee, C.-K., Park, S.-G., Moon, S., and Lee, B., “Viewing zone duplication of multi-projection 3d display system using uniaxial crystal,” Optics express 24(8), 8458–8470 (2016). 10.1364/OE.24.008458Google Scholar

[19] 

Yeom, H.-J., Kim, H.-J., Kim, S.-B., Zhang, H., Li, B., Ji, Y.-M., Kim, S.-H., and Park, J.-H., “3d holographic head mounted display using holographic optical elements with astigmatism aberration compensation,” Opt. Express 23(25), 32025–32034 (2015). 10.1364/OE.23.032025Google Scholar

[20] 

Wakunami, K., Hsieh, P.-Y., Oi, R., Senoh, T., Sasaki, H., Ichihashi, Y., Okui, M., Huang, Y.-P., and Yamamoto, K., “Projection-type see-through holographic three-dimensional display,” Nature communications 7, 12954 (2016). 10.1038/ncomms12954Google Scholar

[21] 

Akeley, K., Watt, S. J., Girshick, A. R., and Banks, M. S., “A stereo display prototype with multiple focal distances,” ACM Trans. Graph. 23(3), 804–813 (2004). 10.1145/1015706Google Scholar

[22] 

Love, G. D., Hoffman, D. M., Hands, P. J., Gao, J., Kirby, A. K., and Banks, M. S., “High-speed switchable lens enables the development of a volumetric stereoscopic display,” Opt. Express 17(18), 15716–15725 (2009). 10.1364/OE.17.015716Google Scholar

[23] 

Liu, S. and Hua, H., “A systematic method for designing depth-fused multi-focal plane three-dimensional displays,” Opt. Express 18(11), 11562–11573 (2010). 10.1364/OE.18.011562Google Scholar

[24] 

Butler, D. J., Wulff, J., Stanley, G. B., and Black, M. J., “A naturalistic open source movie for optical flow evaluation,” in [European Conf. on Computer Vision (ECCV)], A. Fitzgibbon et al. (Eds.), ed., Part IV, LNCS 7577, 611–625, Springer-Verlag (Oct. 2012).Google Scholar

© (2018) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Seungjae Lee, Seungjae Lee, Youngjin Jo, Youngjin Jo, Dongheon Yoo, Dongheon Yoo, Jaebum Cho, Jaebum Cho, Dukho Lee, Dukho Lee, Byoungho Lee, Byoungho Lee, } "Shape scanning displays: tomographic decomposition of 3D scenes", Proc. SPIE 10676, Digital Optics for Immersive Displays, 1067617 (21 May 2018); doi: 10.1117/12.2315655; https://doi.org/10.1117/12.2315655
PROCEEDINGS
6 PAGES


SHARE
RELATED CONTENT


Back to Top