28 July 2014 Automatic geometrical calibration for multiprojector-type light field three-dimensional display
Author Affiliations +
Abstract
An automatic geometrical calibration approach has been developed to calibrate a multiprojector-type light field (LF) display automatically and accurately. The calibration framework based on image mapping is detailed, which transfers the calibration of three-dimensional (3-D) scene into the calibration of two-dimensional image in the diffuser interface. A multiprojectors-type LF display prototype is applied to implement the experimental calibration. Comparison results of the reconstructed 3-D scene before and after calibration show that a better overall performance is obtained through the proposed calibration approach.
Chen, Zhong, Li, Liu, and Xu: Automatic geometrical calibration for multiprojector-type light field three-dimensional display

1.

Introduction

Recent years have witnessed the promising development of three-dimensional (3-D) display techniques.1,2 Levoy and Hanrahan proposed a light field (LF) rendering technique for three-dimensional (3-D) scene reconstruction.3 The major idea is a representation of the LF, which aims to reconstruct the LF of a 3-D scene in a pixel-based rather than a view-based perspective through designed imaging process. Compared to other conventional methods, it generates a higher density of viewing zones such that it provides observers a more comfortable viewing experience.

Substantial efforts have been conducted to achieve a 3-D display using LF rendering. A Hungarian company named Holografika proposed the first commercial LF display system.4 Researchers from MIT proposed several display prototypes using multilayer attenuators.5,6 Zhejiang University proposed a liquid-crystal-display-based LF 3-D display system.7 Moreover, Samsung announced a large-scale system using 300 projectors in the 2013 Society for Information Display technical symposium.8 From this, it can be concluded that a large-scale system employed with multiprojections is becoming the trend of 3-D display and has commercial prospects.

In a multiprojector-type 3-D display, geometrical calibration is definitely crucial in adjusting the projection accuracy. Alternative methods have been proposed for two-dimensional (2-D) image calibration.910.11.12 Mainstream methods mentioned in the references are derived from a combination of a standard camera and several uncalibrated projectors, by which excellent calibration results have been achieved. However, our previous studies mainly focus on calibrating a projected image onto a 2-D plane, for example, making one or more projectors display a stitched and undistorted image on a surface of unknown geometry.1314.15 Meanwhile, there are few reports in the literature of flexible and efficient calibration dedicated for multiprojector-type 3-D displays so far. Different from the calibration of a 2-D image, the light rays in a 3-D scene are considered as vectors such that projected images are interpreted as a combination of vectors. Therefore, the calibration in multiprojector-type LF 3-D display requires that more factors be taken into consideration. Moreover, it shall be implemented more accurately and efficiently by taking the integral imaging process into consideration.

In this study, an automatic geometrical calibration framework for a multiprojector-type LF 3-D display is investigated. Different from the conventional methods, a precise rotary table is added into the calibration framework to enable the usage for a large panoramic viewing angle with a large-scale curved screen. Together with the robust calibration algorithm, which transfers the calibration of a 3-D scene into the calibration of a 2-D image on the diffuser interface, the display performance can be calibrated automatically and accurately. The experimental results verify the effectiveness of the proposed calibration framework. Given that the properties of rays have been taken into consideration, we believe that the calibration algorithm can be applied to most projector-type 3-D displays, not only based on LF reconstruction, but also on a multiview reconstruction.

2.

Principle of Multiprojector-type LF 3-D Display

By convention, rays can be parameterized in a couple of ways in computer graphics.3,16 Here, we configure it by a point (x, y, and z) and a direction (u,v) as Fig. 1 shows. We define the LF rendering of an object in the following form using homogeneous coordinates:

(1)

LF=P(x,y,z,u,v).

Fig. 1

The representation of rays in space.

OE_53_7_073107_f001.png

When trying to reconstruct the light filled rendering of an object by multiprojection or other techniques, it is important to guarantee the accuracy of the parameters.

Figure 2 shows a general configuration of a multiprojectors-type LF display, which consists of a series of projectors arranged in a circle and a directional diffuser. Images are projected upward to the opposite side of the screen with the same height. Because of the angle-interval between the projectors, the special diffuser provides a large diffuse angle in the vertical direction for image perspective, while providing a small angle in the horizontal direction for pupil jointing.17 The vertical diffuse angle of the diffuser depends on the interval angle between the adjacent projectors.

Fig. 2

Illustration of display principle. The two circles show the images seen from views V1 and V2.18

OE_53_7_073107_f002.png

Consequently, observers will see the jointed stripe images as Fig. 2 shows. At viewing points V1 and V2, the two spatial points A and B are represented by light rays emitted from different projectors and can be seen at different positions. By adding sufficient projectors with appropriate intervals, the stripe images will join together to make up an integral image.18

3.

Framework of Geometrical Calibration

The image generation algorithm for the display system is deduced under a theoretical situation. In reality, as the number of projectors increases, the installation and adjustment of all projectors become a huge task. It is rather difficult and inefficient to calibrate each projector manually and separately. If any projector is not adjusted well, it results in additional distortion and a defective 3-D reconstruction. To solve this problem, our research has been conducted on automatic geometrical calibration. Taking one projector as an example, the framework of geometrical calibration is presented below.

First, capture the printed standard spot diagram (s-image) and the projector spot diagram (p-image), and calculate the mapping relationship Matrix 1 (M1) between the p-image in the camera space and the original spot diagram (o-image) in the projector space

(2)

p-image=o-image×M1.

The calibrated spot diagram (c-image) in the projector space can be obtained by multiplying the s-image with Matrix 1

(3)

c-image=s-image×M11.

Then calculate the mapping relationship Matrix 2 (M2) between the o-image and c-image

(4)

M2=o-image1×c-image.

A transformation matrix M2 is then applied for any image generated from the LF rendering algorithm for the projector. This calibration approach can be naturally extended to a system containing many projectors. Repeat this process for every projector and all projections in the system will finally have a uniform standard.

Without loss of generality, the main module of autocalibration can be summarized as Fig. 3 shows.

Fig. 3

Flowcharts of autocalibration approach.

OE_53_7_073107_f003.png

Figure 4 presents the captured photos taken during the calibration proceeding mentioned above. The spot diagram in Figs. 4(a) and 4(b) is identified and marked with red circles during image processing. Also, Figs. 4(c) and 4(d) are the spot diagram in the projector space before and after the calibration.

Fig. 4

Pictures taken in calibration proceeding: (a) s-image, (b) p-image, (c) o-image, and (d) c-image.

OE_53_7_073107_f004.png

The validity of this method can be demonstrated through discussing several different scenarios. Without loss of generality, the deviation of the projector is divided into two main types, angle deviation and position deviation.

3.1.

Angle Deviation

We interpret the projected image as a combination of vectors. Due to the display principle illustrated in the above paragraphs, the image observed at any viewing position is derived from certain rays emitted from a series of projectors. On the other hand, the voxel in the constructed 3-D scene is also composed of rays from different projectors. If one projector is fixed in the right place, what is left is projecting a correct image onto the correct place in a fixed 2-D plane.

In Fig. 5, the solid lines represent the practical light rays projected from projectors, and the dashed lines represent the extension of rays connecting the viewing position to a virtual 3-D voxel. Note that herein blue lines stand for the incorrect rays before calibration, while red lines stand for the correct rays after calibration. As Fig. 5(a) shows, we assume a projector is placed at the correct place but toward an incorrect direction. In the diffuser interface, the pixel of the projection image which should be located at A is wrongly located at A. Also, the observer will see an incorrect voxel p from this projector, resulting in the observer seeing an incorrect 3-D scene. The major idea behind the calibration framework is to calculate the mapping relationship between A and A and assign the information of light ray OA back to OA. In this way, the incorrect voxel p′ will be replaced with the correct voxel p, and a correct projection image can be regenerated after calibration.

Fig. 5

(a)–(c) Illustration of angle deviation, vertical and horizontal position deviations.

OE_53_7_073107_f005.png

3.2.

Vertical and Horizontal Position Deviations

Figure 5(b) shows the case of position deviation in the vertical direction. This algorithm also works because of the large diffuse angle in the vertical direction. The observer will see the correct voxel p after calibration.

In case there is a horizontal position deviation for the projector as shown in Fig. 5(c), we can tolerate half of a horizontal diffuse angle deviation of the projector horizontal position when considering the horizontal diffusing property. Through calculating the mapping relationship between A and A and assigning the information of light ray OA back to OA, the observer still can see the correct voxel p and obtain the correct 3-D scene provided by this projector. In other words, in this proposed system, the maximum tolerable horizontal position deviation is given by

(5)

Δx=θ2×180deg×π×D,
where D stands for the distance between the projector and diffuser and θ stands for the horizontal diffuse angle of the diffuser. In our experiments mentioned below, Δx is 24.7 mm when θ equals to 1 deg. Without loss of generality, we can limit the horizontal position deviation within 20 mm in the manufacturing operation. Therefore, the horizontal position deviation has relatively less effect on the system.

In conclusion, the proposed calibration algorithm transfers the calibration of 3-D scenes into the calibration of a 2-D image in the diffuser interface, which makes it easier to solve.

4.

Experiments and Results

To verify the principle and calibration algorithm, a prototype is constructed as Fig. 6 shows, which consists of a series of circularly arranged projectors and a cylindrical directional diffuser. All projectors are arranged in a circle under the diffuser and staggered horizontally to condense the pupil array. The specifications are shown in Table 1.

Fig. 6

Schematic of the 360 deg display system.18

OE_53_7_073107_f006.png

Table 1

Specifications of the 360 deg display system.

ContentsSpecifications
Number of projectors (n)360
Interval angle of adjacent projectors (deg)1
Resolution of each projector (pixel)800×600
Radius of cylindrical diffuser (mm)1500
Radius of projectors array (mm)1700
Screen height (mm)1800
Horizontal diffuse angle of diffuser (deg)1
Vertical diffuse angle of diffuser (deg)60

As Fig. 7 shows, the calibration prototype system used in the experiment employs a precise rotary table set in the center. Meanwhile, a CCD camera is fixed onto the table and a printed standard spot diagram is pasted on the diffuser interface. To be clear, the printed standard spot diagram is generated from the LF reconstruction algorithm based on an ideal prototype.

Fig. 7

Structure of an automatic calibration system: 1. diffuser; 2. printed standard spot diagram; 3. projects array; and 4. CCD camera fixed on precise rotary table.

OE_53_7_073107_f007.png

Figures 8 and 9 show the comparison of the projected spot diagram before and after the calibration. Figures 8(b) and 8(d) are the partial zoomed views of Figs. 8(a) and 8(c). The autocalibration result of spot diagram is illustrated in Fig. 8(d), which is much better than the result for the manual adjustment showed in Fig. 8(c).

Fig. 8

Comparison of projected spot diagram before and after calibration: (a) projected spot diagram before calibration, (b) partial zoomed views of (a), (c) projected spot diagram after calibration, (d) partial zoomed views of (c).

OE_53_7_073107_f008.png

Fig. 9

Comparison of projected three-dimensional scene before and after calibration: (a–b) viewed from left position, (c–d) viewed from center position, (e–f) viewed from right position.

OE_53_7_073107_f009.png

To prove the effectiveness of the calibration framework, we take five spots in different areas marked with yellow circles in Fig. 8(c) to perform the calculation. The calibration accuracies of the spots are listed in Table 2. From the measurement result, the maximum mean deviation is 0.575 pixels.

Table 2

Measurement calibration accuracy analysis.

PositionMean deviation (pixel)Standard deviation (pixel)
A0.4880.393
B0.3750.406
C0.3710.329
D0.5750.338
E0.4120.349

Figure 9 shows that a better 3-D scene is achieved after calibration from the left, center, and right viewing positions, which proves the effectiveness of the calibration framework. Figures 9(a), 9(c), and 9(e) are the projected 3-D scene before calibration viewed from the left, center, and right view positions, respectively. Figures 9(b), 9(d), and 9(f) are the projected 3-D scene after calibration viewed from left, center, and right view positions, respectively. The performance enhancement is obvious.

5.

Conclusion

This paper presents an automatic geometrical calibration approach for a multiprojector-type LF 3-D display. The calibration framework is proposed and then detailed for calibrating the geometrical deviation and distortion of a projected 3-D scene. Experimental results demonstrate that the projectors in the display system can be adjusted automatically and accurately. The validity of this method has been discussed and we argue that this approach has the robustness to be extended to most existing multiprojector-type displays.

Therefore, with the aid of this automatic geometrical calibration, multiprojector-type LF 3-D displays will see great potential in commercial applications in the near future because of its better performance and easier calibration.

Acknowledgments

This work is supported by research grants from the National Basic Research Program of China (973 Program) (2013CB328802), National High Technology Research and Development Program of China (2012AA011902), and the National Natural Science Foundation of China (61177015). Great thanks to the State Key Laboratory of Modern Optical Instrumentation, Zhejiang University, for providing the necessary support. Also, we thank Yifan Peng from the Department of Computer Science, University of British Columbia, for providing fruitful support.

References

1. J. Honget al., “Three-dimensional display technologies of recent interest: principles, status, and issues [Invited],” Appl. Opt. 50(34), 87–115 (2011).APOPAI0003-6935 http://dx.doi.org/10.1364/AO.50.000H87 Google Scholar

2. N. S. Hollimanet al., “Three-dimensional displays: a review and applications analysis,” IEEE Trans. Broadcast 57(2), 362–370 (2011).IETBAC0018-9316 http://dx.doi.org/10.1109/TBC.2011.2130930 Google Scholar

3. M. LevoyP. Hanrahan, “Light field rendering,” in Proc. of the 23rd Annual Conf. on Computer Graphics and Interactive Techniques, ACM, New Orleans, LA (1996). Google Scholar

4. T. BaloghP. T. KovácsZ. Megyesi, “Holovizio 3D display system,” in Proc. of the First Int. Conf. on Immersive Telecommunications, Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, Kos Island, Greece (2007). Google Scholar

5. G. Wetzsteinet al., “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graphics 30(4), 017402 (2011).ATGRDF0730-0301 http://dx.doi.org/10.1145/2010324 Google Scholar

6. D. Lanmanet al., “Polarization fields: dynamic light field display using multi-layer LCDs,” ACM Trans. Graphics 30(6), 186 (2011).ATGRDF0730-0301 http://dx.doi.org/10.1145/2070781 Google Scholar

7. Y. Penget al., “Liquid-crystal-display-based touchable light field three-dimensional display using display-capture mapping calibration,” Appl. Opt. 51(25), 6014–6019 (2012).APOPAI0003-6935 http://dx.doi.org/10.1364/AO.51.006014 Google Scholar

8. J. Parket al., “48.2: light field rendering of multi-view contents for high density light field 3D display,” in SID Symposium Digest of Technical Papers, Vol. 44, No. 1, Blackwell Publishing Ltd., Vancouver, Canada (2013). Google Scholar

9. B. SajadiA. Majumder, “Auto-calibration of cylindrical multi-projector systems,” in Virtual Reality Conf. (VR), 2010 IEEE, IEEE, Waltham, MA (2010). Google Scholar

10. R. Yanget al., “Toward the light field display: autostereoscopic rendering via a cluster of projectors,” IEEE Trans. Visual. Comput. Graphics 14(1), 84–96 (2008).TVCG1077-2626 http://dx.doi.org/10.1109/TVCG.2007.70410 Google Scholar

11. Y. Chenet al., “Automatic alignment of high-resolution multi-projector display using an un-calibrated camera,” in Proc. of the Conf. on Visualization’00, pp. 125–130, IEEE Computer Society Press, Salt Lake City, Utah (2000). Google Scholar

12. A. RaijM. Pollefeys, “Auto-calibration of multi-projector display walls,” in Proc. of the 17th Int. Conf. on Pattern Recognition, 2004 (ICPR 2004), Vol. 1, IEEE (2004). Google Scholar

13. R. Raskaret al., “Multi-projector displays using camera-based registration,” in Proc. Visualization’99, pp. 161–522, IEEE, San Francisco, CA (1999). Google Scholar

14. H. Chenet al., “Scalable alignment of large-format multi-projector displays using camera homography trees,” in Proc. of the Conf. on Visualization’02, pp. 339–346, IEEE Computer Society, Boston, MA (2002). Google Scholar

15. J. Zhouet al., “Multi-projector display with continuous self-calibration,” in Proc. of the 5th ACM/IEEE Int. Workshop on Projector camera systems, ACM, Marina del Rey, CA (2008). Google Scholar

16. L. McMillanB. Gary, “Plenoptic modeling: an image-based rendering system,” in Proc. of the 22nd Annual Conf. on Computer Graphics and Interactive Techniques, ACM, New York (1995). Google Scholar

17. Y. Penget al., “Large-sized light field three-dimensional display using multi-projectors and directional diffuser,” Opt. Eng. 52(1), 017402 (2013).OPEGAR0091-3286 http://dx.doi.org/10.1117/1.OE.52.1.017402 Google Scholar

18. Q. Zhonget al., “Multiview and light-field reconstruction algorithms for 360° multiple-projector-type 3D display,” Appl. Opt. 52(19), 4419–4425 (2013).APOPAI0003-6935 http://dx.doi.org/10.1364/AO.52.004419 Google Scholar

Biography

Bei-shi Chen received his BS degree in information engineering from Zhejiang University, China, in 2012. Currently, he is studying for his MS degree in State Key Laboratory of Modern Optical Instrumentation, Zhejiang University. His research interests include three-dimensional imaging acquisition and display and display characteristic testing.

Qing Zhong received his BS degree in information engineering from Zhejiang University, China, in 2011. Currently, he is studying for his PhD degree at the State Key Laboratory of Modern Optical Instrumentation, Zhejiang University. His research interests include projection techniques, three-dimensional display, and display characteristic testing.

Hai-feng Li received his BA and MS degrees in physics from Nankai University in 1988 and 1991, respectively, and his PhD degree in optical engineering from Zhejiang University, China, in 2002. He is a professor in the Department of Optical Engineering, Zhejiang University, China. He has been a faculty member at Zhejiang University since 1991. His current research works include three-dimensional light field display, projection display, and liquid crystal devices.

Xu Liu obtained his BS and MS degrees in Zhejiang University, China, in 1984 and 1986, respectively, and his PhD in ENSPMUniversité d’Aix-Marseille III, France, in 1990. Now, he is a professor in the Department of Optical Engineering, Zhejiang University, the director of the State Key Laboratory of Modern Optical Instrumentation, and the executive dean of the Faculty of Information Science of Zhejiang University. He is the vice-chair of the Chinese Optics Society. His research fields are optical thin films and thin film technology, optical precision detection, projection display, and three-dimensional display.

Hai-song Xu is a professor in the Department of Optical Engineering, Zhejiang University, China. His current research works include color science, video technology, and lighting.

© The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.
Bei-shi Chen, Qing Zhong, Hai-feng Li, Xu Liu, Hai-song Xu, "Automatic geometrical calibration for multiprojector-type light field three-dimensional display," Optical Engineering 53(7), 073107 (28 July 2014). https://doi.org/10.1117/1.OE.53.7.073107 . Submission:
JOURNAL ARTICLE
6 PAGES


SHARE
Back to Top