2.1.

Fourier Transform Optical Systems

In electroholography, the pixel density of the display device for a hologram determines the visual field and viewing zone, which indicate the maximum displayable size and the maximum area that viewers can see, respectively. Figure 1(a) outlines an ordinary arrangement of electroholography, where a hologram is illuminated by parallel light. The light is diffracted by the hologram, and the visual field is displayed across the hologram plane. In Fig. 1(a), the visual field angle $\theta$ is given by

Fig. 1

Outlines the optical reconstruction arrangements (a) conventional and (b) FTOS.

To improve this, we adopted a reconstruction method based on an FTOS developed device. The FTOS consists of an SLM, a lens, and a point light source. This simple structure has a great advantage for developing small holographic displays. When the SLM is reflective, the structure is represented as in Fig. 1(b). The point light source is arranged at the focal point of the lens, so the emitted light from the point light source is reflected by the SLM and converges at the focal point of the lens. The fringe pattern on the hologram used in the FTOS is different from that on an ordinary hologram, which is described in Sec. 3. The light diffracted by the hologram reconstructs images around the focal point of the lens. All the light reconstructing images pass through a specified area in front of the SLM. The area is called a viewing window. The width of the viewing window $w$ is given as

The visual field ${\theta }_F$ is enlarged as follows:

2.2.

Colorization Method

There are two colorization methods in electroholography. One is a method in which images of primary colors are spatially overlapped to reconstruct full-color images.^3,9,10 This method has the advantage of being able to use SLMs with a low refresh rate. However, the size of the optical system tends to be large because some optical components, such as lenses, the SLM, and point light sources for each of the primary colors, are necessary. Another colorization method is the field sequential color method,^11,12 in which images of the primary colors are successively reconstructed in the same position. Although SLM with a high refresh rate is needed to reconstruct full-color images without flickering, this method has the advantage of having a small reconstruction unit because it uses only one SLM to display holograms for all the primary colors. For the HMD we developed, the field sequential color method was implemented for the colorizing of the reconstructed images.

With the field sequential color method, the calculated holograms of each primary color need to be successively displayed only while the reconstruction light for this color is on. These successively reconstructed images of the primary colors are integrated into a full-color image by human vision. Sequential images with a frequency of more than 60 Hz allow observation of full-color images without flickering. A 180-Hz SLM was used for our experiments to display the holograms to keep the frequency for displaying the sets of the three colors at 60 Hz.

2.3.

Resolution of Reconstructed Image

The resolution of reconstructed images displayed by a hologram needs to be higher than that of the human visual system because insufficient resolution creates image blur in reconstructed images. In the FTOS, the size of the viewing window and the aperture size of the light source for reconstruction light determine the resolution.

When the size of the viewing window $w$ determined by Eq. (2) is less than that of the pupil, the hologram cannot provide enough light to the whole aperture area of the pupil. Thus, the resolution is less than that of human vision. Under the condition $w<\text{pupil}$ aperture, the resolution of reconstructed images is

According to the theory, a reconstruction light is assumed to be an ideal point light, but an actual reconstruction light has the definite aperture size $d_A$. The aperture size of the reconstruction light makes the resolution as follows:

The lower of the two resolutions determines the resolution of the system.

Our system uses RGB LEDs as reconstruction light, whose spectral bandwidths are broad, so the resolution of a reconstructed image is affected. As the diffraction angle on the hologram plane is approximately inverse proportion to the wavelength, the depth of the reconstructed image will be changed as follows:

3.1.

Point Source Method

There are a number of methods to calculate CGHs, and the point source method is used most often.¹³ This method considers objects as clouds of independent point light sources, and it allows the expression of arbitrarily shaped objects.

As suggested in Fig. 2, with the coordinate of an $i$’th point source of an object dataset defined as $p_i(x_i,y_i,z_i)$ and the coordinates of the hologram plane defined as $p_h(x_h,y_h,0)$, the complex amplitude distribution on the hologram $u_i$ is expressed as

Fig. 2

Outline of concepts involved in the point source method.

3.2.

Calculation of the Propagation Distance

In the FTOS, the propagation distance from the point source to the hologram plane is not like that in usual CGH calculations. A depth-free calculation method¹⁴ has been introduced to reconstruct images at arbitrary depths. Through the FTOS, a hologram is slightly enlarged by a lens. As the reconstructed images would be expanded and deformed with the FTOS, the CGH calculation method for FTOS is different from an ordinary hologram. It is necessary to compensate for the change in coordinates for the FTOS. When propagation distances from $p_i$ to $p_h$ are $r_i^{\prime }$ and the coordinates of the objects are $(x_i,y_i,z_i)$, the coordinates of the optimized location of the object $(x_i^{\prime },y_i^{\prime },z_i^{\prime })$ are obtained with

3.3.

Calculation for a Binocular System

The points to be reconstructed in a binocular view must have parallax information and must display different interference patterns on each of the two SLMs in the respective display units. To ensure the consistency of these two images reconstructed by each of the display units, two coordinate systems for each eye and the world coordinate system that indicates the position of the virtual point to be reconstructed are shown as Fig. 3. The origin of the world coordinate system $O_W$ is located at the center of the view points of the left and right eyes, and the $z$-axis is set inverse to the viewing direction with the $x$-axis orthogonally crossing the viewing direction and passing through the left and right view points. The coordinates of the virtual objects represented by the world coordinate system need to be transformed into the two coordinate systems to calculate consistent CGHs for the left and right display units. When the origin of the coordinate system of the left eye in the world coordinate system is defined as $O_L(x_{W{O}_L},y_{W{O}_L},z_{W{O}_L})$, the coordinates of the point to be reconstructed in the world coordinate system $(x_W,y_W,z_W)$ are transformed into the left coordinates system $(x_L,y_L,z_L)$ using the movement and rotation transformation, expressed as

Fig. 3

Outline of the binocular system.

In a binocular system, the binocular visual field angle ${\theta }_B$ is expressed as

3.4.

Calibration for Installation Errors

Optical units commonly have some installation errors, and these errors often cause considerable errors in reconstructed images. However, these errors are difficult to remove manually. Here we propose a calibration method to correct installation errors. The calibration method is divided into three steps.

The first step corrects the depth of the reconstructed images. In this step, depth-direction errors are corrected using a method with the linear least squares. First, $N$ measurement depths are defined as $z_1,z_2,\dots ,z_N$, and the measured depth $z_{ei}$ which is obtained by measuring the object located at $i$’th depth $z_i$ is determined. We assume that the relation between the $z_i$ and $z_{ei}$ is represented as a linear model, and the fitting line $F(z_i)$ is defined as

In the second step, the image size is corrected. The size of the image is changed after the first step, so the size of the image has to be changed to the original object size. The size of the depth-corrected object $H_i^{\prime }$ is represented by the original object size $H_i$ as follows:

In the third step, the vergence value is also corrected using linear least squares. The vergence value can be determined by the $x$-coordinates of the images reconstructed by the left and right display units. As shown in Fig. 4, when the $x$-axis-direction error in the world coordinate system between the objects reconstructed by the left and right display units is measured to be $d_{ei}$ at the depth $z_i$, the linear function $G(z)$ followed by $d_{ei}$ is expressed with the constants $b_1$ and $b_2$ as

Fig. 4

Measurement of errors of target parallax and conversion to depths.

In summary, the object data $(x_i,y_i,z_i)$ are corrected to $(x_i^{\prime },y_i^{\prime },z_i^{\prime })$, which are represented as

This is a practical calibration method because it adjusts the installation errors of lenses, reconstruction lights, and other optical elements, and the depth-direction distortions caused by the lenses are also corrected at once.

4.1.

Optical Structure

We fabricated a holographic HMD based on the aforementioned details with a sufficiently solid structure to facilitate adjustments of the lens, light source unit, and SLM as shown in Fig. 5. The optical parts are integrated into one component with a surrounding frame. The reconstruction light should be arranged to be located just at the focal point of the lens to reduce installation errors.

Fig. 5

Photo and outline of holographic HMD.

To create binocular vision, the proposed our HMD has two display units, one for the left eye and one for the right, as shown in Fig. 5. These two units are independent and combined symmetrically for the left and right eyes, and both units are attached on a helmet. Because each display unit has a narrow viewing zone, the binocular display system is equipped with a sliding structure under the right display unit to enable it to be adjusted to the individual PDs of observers.

For each of the display units, the FTOS is comprised of an SLM, a lens, and a light source unit, all of which are arranged on one axis. To superpose the real object and virtual image in the observer’s vision, a half mirror is located between the lens and the light source unit. See-through type displays have to make the reconstructed images bright enough to be observed with ordinary room light. To avoid the attenuation of the reconstruction light, the propagation distance is shortened by a lens with short focal length, and only a one-half mirror is used to make see-through vision possible.

To cut out unexpected light, barriers are located in front of the eyes. The unexpected light is blocked by the barriers, and only the reconstruction light is propagated through the barriers.

4.2.

Full-Color Point Light Source Unit

To obtain sharp and clear full-color images, very small and high-power point light sources are necessary because the resolution of the reconstructed images depends on the quality of the reconstruction light. Lasers for each of the primary colors combined by half-mirrors have usually been adopted as one high-power and full-color point light source.^3,9,10,15 However, multiple lasers result in a bulky apparatus and speckle noise in the reconstructed images due to the high coherence of laser light. For these reasons, we used small and high-power light-emitting diodes (LEDs) for the reconstruction light.

The full-color LED used has independent LED tips for the primary colors arranged at slightly different positions. Due to the minor errors it gives rise to, the reconstructed images also have slight errors, which would normally cause some color blurring. To avoid that, a sharpening transparency acryl fiber is put on the LEDs as shown in Fig. 6. The custom-made light source was designed and developed by us. The fiber is about 18-mm long and has a diameter of 5 mm. Except for the top and bottom of this fiber, the surface is mirror-coated. Light emitted from the LEDs is combined inside the fiber, and the colors of each beam of the light are emitted only at the top of the mirror-coated fiber.

Fig. 6

Photo and details of full-color point light source unit with mirror-coated fiber.

Figure 7 shows the top of the fiber. The aperture of the fiber, with the light spot area shown in the figure, is about 0.2 mm. The size is determined by the balance of the brightness, resolution, and directivity. This point light source unit has a little directivity and a half-power angle of 10.8 deg but is still sufficiently large enough to cover the entire active area of the SLM.

Fig. 7

Aperture of LED fiber.

To synchronize the timing of lighting of a primary color LED and displaying the color of the hologram, a synchronizing circuit connects the SLM and the light source unit. This circuit transmits the color information signal from the SLM driver to the light source unit, and each color the LED is lit corresponding to the received color signal.

5.1.

Optical Characteristics

Figure 8 shows the left and right images of this holographic HMD. This reconstructed image is generated from the virtual objects shown in Fig. 8(a), with a metal ball at the center, a transparent object on the right, and a diffuse reflection sphere on the left. It can be seen that the checkerboard below is reflected on the surface of the reflective sphere. Figures 8(b) and 8(c) have parallax, and they can be viewed stereoscopically by the parallel method. In this manner, the texture of the object’s surface can be displayed. Also, when the observer is focusing on the sphere in the front, the rear sphere became blurred and a natural expression regarding the focal length was observed.

Fig. 8

Parallax of the holographic HMD (a) schematic diagram and (b) view from left and (c) view from right.

Figure 9 shows the reconstructed image of a visual field chart with a dot interval of 1.0 deg, except for 0.5 deg at both ends of the horizontal row. This result indicates that visual fields of the holographic HMD are 9.4 deg horizontally and 5.6 deg vertically, respectively. As a lens with a small diameter and a short focal length is used, the chromatic aberration can be seen. To correct this, it is necessary to considerate calculation algorithm about the chromatic aberration correction.

Fig. 9

Reconstructed image of a visual field chart.

Figure 10 shows the reconstructed images of test charts to reveal the resolutions of the proposed holographic HMD. The charts are located at a depth of 500 mm, and the numbers on them are line spaced (mm). The figures indicate that both horizontal and vertical resolutions are slightly larger than 1.0 mm. The measured resolutions agree with that predicted by Eq. (5) and the aperture size of reconstruction light described in Sec. 2.3.

Fig. 10

Resolutions of reconstructed images (a) horizontal and (b) vertical resolutions.

Figure 11 shows reconstructed images of two Maltese-cross targets arranged at a depth of 300 and 1000 mm, respectively. In Fig. 11(a) in which the focus is on the left target, the left target image is sharp whereas the right one is defocused. Focusing on the opposite target produced an effect opposite to that as shown in Fig. 11(b). These results indicate that the reconstructed images provide accommodation stimulus.

Fig. 11

Difference in reconstructed images of focal depths.

5.2.

Calibration of the Depth of the Reconstructed Images

In the first experiment, the depth of the reconstructed noncorrected images were measured and corrected using the calibration method described in Sec. 3.4 and we tested the accuracy of the depths and vergence values of the images. The measured depths $z_i$ were defined from 400 to 1000 mm with 100-mm intervals, and the depths of the targets were measured using the focus feature of a camera. The focus of the camera was aligned with the reconstructed image, and in this state, the actual index was placed at the focus of the camera. The depth of this real index was measured. The camera was a Nikon D5100, the lens an $f/22$, and the depth of field was $\pm 90\mathrm{mm}$ at 1 m. The use of the RGB LEDs affects the image depths described in Sec. 2.3. The central wave length (spectral bandwidths) of the RGB LEDs are 625 nm (17 nm), 525 nm (34 nm), and 465 nm (23 nm), respectively. The largest depth change (green color) is 16 mm at the depth of 1000 mm, which is smaller than the depth of field of the camera. The resolution occurred by bandwidth of the wavelength is 0.24 mm (green color) at a depth of 1000 mm, which is smaller than the resolution occurred by the aperture size.

Figure 12 shows the results of this experiment: Fig. 12(a) is the results of the left optical unit and (b) shows the results of the right one. In this figure, the horizontal and the vertical axes correspond to the target depth $z_i$ and the measured depth $z_{ei}$, respectively. The measured depth of the noncorrected targets $z_{e}i$ is plotted as “+” marks and the approximated curve of these values $F(z)$ is expressed as the chained line. These data were corrected using Eqs. (20) and (25) and expressed as “$\times$” marks. The measured depths of the corrected images $z_{ei}^{\prime }$ are in agreement with the theoretically ideal line that satisfies $z_{ei}=z_i$ and are represented by the dotted line. These results after correction show that both the left and right units reconstruct the images at the correct depths.

Fig. 12

Results of depth correction (a) measured depths and corrected depths of left optical unit, (b) right unit.

Next, the errors of the vergence values of the display depth-corrected targets located at the theoretically ideal depth were measured. These errors were also measured using a camera with the following procedure: a scale was arranged in the depth displaying the reconstructed images, and we captured the reconstructed images and measure from the left and right viewpoints using camera. If the parallax is correct, the reconstructed image should be located at the same position on the scale. Figure 13 shows difference of positions between the left and right images.

Fig. 13

Results of the vergence-value correction (a) measured errors $d_{ei}$ related to the vergence and (b) related to PDs.

In Fig. 13(a), the measured errors $d_{ei}$ were sufficiently small and needed no correction; the maximum value was a 10-mm error at the 1000-mm deep target. Figure 13(b) shows these experimental results with the measured errors related to the PDs. No calibration of the vergence value was conducted for these experiments. The error became almost negligible.

5.3.

Subjective Evaluation

The second experiment was subjective evaluations of depth perceptions. In this experiment, the depths of the depth-corrected images were evaluated binocularly by five subjects in their 20s, all of whom had 20/20 vision. The target depths were located from 400- to 1000-mm distance from the observers at 100-mm intervals. The observed depth was measured by moving the actual index to a position, where a subject recognized as the same as the image. The movement is controlled by a subject using an electric laser. There was one experiment for each subject.

The results of the evaluation are shown in Fig. 14, where $D$ (diopter) is a metric expression of $1/\mathrm{m}$; a unit expressing the distortion power of the lens. Figure 14(a) shows the relationship between the stimuli depths $Z_i$ [D] of the displayed targets and the observed depths $Z_{ei}$ [D] after the depth correction. In the figure, the dotted line is an ideal line satisfying $Z_{ie}=Z_i$ and the plotted marks are observed depth. These results indicate that the proposed calibration is able to correct the depths of reconstructed images.

Fig. 14

Results of subjective evaluation (a) observed depths reported by five subjects and (b) relationship between observed depths and PDs.

Figure 14(b) shows the individual PDs in the horizontal axis and the stimuli of the depths of the observed targets in the vertical axis. These results indicate that the holographic HMD regenerates images of the objects at correct depths for the various individual PDs.