Experimental verification of accommodation-convergence conflict in viewing integral photography

Abstract. We experimentally verified the depth perception and accommodation-convergence conflict in viewing integral photography. For comparison, the same measurements were performed with binocular stereoscopic images and real objects. First, the depth perception in viewing an integral three-dimensional (3D) target was measured at three display resolutions: 153, 229, and 458 ppi. The results showed that the depth perception was dependent on the display resolution. The results were also evaluated in a statistical test at a significance level of 5%. The results showed that the recognized depth perception ranges were 180, 240, and 330 mm when the display resolutions were 153, 229, and 458 ppi, respectively. The results were also analyzed in terms of image resolution. This suggested that depth perception occurred at over 1.0 cpd. The accommodation and convergence responses in viewing an integral 3D target displayed on a 3D display with 458 ppi were measured using PowerRef 3. The experimental results were evaluated with a multiple comparison test. It was found that 6 of the 10 observers did not have an accommodation-convergence conflict when viewing the integral 3D target inside and outside the depth of field. In conclusion, integral photography can provide a natural 3D image that looks like a real object.


Introduction
Binocular stereoscopic three-dimensional (S3D) methods are now popular in movies. In the S3D method, parallax images are input to the eyes of the viewer, so special glasses, lenticular lens, or other such mechanism are required for presenting different images to the left and right eyes. The parallax enables stereoscopic vision; thus, the convergence reaction of the viewer is consistent with the depth position of the S3D target. However, it has been pointed out that the accommodation position is near the displayed position, so the consequent inconsistency in accommodation and convergence produces visual fatigue. 1 Integral photography is a promising way to display threedimensional (3D) optical images by reproducing the same light rays as emitted from real objects. 2 This method duplicates the conditions of viewing real objects. Therefore, the accommodation and convergence responses have been predicted to be consistent with the depth position of the 3D target. The convergence responses to integral 3D (I3D) and S3D targets are expected to be the same as those to a real target. Here, the accommodation response to I3D displays has been theoretically analyzed by many researchers. [3][4][5] The reports indicate that satisfying the super multiview (SMV) condition is the most important requirement for obtaining a proper accommodation response. The SMV condition means that two or more light rays from the point lights of the reconstructed 3D object reach the pupil of an observer. The accommodation responses to an I3D target have been verified in a theoretical analysis, computer simulation, and experiment. 4 Although the experiment in Ref. 4 showed that different depth positions of the I3D target could be captured, the accommodation responses of the observers were not measured. In addition, the accommodation responses have been evaluated; 3,6 however, they were not compared with those of a real object. Therefore, it was not clarified whether they were consistent with the depth position where it should have been. Furthermore, the accommodation responses in monocular viewing of an I3D target have been reported. 7 The experimental setup satisfied their proposed SMV condition, 3 and the experimental results indicated that the accommodation responses of over 73% of the participants were induced for the I3D target presented in front of the I3D display. However, no accommodation was induced for the target presented behind the display. Their proposed SMV condition for I3D displays would be very interesting and useful, but experimental verification of it is also important. To measure the accommodation and convergence of the human eye, we consider that it is necessary to satisfy three conditions. The first condition is that the pupil of the observer should dilate to narrow the depth of field (DOF) of the eye. The second condition is to avoid the effect of size cues for depth perception. The third condition is that the accommodation results for 3D targets and real objects should be compared in order to deal with personal differences. Some of these conditions were not satisfied in the previous reports. We reported on the accommodation responses to I3D targets under monocular and binocular viewing conditions. 8 The results showed that integral photography has an advantage over the binocular stereoscopic method in terms of accommodation response. The relationship between depth perception and accommodation responses in viewing I3D targets was also reported. 9 In that report, the depth perception and accommodation responses were verified to be in accordance with the depth positions of the reconstructed I3D target. In recent research, we measured the accommodation and convergence responses to I3D targets. 10 In that research, the responses to I3D targets were found to be nearly the same as those to real targets. However, the resolution dependency on the depth perception was not verified in the case viewing of the I3D target, and neither were the depth perception or accommodation-convergence conflict. The purpose of this study is thus to clarify the resolution dependency on the depth perception and to verify the relationship between the depth perception and accommodation-convergence conflict in viewing I3D targets.
In this paper, we first clarify the depth perception in viewing an I3D target with three display resolutions. Next, we show experimental results of measuring the accommodation and convergence responses to I3D and S3D targets in comparison with the responses to a real target. After that, we discuss the accommodation-convergence conflict.
2 Overview of the Experiment Figure 1 is a schematic diagram of the experiment. The equipment comprised a 3D display device for presenting the target and a system for measuring the accommodation and convergence responses. The 3D display device consisted of an LCD panel and a lens array set 600 mm away from the observer. The LCD had a pixel count of 1920 ðHÞ × 1080 ðVÞ, pixel pitch of 55.5 μm (458 ppi), diagonal screen size of 4.8 in., and RGB stripe pixel structure, and it was driven at 60 Hz. The lens array was composed of small lenses with 1.0 mm diameter with a focal length of 3.0 mm arranged in a honeycomb pattern.

Experimental Setup
The system for measuring the accommodation and convergence responses was an optometry device (PowerRef 3, Plusoptix Inc.) that used the photoreflection method. 11 The measuring equipment was set up to be optically 1.0 m away from the observer, and the measurements were made with infrared light. A hot mirror and a mirror were placed in front of the observer between the measuring equipment and the observer. The hot mirror is a complete reflector of infrared light, but is transparent to visible light; thus, it enables measurement of the accommodation and convergence responses while the observer is intently viewing the target. Note that the measurement was made at 1.0 m from the observer, so þ1.0 D accommodation was included in the result. Table 1 lists the specifications of the 3D display and measuring equipment.
The physiological factors of stereoscopic vision are convergence, accommodation, binocular disparity, and motion parallax. To suppress the motion parallax effect, the depth perception and objective measurement were performed in a state with the observer's head fixed in a chin rest. In addition, the viewing angle of the target size was fixed at 1.9 deg in all depth positions to avoid size cues for depth perception.

Depth Perception
The I3D targets were made from a 2D Maltese cross pattern, as shown in Fig. 2(a). The targets were presented at 13 different depth positions, 420, 450, 480, 510, 540, 570, 600, 630, 660, 690, 720, 750, and 780 mm. The targets were randomly presented at each depth position. For measuring the depth perception at three display resolutions (153, 229, and 458 ppi), 153 and 229 ppi targets were made from the 458 ppi target by downconversion with the nearest-neighbor algorithm.
The depth perception was measured by Scheffe's method of paired comparison. The method uses a criterion and an evaluation stimuli. In this experiment, the targets were divided into upper and lower parts and were presented at the same time. The upper and lower targets were used as the criterion and evaluation stimuli, respectively. The depth position of the lower target (evaluation target) was evaluated from þ3 to −3 against the depth position of the upper target (criterion target). The value was positive (negative) when the depth position of the evaluation target was close to (far from) the observer compared with the depth position of the criterion target. And the values (3, 2, 1, and 0) indicated the perceived difference between the depth of the evaluation target and depth of the criterion  target, with values of 3, 2, 1, 0 showing large, moderate, small, and no difference, respectively. The targets were presented at the 13 depth positions; therefore, the target combination number of one sequence was 169 times at each resolution. One measurement was done when the observer viewed the target. The target presentation of a measurement consisted of 10 s in which the target was shown for, an interval of 5 s in which no image was shown, followed by 10 s in which the target was shown. The rest time between the measurement sequences was more than 5 min to avoid visual fatigue.

Accommodation and Convergence Responses
The accommodation and convergence responses were measured when the targets were viewed with both eyes. Three types of targets were used, i.e., a real target and 3D targets produced by integral photography and the binocular stereoscopic method. In integral photography, the resolution of the 3D images varies with the depth positions of the targets from the display. 12 To compare the accommodation responses to the I3D and S3D targets, the resolution and depth positions of the targets were determined. The resolution of the S3D targets was set to almost the same resolution as the I3D targets. The targets generated by each imaging method were presented at eight different depth positions, as shown in Fig. 2(b). The target depth positions from the observer were 450, 500, 550, 600, 650, 700, 750, and 900 mm. The depth positions of the targets were determined for two reasons. The first reason is that the accommodation responses should be measured in order to consider the effect of the convergence accommodation in the DOF of the ocular optics. The DOF of the human eye is AE0.2 to 0.3 D. 13 The maximum target depth positions, 450 and 900 mm, were AE0.56 D from the 3D display. Therefore, the target depth positions were inside and outside the DOF. The second reason is that the maximum target depth positions were determined from the results of the depth perception experiment. The details of this reason are discussed in Sec. 3.1. For the real target, the target was displayed on the 3D display plane and the 3D display was moved to depth positions that were the same as those of the targets of the 3D imaging methods, as shown in Fig. 2(c). Thus, the three types of targets were displayed at almost the same level of brightness. The resolution of the S3D target was constant at all depth positions; however, the resolution of the I3D target varied with the depth position 12 as shown in Fig. 2(d). In the experiment, the calculated I3D target resolutions displayed from 450 to 900 mm were 1.4, 2.4, 5.2, 10.5, 6.1, 3.3, 2.4, and 1.4 cpd. It is reported that the accommodation responses to the S3D target depend on the target resolution. 14 The accommodation position is induced to the convergence position by convergence accommodation when viewing a lowresolution S3D target. Therefore, it was necessary to make the S3D target resolution about the same as for the I3D target. The I3D target was generated at each depth position by computer simulated ray tracing, and the S3D target was generated from the I3D target. Generation of the S3D target required information on the pupillary distance and the viewing distance from the observer to the 3D display device. Here, the pupillary and viewing distances were assumed to be 65 and 600 mm, respectively.
The observers viewed the three types of target (I3D, S3D, or real object); a measurement sequence consisted of the observer viewing the target at the eight depth positions (each observation at a depth position was a trial). In each experimental trial, the accommodation and convergence responses of the observers were measured using PowerRef 3. One measurement was done for each sequence in which a randomly selected target (I3D, S3D, or real object) was placed at the eight depth positions. The selected target was positioned at one of the eight depth positions. We measured the accommodation and convergence responses of the observer while he or she viewed the target for 10 s. The rest time between the measurement sequences was more than 5 min to avoid visual fatigue. To improve the accuracy of the results, the measurements were done four times. Thus, measurements were made 96 times for each observer (three types of target × eight depth positions × four repetitions).

Participants
Each observer was instructed to look at the target in such a way that it was not seen as a double image, without knowing the target depth. The experiment was performed in a darkened room. The observer's pupils were dilated; however, an artificial pupil or other such device was not used. Ten observers ranging in age from 20-to 30-years old participated in the experiment. The visual acuity of the observers was higher than 0.7. We confirmed that they all had normal stereo vision by using a stereo fly test produced by Stereo Optical Company. The results show that depth perception was dependent on the display resolution. Moreover, the results for high-resolution 458 ppi display were in more accordance with the depth position of the target compared with the results of the lowresolution 153 ppi display. A statistical test evaluated the results at a significance level of 5%. It showed that the recognized depth perception ranges were 180 (from 540 to 720 mm), 240 (from 480 to 720 mm), and 330 mm (from 450 to 780 mm) when the display resolutions were 153, 229, and 458 ppi, respectively. Here, it was expected that the recognized depth perception range would expand as the display resolution increased. Therefore, the maximum recognized depth positions were plotted in the resolution domain, as shown in Fig. 3(b). The graph shows the I3D resolution characteristics of the three display resolutions as calculated by the equations described in Ref. 12. The horizontal axis of the graph indicates the viewing distance from the observer, and the vertical axis represents the resolution. The circles, triangles, and crosses indicate the maximum recognized depth positions (540 and 720 mm in 153 ppi, 480 and 720 mm in 229 ppi, 450 and 780 mm in 458 ppi). As a result, the resolution of the plotted symbols was ∼1.0 cpd (cycles per degree). This result suggests that depth perception is obtained at over 1.0 cpd. Figure 4 shows the experimental results of the accommodation and convergence responses to the I3D and S3D targets. The plots are average results of the 10 observers. Although the accommodation responses were measured for both eyes, only the results for the dominant (left or right) eye are plotted in the graph. In addition, the accommodation and convergence responses were measured for 10 s per measurement. The measured data included incorrect values when the observers blinked their eyes. Thus, the incorrect values were omitted from the calculation of the average results. To compare the experimental results of the 3D and real targets, the horizontal axes indicate the experimental results for the real target, while the vertical axes represent the experimental results for the 3D targets. The diagonal solid line of slope one means that the responses to the 3D targets are the same as the responses to the real target. Figure 4 compares each response to the I3D and S3D targets with the response to the real target. The accommodation responses to the I3D target were closer to that of the real target than to the S3D target. The convergence responses to the I3D and S3D targets showed almost the same tendency. Figure 5(a) shows the relationship between the accommodation and convergence responses to the I3D, S3D, and real targets. The circles, crosses, and triangles denote the average results of the 10 observers. The horizontal axis indicates the experimental results for the convergence responses, while the vertical axis represents the experimental results for the accommodation responses. The diagonal solid line of slope one means that the accommodation responses are the same as the convergence responses. All the experimental results were above the diagonal solid line. However, this is not a significant problem because Fig. 5(a) relatively compares the 3D target data with real target data. The results suggest that fewer observers experienced accommodation-convergence conflict when viewing the I3D target than when viewing the S3D target. To clarify the accommodation-convergence conflict, the results were evaluated using a multiple comparison test at a significance level of 5%. We assumed that the accommodation-convergence conflict occurred when the accommodation or convergence responses to the 3D targets were significantly different from those to the real target. This experiment tested for a significant difference between the accommodation responses to the 3D targets and the accommodation responses to the real target. Note that the differences between the 3D targets and real target were assumed significant when the results of the left and right eyes showed a significant difference. The results of the multiple comparison test are listed in Table 2 and shown in Fig. 5(b). In Table 2, the number of observers with accommodation, convergence, and accommodation-convergence conflicts is listed at each depth position. In Fig. 5

Discussion
From the results of the depth perception test described in Sec. 3.1, the image resolution had to be more than  1.0 cpd to provide depth perception for the I3D target. In the objective evaluation, therefore, the depth positions of the target were set from 450 to 900 mm, as shown in Figs. 2(b) and 2(c). This means that the resolution was greater than 1.0 cpd. Nevertheless, the experimental results for the I3D target shown in Fig. 5(b) indicated that four observers had an accommodation-convergence conflict at the depth position of 450 mm (−2.22 D). It was the main factor that the accommodation responses in Table 2 had a more significant difference than the convergence responses. The accommodation responses to the I3D target were closer to that of the real target than to the S3D target as shown in Fig. 4(a). The accommodation responses to the I3D target were not entirely consistent with those to the real target. The I3D target resolution decreased with distance from the 3D display because of the performance limitations of the 3D display used in this experiment. It is considered that the depth perception and accommodation responses to the I3D target were mainly limited by the I3D target resolution. It is reported that the accommodation responses to an I3D target depend on the target resolution. 6 In that report, the accommodation responses to a high-resolution I3D target obtained better results. Therefore, the results of the resolution dependency on the depth perception indicated that the accommodation-convergence conflict in I3D target decreases with an increase in the resolution of the 3D display. In addition, the accommodation-convergence conflict also appeared at a depth position of 600 mm (−1.66 D). The measurements of the accommodation and convergence responses to the I3D, S3D, and real targets were conducted under the same viewing conditions at a depth position of 600 mm (−1.66 D). In other words, the accommodation-convergence conflict did not occur at the depth position of 600 mm (−1.66 D), in principle. Taking the measurement error into consideration, it is expected that the number of observers with accommodation-convergence conflict in viewing the I3D target would decrease compared with the results of where p p is the pixel pitch of the display, f is the focal length of each lens of the lens array, and d p is the pupil diameter. In our experimental setup, p p and f were 55.5 and 3.0 mm, and d p was assumed to be 5.0 mm. Although our experimental setup did not satisfy the SMV condition using Eq. (1), our measurement results indicated that accommodation responses to the I3D target were almost the same as that to the real target. In other research (Refs. 5 and 6), the experimental setups did not satisfy the SMV condition. In those experimental setups as well as ours, however, the accommodation responses to the I3D target were induced to the depth positions of the I3D target. The SMV condition analysis proposed in Ref. 3 is very powerful tool to analyze the 3D images; however, it may not be always sufficient for analyzing the accommodation responses to an I3D display. On the other hand, the measurement results of accommodation and depth perception in viewing I3D displays are useful references for designing such displays. The experimental results on depth perception presented here suggest that the recognized depth range was ∼1.0 cpd.

Conclusion
The accommodation-convergence conflict in viewing I3D targets was experimentally verified by subjective and objective evaluations. The subjective evaluation was carried out with three different display resolutions to clarify the image resolution dependency on depth perception, and indicated that the image resolution had to be more than 1.0 cpd to provide depth perception for the I3D target. We have evaluated the accommodation and convergence responses to the I3D target as well as a binocular stereoscopic 3D (S3D) target and a real target for comparison. The results suggest that the accommodation-convergence conflict in integral photography method was much smaller than in the binocular