This PDF file contains the front matter associated with SPIE Proceedings Volume 7863, including the Title Page, Copyright information, Table of Contents, Introduction, and the Conference Committee listing

The 3D shape perceived from viewing a stereoscopic movie depends on the viewing conditions, most notably on the screen size and distance, and depth and size distortions appear because of the differences between the shooting and viewing geometries. When the shooting geometry is constrained, or when the same stereoscopic movie must be displayed with different viewing geometries (e.g. in a movie theater and on a 3DTV), these depth distortions may be reduced by novel view synthesis techniques. They usually involve three steps: computing the stereo disparity, computing a disparity-dependent 2D mapping from the original stereo pair to the synthesized views, and finally composing the synthesized views. In this paper, we focus on the second and third step: we examine how to generate new views so that the perceived depth is similar to the original scene depth, and we propose a method to detect and reduce artifacts in the third and last step, these artifacts being created by errors contained in the disparity from the first step.

In this paper, we proposed a visual fatigue monitoring system based on eye-movement and eye-blink detection. It analyzes the eye-movement and number of blinks based on the assumption that saccade movement of the eye decreases and the number of eye blink increases when visual fatigue of viewer is accumulated. The proposed system has an infrared single camera and an infrared light source. Then, the pupil of the eye can be detected by applying binary threshold to Purkinje image. The threshold is automatically selected by two constraints which are the eccentricity of ellipse fitting and the size of the pupil. Finally, total amount of eye movement and the number of eye blinks are measured by using the positions of the pupil. The results were obtained while watching stereoscopic videos after personal calibration procedure. The results show that saccade movement of the eye decreases as the visual fatigue of the viewer is accumulated. However, the number of eye blinks shows large variance along the time axis which implies it is not proper for visual fatigue monitoring system.

The stereoscopic 3D industry has fallen short of achieving acceptable Quality of Experience (QoE) because of various technical limitations, such as excessive disparity, accommodation-convergence mismatch. This study investigates the effect of scene content, camera baseline, screen size and viewing location on stereoscopic QoE in a holistic approach. 240 typical test configurations are taken into account, in which a wide range of disparity constructed from the shooting conditions (scene content, camera baseline, sensor resolution/screen size) was selected from datasets, making the constructed disparities locate in different ranges of maximal disparity supported by viewing environment (viewing location). Second, an extensive subjective test is conducted using a single stimulus methodology, in which 15 samples at each viewing location were obtained. Finally, a statistical analysis is performed and the results reveal that scene content, camera baseline, as well as the interactions between screen size, scene content and camera baseline, have significant impact on QoE in stereoscopic images, while other factors, especially viewing location involved, have almost no significant impact. The generated Mean Opinion Scores (MOS) and the statistical results can be used to design stereoscopic quality metrics and validate their performance.

This paper investigates visual discomfort induced by fast motion of salient object in a stereoscopic video. We have conducted a subjective assessment to investigate the degree of visual discomfort caused by motion characteristics of a controlled graphics object in a video scene. As results of the subjective assessment, we observe the changes of the degree of visual discomfort with varying velocity and direction of object motion. In order to verify the acceptability of our observation for real stereoscopic 3D videos, we exploit the concept of visual saliency to define the salient object motion severely affecting the degree of visual discomfort in a video scene. The salient object motion feature is extracted and a visual comfort model is derived from our observation. Then we predict the degree of visual discomfort by using the extracted motion feature and the visual comfort model. We have conducted a subjective test to compare the predicted visual comfort score with actual subjective score. The experiment results show that the predicted visual comfort score correlates well with the actual subject score.

One of the key issues associated with 3D TVs is the tradeoff between comfort and 3D visual impact. Big disparity is often preferred for strong visual impact but often lead to viewer discomfort depending on viewer's condition, display size and viewing distances. The algorithm proposed in this paper is to provide viewers a tool to adjust disparity according to the environment, contents and their preference in order to have more comfortable and higher quality 3D experiences. More specifically, given a planar stereoscopic display, the algorithm takes in a stereoscopic image pair that causes viewing discomfort/fatigue, and outputs a modified stereoscopic pair that causes less or no viewing discomfort/fatigue. The algorithm fulfills the functions of disparity estimation, occlusion detection, disparity adjustment and view synthesis. A novel pixel weighting mechanism in regularized-block-matching based disparity estimation helps improve the robustness, accuracy and speed of matching. Occlusion detection uses multiple cues in addition to matching errors to improve the accuracy. An accommodation/vergence mismatch visual model is used in disparity adjustment to predict discomfort/fatigue from the disparity information, the viewing conditions and display characteristics. The hole filling is in the disparity map of the new view instead of the new view itself to reduce the blurriness.

The creation of binocular images for stereoscopic display has benefited from significant research and commercial development in recent years. However, perhaps surprisingly, the effect of adding 3D sound to stereoscopic images has rarely been studied. If auditory depth information can enhance or extend the visual depth experience it could become an important way to extend the limited depth budget on all 3D displays and reduce the potential for fatigue from excessive use of disparity. Objective: As there is limited research in this area our objective was to ask two preliminary questions. First what is the smallest difference in forward depth that can be reliably detected using 3D sound alone? Second does the addition of auditory depth information influence the visual perception of depth in a stereoscopic image? Method: To investigate auditory depth cues we use a simple sound system to test the experimental hypothesis that: participants will perform better than chance at judging the depth differences between two speakers a set distance apart. In our second experiment investigating both auditory and visual depth cues we setup a sound system and a stereoscopic display to test the experimental hypothesis that: participants judge a visual stimulus to be closer if they hear a closer sound when viewing the stimulus. Results: In the auditory depth cue trial every depth difference tested gave significant results demonstrating that the human ear can hear depth differences between physical sources as short as 0.25 m at 1 m. In our trial investigating whether audio information can influence the visual perception of depth we found that participants did report visually perceiving an object to be closer when the sound was played closer to them even though the image depth remained unchanged. Conclusion: The positive results in the two trials show that we can hear small differences in forward depth between sound sources and suggest that it could be practical to extend the apparent depth in a stereoscopic image by using 3D sound, providing a controlled way to compensate for the depth budget limits on 3D displays.

The perception of space in the real world is based on multifaceted depth cues, most of them monocular, some binocular. Developing 3D-displays raises the question, which of these depth cues are predominant and should be simulated by computational means in such a panel. Beyond the cues based on image content, such as shadows or patterns, Stereopsis and depth from motion parallax are the most significant mechanisms supporting observers with depth information. We set up a carefully designed test situation, widely excluding undesired other distance hints. Thereafter we conducted a user test to find out, which of these two depth cues is more relevant and whether a combination of both would increase accuracy in a depth estimation task. The trials were conducting utilizing our autostereoscopic "Free2C"-displays, which are capable to detect the user eye position and steer the image lobes dynamically into that direction. At the same time, eye position was used to update the virtual camera's location and thereby offering motion parallax to the observer. As far as we know, this was the first time that such a test has been conducted using an autosteresocopic display without any assistive technologies. Our results showed, in accordance with prior experiments, that both cues are effective, however Stereopsis is by order of magnitude more relevant. Combining both cues improved the precision of distance estimation by another 30-40%.

This paper describes an algorithm for estimating the disparity between 2 images of a stereo pair. The disparity is related to the depth of the objects in the scene. Being able to obtain the depth of the objects in the scene is useful in many applications such as virtual reality, 3D user interfaces, background-foreground segmentation, or depth-image-based synthesis. This last application has motivated the proposed algorithm as part of a system that estimates disparities from a stereo pair and synthesizes new views. Synthesizing virtual views enables the post-processing of 3D content to adapt to user preferences or viewing conditions, as well as enabling the interface with multi-view auto-stereoscopic displays. The proposed algorithm has been designed to fulfill the following constraints: (a) low memory requirements, (b) local and parallelizable processing, and (c) adaptability to a sudden reduction in processing resources. Our solution uses a multi-resolution multi-size-windows approach, implemented as a line-independent process, well-suited for GPU implementation. The multi-resolution approach provides adaptability to sudden reduction in processing capabilities, besides computational advantages; the windows-based image processing algorithm guarantees low-memory requirements and local processing.

Interactive free-viewpoint selection applied to a 3D multi-view video signal is an attractive feature of the rapidly developing 3DTV media. In recent years, significant research has been done on free-viewpoint rendering algorithms which mostly have similar building blocks. In our previous work, we have analyzed the principal building blocks of most recent rendering algorithms and their contribution to the overall rendering quality. We have discovered that the first step, Warping determines the basic quality level of the complete rendering chain. In this paper, we have analyzed the warping step in more detail since it leads to ways for improvement. We have observed that the accuracy of warping is mainly determined by two factors which are sampling and rounding errors when performing pixel-based warping and quantization errors of depth maps. For each error factor, we have proposed a technique that can reduce the errors and thus increase the warping quality. Pixel-based warping errors are reduced by employing supersampling at the reference and virtual images and we decrease depth map errors by creating depth maps with more quantization levels. The new techniques are evaluated with two series of experiments using real-life and synthetic data. From these experiments, we have observed that reducing warping errors may increases the overall rendering quality and that the impact of errors due to pixel-based warping is much larger than that of errors due to depth quantization.

We are developing technologies for FTV in which the viewer can freely change the viewpoint. The free-viewpoint image can be generated by using images captured by an static multi-camera system. However, it is hard to render an object that moves widely in the scene. In this paper, we address this problem by proposing moving camera array and the free-viewpoint image synthesis algorithm. In our synthesis method, we use the temporal and spatial information together, in order to further improve the view generation quality. Experiments using a sequence captured by simulated moving multi-camera systems demonstrate the improvement of view synthesis quality in comparison with conventional view synthesis scheme.

3D Video (3DV) delivery standardization is currently ongoing in MPEG. Now time is to choose 3DV data representation format. What is at stake is the final quality for end-users, i.e. synthesized views' visual quality. We focus on two major rival depth-based formats, namely Multiview Video plus Depth (MVD) and Layered Depth Video (LDV). MVD can be considered as the basic depth-based 3DV format, generated by disparity estimation from multiview sequences. LDV is more sophisticated, with the compaction of multiview data into color- and depth-occlusions layers. We compare final views quality using MVD2 and LDV (both containing two color channels plus two depth components) coded with MVC at various compression ratios. Depending on the format, the appropriate synthesis process is performed to generate final stereoscopic pairs. Comparisons are provided in terms of SSIM and PSNR with respect to original views and to synthesized references (obtained without compression). Eventually, LDV outperforms significantly MVD when using state-of-the-art reference synthesis algorithms. Occlusions management before encoding is advantageous in comparison with handling redundant signals at decoder side. Besides, we observe that depth quantization does not induce much loss on the final view quality until a significant degradation level. Improvements in disparity estimation and view synthesis algorithms are therefore still expected during the remaining standardization steps.

In order to compress the multiview image efficiently, we propose a new basis representation by using directional sampling.Because the multiview image has the static correlation related to the camera position, directional sampling can eliminate the correlation efficiently. We apply directional three dimensional Discrete Cosine Transform (directional 3D-DCT), directional three dimensional Discrete Wavelet Transform (directional 3D-DWT) and directional three dimensional Principal Component Analysis (directional 3D-PCA) to three dimensional data that has vertical, horizontal and view axis. In the experimental results, the method by using directional sampling showed better quality than conventional sampling, not only in the objective evaluation, but also in the subjective evaluation. Furthermore, the proposed method showed better Rate-PSNR than conventional method (disparity compensation).

Multiview displays suffer from two common artifacts - Moiré, caused by aliasing, and ghosting artifacts caused by crosstalk. By measuring the angular brightness function of each TFT element we create so-called brightness mask, which allows us to simulate the display output for a given input image. We consider multiview display as image processing channel and model the artifacts as distortions of the input signal. We test the channel by using a set of signals with various frequency components as input, and analyzing the output in the frequency domain. We derive the so-called bandpass region of the display, where the distortions introduced to the input signals are under certain threshold. Then, we extend the simulations including input signals with varying disparity, and obtain multiple passbands - one for each disparity level. We approximate each passband with a rectangle and store the height and width of that rectangle in a table. We propose an artifact mitigation framework which can be used for realtime processing of textures with known apparent depth. The framework gives the user ability to set so-called "3D-shapness" - a parameter, which controls the trade-off between visibility of details and presence of artifacts. The "3D-sharpness parameter determines what level of distortions is allowed in the final image, regardless of its disparity. The framework uses the approximated width and height of the passband areas in order to design an optimal (for the needed disparity and desired distortion level) anti-aliasing filter. We discuss a methodology for filter design, and show example results, based on measurements of an 8-view display.

In recent years, we have seen several different approaches dealing with multiview compression. First, we can find the H264/MVC extension which generates quite heavy bitstreams when used on n-views autostereoscopic medias and does not allow inter-view reconstruction. Another solution relies on the MVD (MultiView+Depth) scheme which keeps p views (n > p > 1) and their associated depth-maps. This method is not suitable for multiview compression since it does not exploit the redundancy between the p views, moreover occlusion areas cannot be accurately filled. In this paper, we present our method based on the LDV (Layered Depth Video) approach which keeps one reference view with its associated depth-map and the n-1 residual ones required to fill occluded areas. We first perform a global per-pixel matching step (providing a good consistency between each view) in order to generate one unified-color RGB texture (where a unique color is devoted to all pixels corresponding to the same 3D-point, thus avoiding illumination artifacts) and a signed integer disparity texture. Next, we extract the non-redundant information and store it into two textures (a unified-color one and a disparity one) containing the reference and the n-1 residual views. The RGB texture is compressed with a conventional DCT or DWT-based algorithm and the disparity texture with a lossless dictionary algorithm. Then, we will discuss about the signal deformations generated by our approach.

We consider the problem of upsampling a low-resolution depth map generated by a range camera, by using information from one or more additional high-resolution vision cameras. The goal is to provide an accurate high resolution depth map from the viewpoint of one of the vision cameras. We propose an algorithm that first converts the low resolution depth map into a depth/disparity map through coordinate mappings into the coordinate frame of one vision camera, then classifies the pixels into regions according to whether the range camera depth map is trustworthy, and finally refine the depth values for the pixels in the untrustworthy regions. For the last refinement step, both a method based on graph cut optimization and that based on bilateral filtering are examined. Experimental results show that the proposed methods using classification are able to upsample the depth map by a factor of 10 x 10 with much improved depth details, with significantly better accuracy comparing to those without the classification. The improvements are visually perceptible on a 3D auto-stereoscopic display.

Attack! of the S. Mutans is a multi-player game designed to harness the immersion and appeal possible with wide-fieldof- view stereoscopic 3D to combat the tooth decay epidemic. Tooth decay is one of the leading causes of school absences and costs more than $100B annually in the U.S. In 2008 the authors received a grant from the National Institutes of Health to build a science museum exhibit that included a suite of serious games involving the behaviors and bacteria that cause cavities. The centerpiece is an adventure game where five simultaneous players use modified Wii controllers to battle biofilms and bacteria while immersed in environments generated within a 11-foot stereoscopic WUXGA display. The authors describe the system and interface used in this prototype application and some of the ways they attempted to use the power of immersion and the appeal of S3D revolution to change health attitudes and self-care habits.

Multi-perspective image acquisition constitutes a novel method of documenting the entireness of an event, by recording from multiple points of view. In combination with stereoscopic imaging, it captures two modalities of three-dimensional representation, perspective and depth. These modalities are then applied for display in a hexagonal stereoscopic multi-screen projection environment, the RE-ACTOR platform. In this paper the author describes, in form of two case studies, the implementation of two projects for the platform within the performing art context.

Purpose: Machine vision with 3DHD video theoretically offers resolving capacity approaching that of the operating microscope and may have applications in vitreoretinal surgery. Methods: 9 eyes of 9 patients were operated. Surgical visualization was provided by one of two 3DHD video systems. The surgeon did not look through the operating microscope at any time from the beginning to the end of the surgical procedure. Results: 9 pars plana vitrectomies, 8 epiretinal membrane peels, 7 ILM peels, 3 cataract surgeries, one air fluid exchange, and one retrieval of a luxed intraocular lens implant from the voitreous cavity were performed successfully. Anatomic and visual results were excellent. No complications occurred. Surgeon comfort during surgery was superb. Conclusions: 3DHD machine vision may be utilized for surgical visualization in select vitreoretinal patients. The implications and applications of this new technology remain unknown.

In this study, we proposed methods to reduce a black-white (BW) 3D crosstalk and gray-to-gray (GtoG) 3D crosstalk in active retarder 3D (AR3D) technology. To obtain the BW 3D crosstalk to 2.5% without backlight scanning, we first optimized the switching time of AR panel. Then, the BW 3D crosstalk of 1.0% was achieved by scanning the backlight synchronized with the liquid crystal display (LCD) panel and AR panel. Finally, with the over-driving method for various condition of gray-to-gray transition, the average GtoG 3D crosstalk was obtained to be under 1.8% showing clear 3D images. With these concepts of reducing the 3D crosstalk in AR3D technology, we developed AR3D monitor having 23 inches in diagonal with full HD resolution. The luminance of our prototype is 73 nits at 3D mode through polarizer glasses and 300 nits at 2D mode for bare eyes, showing that 24% light efficiency in 3D mode. In conclusion, our prototype shows clear 3D image with full HD resolution and high luminance even though it employs polarizer glasses.

A stereoscopic 35mm motion picture projection system is described consisting of a sideframe format with high effective frame area utilization and high brightness optics. The system has been designed to work within the existing motion picture content creation and distribution infrastructures.

A number of Pulfrich 3-D movies and TV shows have been produced, but the standard implementation has inherent drawbacks. The movie and TV industries have correctly concluded that the standard Pulfrich 3-D implementation is not a useful 3-D technique. Continuously Adjustable Pulfrich Spectacles (CAPS) is a new implementation of the Pulfrich effect that allows any scene containing movement in a standard 2-D movie, which are most scenes, to be optionally viewed in 3-D using inexpensive viewing specs. Recent scientific results in the fields of human perception, optoelectronics, video compression and video format conversion are translated into a new implementation of Pulfrich 3- D. CAPS uses these results to continuously adjust to the movie so that the viewing spectacles always conform to the optical density that optimizes the Pulfrich stereoscopic illusion. CAPS instantly provides 3-D immersion to any moving scene in any 2-D movie. Without the glasses, the movie will appear as a normal 2-D image. CAPS work on any viewing device, and with any distribution medium. CAPS is appropriate for viewing Internet streamed movies in 3-D.

Prolonged use of conventional stereo displays causes viewer discomfort and fatigue because of the vergenceaccommodation conflict. We used a novel volumetric display to examine how viewing distance and the sign of the vergence-accommodation conflict affect discomfort and fatigue. In the first experiment, we presented a fixed conflict at short, medium, and long viewing distances. We compared subjects' symptoms in that condition and one in which there was no conflict. We observed more discomfort and fatigue with a given vergence-accommodation conflict at the longer distances. The second experiment compared symptoms when the conflict had one sign compared to when it had the opposite sign at short, medium, and long distances. We observed greater symptoms with uncrossed disparities at long distances and with crossed disparities at short distances. These findings help define comfortable viewing conditions for stereo displays.

Vergence and accommodative responses in viewing near objects in the real world are behaviorally coupled to maintain clarity and singularity for the object of regard. However virtual stereoscopic stimuli, such as in 3D displays, create non-normal coupling that may cause improper vergence and accommodative responses, possibly resulting in visual discomfort. The present study examined whether the dynamic aspect of current 3D displays is the underlying cause of visual and physical discomfort. To this end, subjects' vergence and accommodative responses were measured while they tracked an approaching 2D or 3D target, and while watching a 2D or 3D movie. The tracking target either moved in steps or continuously, and it was either clear or intentionally blurred. Results show that convergence insufficiency and improper accommodation were greater when a 3D target was moving continuously toward the near position compared to a 2D target and a 3D stimulus moving in steps. Clear targets also resulted in greater vergence and accommodative responses than blurred targets. Viewing 3D movie resulted in greater vergence and accommodation, as well as more severe vision- and motion-related discomfort than 2D movie. These findings suggest that with 3D displays, disparity-induced vision difficulty and internal conflicts cause perceived visual and motion-related discomfort. Characteristics of 3D stimuli, such as the frequency and amplitude of target motion, likely critically affect the severity of reported discomfort symptoms.

Camera separation affects the perceived depth in stereoscopic movies. Through control of the separation and thereby the depth magnitudes, the movie can be kept comfortable but interesting. In addition, the viewing context has a significant effect on the perceived depth, as a larger display and longer viewing distances also contribute to an increase in depth. Thus, if the content is to be viewed in multiple viewing contexts, the depth magnitudes should be carefully planned so that the content always looks acceptable. Alternatively, the content can be modified for each viewing situation. To identify the significance of changes due to the viewing context, we studied the effect of stereoscopic camera base distance on the viewer experience in three different situations: 1) small sized video and a viewing distance of 38 cm, 2) television and a viewing distance of 158 cm, and 3) cinema and a viewing distance of 6-19 meters. We examined three different animations with positive parallax. The results showed that the camera distance had a significant effect on the viewing experience in small display/short viewing distance situations, in which the experience ratings increased until the maximum disparity in the scene was 0.34 - 0.45 degrees of visual angle. After 0.45 degrees, increasing the depth magnitude did not affect the experienced quality ratings. Interestingly, changes in the camera distance did not affect the experience ratings in the case of television or cinema if the depth magnitudes were below one degree of visual angle. When the depth was greater than one degree, the experience ratings began to drop significantly. These results indicate that depth magnitudes have a larger effect on the viewing experience with a small display. When a stereoscopic movie is viewed from a larger display, other experiences might override the effect of depth magnitudes.

High-resolution stereoscopic images are effective for use in virtual reality and teleoperation systems. However, the higher the image resolution, the higher is the cost of computer processing and communication. To reduce this cost, numerous earlier studies have suggested the use of multi-resolution images, which have high resolution in region of interests and low resolution in other areas. However, observers can perceive unpleasant sensations and incorrect depth because they can see low-resolution areas in their field of vision. In this study, we conducted an experiment to research the relationship between the viewing field and the perception of image resolution, and determined respective thresholds of image-resolution perception for various positions of the viewing field. The results showed that participants could not distinguish between the high-resolution stimulus and the decreased stimulus, 63 ppi, at positions more than 8 deg outside the gaze point. Moreover, with positions shifted a further 11 and 13 deg from the gaze point, participants could not distinguish between the high-resolution stimulus and the decreased stimuli whose resolution densities were 42 and 25 ppi. Hence, we will propose the composition of multi-resolution images in which observers do not perceive unpleasant sensations and incorrect depth with data reduction (compression).

A projector array-based 70-inch screen display, which is our first prototype, has a smooth horizontal parallax and gives a dense viewpoint interval that is narrower than half of the interocular distance. Our final goal is to develop advanced autostereoscopy so that viewers are not compelled to wear 3D glasses and can avoid watching under insufficient resolution. We believe that larger screen size, higher image quality, and such natural image appearances as motion parallax and multiple viewer capability are priority targets for professional 3D display applications. By combining a proprietary screen and our developed projector array, we've designed and implemented a kind of autostereoscopic projection display. Enough pixels to render true high definition are assigned for every viewpoint. The initial implementation has more than 100 million pixels. The actual observed horizontal motion parallax is smooth and reduces flipping. This feasibility study clarified the following two factors: the strong requirement of an array friendly feature ready projector, and the existence of some image glitches. The appearances of moires and ghost images are the most significant factors of visual fatigue in our implementation. Some of these problems were tackled and suppressed. The projectors for the array must be prepared to manage color space, brightness, geometric image compensation, and accurate frame synchronization. Extracting and examining the practical problems with an autostereoscopic projection display are the first steps of our feasibility study. Our goal is to establish an autostereoscopic display with natural and superior horizontal parallax.

We propose full-parallax integral imaging display with 360 degree horizontal viewing angle. Two-dimensional (2D) elemental images are projected by a high-speed DMD projector and integrated into three-dimensional (3D) image by a lens array. The anamorphic optic system tailors the horizontal and vertical viewing angles of the integrated 3D images in order to obtain high angular ray density in horizontal direction and large viewing angle in vertical direction. Finally, the mirror screen that rotates in synchronization with the DMD projector presents the integrated 3D images to desired direction accordingly. Full-parallax and 360 degree horizontal viewing angle 3D images with both of monocular and binocular depth cues can be achieved by the proposed method.

With the rapid emergence of autosterereoscopic 3D (AS3D) displays in the market the need for optical characterization of these devices and their components is increasing. This paper reviews various performance attributes of AS3D displays such as viewing distance, banding offset, banding angle and stereoedge, and reviews measurement results from a parallax barrier, lenticular and directional back light approach from both conventional measurement techniques and a new instrument, called the optical inspection station (OIS).

We propose Depth Cube Display (DCD) method using depth map. The structure of the proposed method consists of two parts: A projection part composed of projector for generating image and a Twisted Nematic Liquid Crystal display (TNLCD) as polarization modulating device for adjusting the proper depth and a display part composed of air-spaced stack of selective scattering polarizers which make the incident light to scatter selectively as the polarization of light rays. The image from projector whose depth is determined as passing through the TN-LCD displaying depth map progresses into the stack of selective scattering polarizers and then three-dimensional image is generated. At that time, the polarization of each polarizer is set 0°, 45° and 90° sequentially, and then the incident light rays are scattered by different polarizer as the polarization of these rays. If the light ray has the polarization between those of polarizers, this light ray is scattered by multi polarizers and the image of this ray is generated on gap between polarizers. The proposed method is more simple structure and implemented easily than previous DCD method.

In this paper, we propose a novel method of representing the complex surface of a 3D object for a new aerial 3D display which can draw dots of light at an arbitrary position in a space. The aerial 3D display that we use in this research can create a dot of light at 50 kHz and can draw dots of light by vector scanning. The proposed method can generate point sequence data for the aerial 3D display from 3D surface models consisting of polygonal patches. The 3D surface model is polygonal model which are generally used in computer graphics. The proposed method represents the surface with contours consisting of intersections of an object and cross sections by a sequence of points for vector scanning. In this research, some polygonal models, for example face and hand, are examined at experiments. From the experiments of drawing, the polygonal models can successfully be drawn by the proposed method.

Crosstalk is a critical factor determining the image quality of stereoscopic displays. Also known as ghosting or leakage, high levels of crosstalk can make stereoscopic images hard to fuse and lack fidelity; hence it is important to achieve low levels of crosstalk in the development of high-quality stereoscopic displays. In the wider academic literature, the terms crosstalk, ghosting and leakage are often used interchangeably and unfortunately very few publications actually provide a descriptive or mathematical definition of these terms. Additionally the definitions that are available are sometimes contradictory. This paper reviews how the terms crosstalk, ghosting and associated terms (system crosstalk, viewer crosstalk, gray-to-gray crosstalk, leakage, extinction and extinction ratio, and 3D contrast) are defined and used in the stereoscopic literature. Both descriptive definitions and mathematical definitions are considered.

Crosstalk (also known as "ghosting", "leakage", or "extinction"), a vitally important concept in stereoscopic 3D displays, has not been clearly defined or measured in the stereoscopic literature (Woods). In this paper, a mathematical definition is proposed which uses a "physical" approach. This derivation leads to a clear definition of leftview or right-view crosstalk and shows that 1), when the display's black level is not zero, it must be subtracted out and 2), when the source intensities are equal, crosstalk can be measured using observed intensities totally within the respective view. Next, a simple method of measuring crosstalk is presented, one that relies on only viewing a test chart on the display. No electronic or optical instruments are needed. Results of the use of the chart are presented, as well as optical measurements, which did not agree well with chart results. The main reason for the discrepancy is the difficulty of measuring very low light levels. With wide distribution, this tool can lead to the collection of useful performance information about 3D displays and, therefore, to the production of the best stereoscopic displays.

Crosstalk is a phenomenon in stereoscopy where an image becomes blurry due to leakage of the left image into the right eye and vice versa, and is considered one of the serious problems impairing stereoscopic experience. The current study examines mental/cognitive activity under a various levels of crosstalk through heart activity and forehead blood flow. In the experiment that presented three still natural images and one graphical video with a various crosstalk levels, heart rate showed a decelerative-accelerative-decelerative pattern for all the stimuli up to the intolerably severe level. The result suggests changes in mental state in accordance to the crosstalk level: i.e. orientation response under no perceived crosstalk, active mental elaboration upon noticing crosstalk, and reduced level of elaboration as crosstalk progressed. The pattern, however, did not always agree amongst the physiological measures and the crosstalk ratios. This suggests that the mental state under crosstalked image viewing could be more complex than a simple combination of orientation response and active mental elaboration.

A method to characterize time sequential stereoscopic 3D displays which is based on the measurement of the temporal behavior of the system versus grey levels is presented. OPTISCOPE SA, especially designed for precise measurement of luminance and temporal behavior of LCD displays is used. The transmittance and response time of the shutter glasses is first evaluated. Then the grey to grey response times of the display alone are measured. Finally, the temporal behavior of the complete system is modeled and grey to grey crosstalks are deduced. Resulting luminance on left and right eyes of the observer are deduced and compared to integrated measurements using a spectrophotometer. Quality control method using grey level test patterns and imaging luminance meter is deduced and applied to different types of commercial 3D TVs.

Stereoscopic displays must present separate images to the viewer's left and right eyes. Crosstalk is the unwanted contamination of one eye's image from the image of the other eye. It has been shown to cause distortions, reduce image quality and visual comfort and increase perceived workload when performing visual tasks. Crosstalk also affects one's ability to perceive stereoscopic depth although little consideration has been given to the perception of depth magnitude in the presence of crosstalk. In this paper we extend a previous study (Tsirlin, Allison & Wilcox, 2010, submitted) on the perception of depth magnitude in stereoscopic occluding and non-occluding surfaces to the special case of crosstalk in thin structures. Crosstalk in thin structures differs qualitatively from that in larger objects due to the separation of the ghost and real images and thus theoretically could have distinct perceptual consequences. To address this question we used a psychophysical paradigm, where observers estimated the perceived depth difference between two thin vertical bars using a measurement scale. Our data show that crosstalk degrades perceived depth. As crosstalk levels increased the magnitude of perceived depth decreased, especially for stimuli with larger relative disparities. In contrast to the effect of crosstalk on depth magnitude in larger objects, in thin structures, a significant detrimental effect was found at all disparities. Our findings, when considered with the other perceptual consequences of crosstalk, suggest that its presence in S3D media even in modest amounts will reduce observers' satisfaction.

The present study investigates whether VIMS, which can be induced in 2D images, is affected by stereoscopic presentation. To do this, we conducted an experiment to measure the effects psychologically and physiologically. Visual stimulus was computer graphics that simulates traveling along streets with additional pitch and roll motion for 10 minutes. The stimulus were presented as either stereoscopic, "3D", images or "2D" images. Before/after and during each trial, psychological and physiological measurements for biomedical effects were conducted. As results, psychological measurements indicate effects of stereoscopic presentations on VIMS. First, subjective score of comfort level measured every one minute significantly decreased to uncomfortable level in the 3D than in the 2D condition. Second, subscore of "Nausea" of Simulator Sickness Questionnaire significantly higher in the 3D than in the 2D condition, while the other subscores and the total score also showed the similar tendency. Moreover, physiological measurements also indicate effects of 3D presentations on VIMS. The LF/HF ratio, which is the index of sympathetic nerve activity, clearly increased more in the 3D than in the 2D condition. We conclude that stereoscopic presentation enhances biomedical effects of VIMS. We speculate that stereoscopic images can be efficient reference of spatial orientation.

Conventional stereoscopic displays present images on a single focal plane. The resulting mismatch between the stimuli to the eyes' focusing response (accommodation) and to convergence causes fatigue and poor stereo performance. One promising solution is to distribute image intensity across a number of relatively widely spaced image planes - a technique referred to as depth filtering. Previously, we found this elicits accurate, continuous monocular accommodation responses with image-plane separations as large as 1.1 Diopters, suggesting that a relatively small (i.e. practical) number of image planes is sufficient to eliminate vergence-accommodation conflicts over a large range of simulated distances. However, accommodation responses have been found to overshoot systematically when the same stimuli are viewed binocularly. Here, we examined the minimum image-plane spacing required for accurate accommodation to binocular depth-filtered images. We compared accommodation and vergence responses to step changes in depth for depth-filtered stimuli, using image-plane separations of 0.6-1.2 D, and equivalent real stimuli. Accommodation responses to real and depth-filtered stimuli were equivalent for image-plane separations of ~0.6-0.9 D, but inaccurate thereafter. We conclude that depth filtering can be used to precisely match accommodation and vergence demand in a practical stereoscopic display, using a relatively small number of image planes.

Stereoscopic displays are known to offer a number of key advantages in visualizing complex 3D structures or datasets. The large majority of studies that focus on evaluating stereoscopic displays for professional applications use completion time and/or the percentage of correct answers to measure potential performance advantages. However, completion time and accuracy may not fully reflect all the benefits of stereoscopic displays. In this paper, we argue that perceived workload is an additional valuable indicator reflecting the extent to which users can benefit from using stereoscopic displays. We performed an experiment in which participants were asked to perform a visual path-tracing task within a convoluted 3D wireframe structure, varying in level of complexity of the visualised structure and level of disparity of the visualisation. The results showed that an optimal performance (completion time, accuracy and workload), depend both on task difficulty and disparity level. Stereoscopic disparity revealed a faster and more accurate task performance, whereas we observed a trend that performance on difficult tasks stands to benefit more from higher levels of disparity than performance on easy tasks. Perceived workload (as measured using the NASA-TLX) showed a similar response pattern, providing evidence that perceived workload is sensitive to variations in disparity as well as task difficulty. This suggests that perceived workload could be a useful concept, in addition to standard performance indicators, in characterising and measuring human performance advantages when using stereoscopic displays.

We compare two different modes of interaction with a large stereoscopic display, where the physical pointing device is in a volume distinct from the display volume. In absolute mode, the physical pointer's position exactly maps to the virtual pointer's position in the display volume, analogous to a 2D graphics table and 2D screen. In relative mode, the connection between the physical pointer's motion and the motion of the virtual pointer in the display volume is analogous to that obtained with a 2D mouse and 2D screen. Both statistical analysis and participants' feedback indicated a strong preference for absolute mode over relative mode. This is in contrast to 2D displays where relative mode (mouse) is far more prevalent than absolute mode (tablet). We also compared head-tracking against no head-tracking. There was no statistically-significant advantage to using head-tracking, however almost all participants strongly favoured head-tracking.

A plenoptic camera is a natural multi-view acquisition device also capable of measuring distances by correlating a set of images acquired under different parallaxes. Its single lens and single sensor architecture have two downsides: limited resolution and depth sensitivity. In a very first step and in order to circumvent those shortcomings, we have investigated how the basic design parameters of a plenoptic camera optimize both the resolution of each view and also its depth measuring capability. In a second step, we built a prototype based on a very high resolution Red One® movie camera with an external plenoptic adapter and a relay lens. The prototype delivered 5 video views of 820x410. The main limitation in our prototype is view cross talk due to optical aberrations which reduce the depth accuracy performance. We have simulated some limiting optical aberrations and predicted its impact on the performances of the camera. In addition, we developed adjustment protocols based on a simple pattern and analyzing programs which investigate the view mapping and amount of parallax crosstalk on the sensor on a pixel basis. The results of these developments enabled us to adjust the lenslet array with a sub micrometer precision and to mark the pixels of the sensor where the views do not register properly.

This paper presents a geometric and subjective analysis of stereoscopic versions of close range I3A clusters (Subject- Camera distances below 5 m). The geometry of the stereoscopic pipeline from the scene to the viewer's eyes is a very relevant issue in stereoscopic media. One important factor is the camera separation, because it can be used to control the perceived depth of stereoscopic images. The computational camera separations were compared to subjectively preferred camera separations. Participants evaluated the strength and naturalness of depth sensation and overall viewing experience from the still images with single-stimulus method. Results showed that participants were able to perceive the change of depth range even though the images were shown in random order without a reference depth scale. A mild depth sensation was preferred over strong depth sensations. The computational camera separation differed from the subjectively preferred camera separation when the depth range of the scene was narrow. This result indicates that scenes with narrow depth should not be imaged with a long camera separation just to fill the depth budget of the display.

Unlike the real world, stereoscopic cinemas and display devices have a bordered frame which can unnaturally cut off our view, and create conflicting visual cues. These perceptual conflicts can diminish the 3-D effect and cause visual fatigue for the audience. Presented is a method to resolve these issues, by developing a controllable articulated 3D border.

As the production of stereoscopic content increases, so does the need for post-production tools for that content. Video inpainting has become an important tool for rig removal but there has been little consideration of the problem in stereo. This paper presents an algorithm for stereo video inpainting that builds on existing exemplar-based video completion and also considers the issues of view consistency. Given user selected regions in the sequence which may be in the same location in several frames and in both views, the objective is to fill in this area using all the available picture information. Existing algorithms lack temporal consistency, causing flickering and other artefacts. This paper explores the use of long-term picture information across many frames in order to achieve temporal consistency at the same time as exploiting inter-view dependencies within the same framework.

'View plus depth' is an attractive compact representation format for 3D video compression and transmission. It combines 2D video with depth map sequence aligned in a per-pixel manner to represent the moving 3D scene in interest. Any different-perspective view can be synthesized out if this representation through Depth-Image Based Rendering (DIBR). However, such rendering is prone to disocclusion errors: regions originally covered by foreground objects become visible in the synthesized view and have to be filled with perceptually-meaningful data. In this work, a technique for reducing the perceived artifacts by inpainting the disoccluded areas is proposed. Based on Criminisi's exemplar-based inpainting algorithm, the developed technique recovers the disoccluded areas by using pixels of similar blocks surrounding it. In the original work, a moving window is centered on the boundaries between known and unknown parts ('target window'). The known pixels are used to select windows which are most similar to the target one. When this process is completed, the unknown region of the target patch is filled with a weighted combination of pixels from the selected windows. In the proposed scheme, the priority map, which defines the rule for selecting the order of pixels to be filled, has been modified to meet the requirement for disocclusion hole filling and a better non-local mean estimate has been suggested accordingly. Furthermore, the search for similar patches has also been extended to previous and following frames of the video under processing, thus improving both computational efficiency and resulting quality.

This paper presents a study on the stereoscopic 3D codec for non-real-time 3DTV services. Delivering stereoscopic 3DTV contents via broadcast networks can be based on either real-time or non-real-time scenarios. For a digital television system where its bandwidth is somewhat limited to accommodate the full 3D HD quality video, a complementary enabler is the non-real-time delivery in which 3D video component is downloaded in advance. In the case of ATSC system, MPEG-2, H.264/AVC and any other codec can be used for stereoscopic video based on Non- Real-Time services. In order to put in the best performance, the analysis on the stereoscopic codec, scenarios and preliminary considerations are required. In this paper, the NRT 3D codec scenarios and preliminary considerations are addressed to develop the prerequisites required for the analysis on NRT 3D codec. The experimentation of both independent and inter-view coding are covered to investigate the codec combination for NRT 3D video services based on considerations and scenarios.

Different compression formats for stereo and multiview based 3D video is being standardized and software players capable of decoding and presenting these formats onto different display types is a vital part in the commercialization and evolution of 3D video. However, the number of publicly available software video players capable of decoding and playing multiview 3D video is still quite limited. This paper describes the design and implementation of a GPU-based real-time 3D video playback solution, built on top of cross-platform, open source libraries for video decoding and hardware accelerated graphics. A software architecture is presented that efficiently process and presents high definition 3D video in real-time and in a flexible manner support both current 3D video formats and emerging standards. Moreover, a set of bottlenecks in the processing of 3D video content in a GPU-based real-time 3D video playback solution is identified and discussed.

Observation and interviews with people viewing autostereoscopic 3D imagery provides evidence that there are many human perception considerations required for 3D content creation. A study was undertaken whereby it was witnessed that certain test autostereoscopic imagery elicited a highly emotional response and engagement, while other test autostereoscopic imagery was given only a passing glance. That an image can be viewed with a certain level of stereopsis does not make it compelling. By taking into consideration the manner in which humans perceive depth and the space between objects, 3D content can achieve a level of familiarity and realness that is not possible with single perspective imagery. When human perception issues are ignored, 3D imagery can be undesirable to viewers and a negative bias against 3D imagery can occur. The preparation of 3D content is more important than the display technology. Where human perception, as it is used to interpret reality, is not mimicked in the creation of 3D content, the general public typically express a negative bias against that imagery (where choices are provided). For some, the viewing of 3D content that could not exist naturally, induces physical discomfort.

Many 3D formats exist and will probably co-exist for a long time even if 3D standards are today under definition. The support for multiple 3D formats will be important for bringing 3D into home. In this paper, we propose a novel and effective method to detect whether a video is a 3D video or not, and to further identify the exact 3D format. First, we present how to detect those 3D formats that encode a pair of stereo images into a single image. The proposed method detects features and establishes correspondences between features in the left and right view images, and applies the statistics from the distribution of the positional differences between corresponding features to detect the existence of a 3D format and to identify the format. Second, we present how to detect the frame sequential 3D format. In the frame sequential 3D format, the feature points are oscillating from frame to frame. Similarly, the proposed method tracks feature points over consecutive frames, computes the positional differences between features, and makes a detection decision based on whether the features are oscillating. Experiments show the effectiveness of our method.

3D film and 3D TV are becoming reality. More facilities and devices are now 3D capable. Compared to capture 3D video content directly, 2D to 3D video conversion is a low-cost, backward compatible alternate. There also exists a tremendous amount of monoscopic 2D video content that are of high interest to be displayed on 3D devices with noticeable immersiveness. 2D to 3D video conversion, therefore, has drawn lots of attention recently. In this paper, a low complexity 2D to 3D conversion algorithm is presented. The conversion generates stereo video pairs by 3D warping based on estimated per-pixel depth maps. The depth maps are estimated jointly by motion and color cues. Subjective tests show that the proposed algorithm achieves 3D perception with acceptable artifact.

The increasing popularity for stereoscopic content in the entertainment industry and computer graphics applications and the availability of affordable capture and display systems is in contrast to the actual knowledge of underlying stereoscopic design principles and fundamental concepts. Content creators and educators inexperienced in stereoscopy require integrated, easy to use and flexible tools which can assist in the process of creating the three-dimensional "look" they are after within the limits of a comfortable viewing experience. The proposed framework in this paper, a custom stereoscopic export plug-in for the popular 3D modeling application Google Sketchup and a flexible stereoscopic format conversion and display engine, allows for stereoscopic previsualisation in near real-time in a format of choice. The user interface can recommend stereoscopic settings according to the scene, camera and display properties, calculates corresponding values according to manual entries but also leaves unrestricted control over all parameters. The display engine allows for different stereoscopic formats to be shown and saves the result in form of images with metadata for reference. Particular attention is put on usability, accessibility and tight integration.

The relationship between color and lightness appearance and the perception of depth has been studied since a while in the field of perceptual psychology and psycho-physiology. It has been found that depth perception affects the final object color and lightness appearance. In the stereoscopy research field, many studies have been proposed on human physiological effects, considering e.g. geometry, motion sickness, etc., but few has been done considering lightness and color information. Goal of this paper is to realize some preliminar experiments in Virtual Reality in order to determine the effects of depth perception on object color and lightness appearance. We have created a virtual test scene with a simple 3D simultaneous contrast configuration. We have created three different versions of this scene, each with different choices of relative positions and apparent size of the objects. We have collected the perceptual responses of several users after the observation of the test scene in the Virtual Theater of the University of Milan, a VR immersive installation characterized by a semi-cylindrical screen that covers 120° of horizontal field of view from an observation distance of 3.5 m. We present a description of the experiments setup and procedure, and we discuss the obtained results.

Coarse integral volumetric imaging (CIVI) combines multiview and volumetric display solutions and presents undistorted floating 3D image by correcting distortion of volumetric image for each view. In the conventional CIVI with limited viewing angle, distortions of image planes can be approximated to be parabolic in the direction of depth, while those in horizontal and vertical directions can be ignored. When the viewing angle becomes wider, however, this approximation cannot realize presentation of undistorted image. To cope with the strong distortions, the method the authors propose calculate z-coordinate of the generated real image is in detail and depicts each pixel on the display panel of the corresponding depth. Also distortions in horizontal and vertical directions are corrected by using texture mapping. To attain precise correction in vertical, horizontal and depth directions, optical paths of light rays between the display panel and each viewpoint are calculated with an optical simulator. Color aberration can also be corrected by mapping red, green and blue textures separately based on the result of the optical simulation.

Coarse integral imaging (CII), where each elemental lens is large enough to cover pixels far more than the number of views, can show clear floating 3D image when distortion is corrected. One of the major problems left to be solved for CII is suppression of pseudo images that appear around the right image to be presented. In this paper we propose two methods to suppress pseudo images. We first propose use of a lens array with a small F number. When a lens array composed of elemental lenses whose F number is small is set in front of the display panel, pseudo images can be erased by total internal reflection on the outskirt of the large aperture lens because the angle of incidence of the light ray that generates pseudo images becomes larger. The second method we propose is use of a lens array behind the display panel paired with segmented backlight. When convex lenses are set in front of the backlight with limited aperture, leak of ray out to adjacent elemental lenses can be avoided. Since the backlight area is reduced, this method can also reduce consumption of electric power without diminishing brightness of the right image.