This PDF file contains the front matter associated with SPIE Proceedings Volume 7524, including the Title Page, Copyright information, Table of Contents, Introduction, and Conference Committee listing.

Introduction: Many operative specialties rely on the use of microscopes and endoscopes for visualizing operative fields in minimally invasive or microsurgical interventions. Conventional optical devices present relevant details only to the surgeon or one assisting professional. Advances in information technology have propagated stereoscopic visual information to all operative theatre personnel in real time, which, in turn, adds to the load of complex technical devices to be handled and maintained in operative theatres. Material and methods: In the last six years, we have been using conventional (SD, 720 x 576 pixels) and high definition (1280 x 720 pixels) stereoscopic video cameras attached to conventional operative microscopes either in parallel to direct visualization of the operative field or as all-digital processing of the operative image for the surgeon including all other staff. Aspects included the type of display used, image quality, time delay due to image processing, visual comfort, time consumption in set-up, ease of use as well as robustness of the system. Results: General acceptance of stereoscopic display technology is high as all staff members are able to share the same visual information. The use of stereo cameras in parallel to direct visualization or as only visualization device mostly depended on image quality and personal preference of the surgeon. Predominant general factors are robustness, ease of use and additional time consumption imposed by setup and handling. Visual comfort was noted as moderately important as there was wide variability between staff members. Type of display used and post-processing issues were regarded less important. Time delay induced by the video chain was negligible. Conclusion: The additional information given by stereoscopic video processing in real time outweighs the extra effort for handling and maintenance. However, further integration with existing technology and with the general workflow enhances acceptance especially in units with high turnover of operative procedures.

The DLR MIRO is the second generation of versatile robot arms for surgical applications, developed at the Institute for Robotics and Mechatronics at Deutsche Zentrum für Luft- und Raumfahrt (DLR) in Oberpfaffenhofen, Germany. With its low weight of 10 kg and dimensions similar to those of the human arm, the MIRO robot can assist the surgeon directly at the operating table where space is scarce. The planned scope of applications of this robot arm ranges from guiding a laser unit for the precise separation of bone tissue in orthopedics to positioning holes for bone screws, robot assisted endoscope guidance and on to the multi-robot concept for endoscopic minimally invasive surgery. A stereo-endoscope delivers two full HD video streams that can even be augmented with information, e.g vectors indicating the forces that act on the surgical tool at any given moment. SeeFront's new autostereoscopic 3D display SF 2223, being a part of the MIRO assembly, will let the surgeon view the stereo video stream in excellent quality, in real time and without the need for any viewing aids. The presentation is meant to provide an insight into the principles at the basis of the SeeFront 3D technology and how they allow the creation of autostereoscopic display solutions ranging from smallest "stamp-sized" displays to 30" desktop versions, which all provide comfortable freedom of movement for the viewer along with excellent 3D image quality.

We applied an autostereoscopic display based on the integral imaging method (II method) to training equipment for medical treatment in an attempt to recover the binocular vision performance of strabismus or amblyopia (lazy eye) patients. This report summarizes the application method and results. The point of the training is to recognize the parallax using both eyes. The strabismus or amblyopia patients have to recognize the information on both eyes equally when they gaze at the display with parallax and perceive the stereo depth of the content. Participants in this interactive training engage actively with the image. As a result, they are able to revive their binocular visual function while playing a game. Through the training, the observers became able to recognize the amount of parallax correctly. In addition, the training level can be changed according to the eyesight difference between a right eye and a left eye. As a result, we ascertained that practical application of the II method for strabismus or amblyopia patients would be possible.

In September 2009 the Fort Leonard Wood Field Element of the US Army Research Laboratory - Human Research and Engineering Directorate, in conjunction with Polaris Sensor Technologies and Concurrent Technologies Corporation, evaluated the objective performance benefits of Polaris' 3D vision upgrade kit for the TALON small unmanned ground vehicle (SUGV). This upgrade kit is a field-upgradable set of two stereo-cameras and a flat panel display, using only standard hardware, data and electrical connections existing on the TALON robot. Using both the 3D vision system and a standard 2D camera and display, ten active-duty Army Soldiers completed seven scenarios designed to be representative of missions performed by military SUGV operators. Mission time savings (6.5% to 32%) were found for six of the seven scenarios when using the 3D vision system. Operators were not only able to complete tasks quicker but, for six of seven scenarios, made fewer mistakes in their task execution. Subjective Soldier feedback was overwhelmingly in support of pursuing 3D vision systems, such as the one evaluated, for fielding to combat units.

Non-contact anterior cruciate ligament (ACL) injuries are serious and debilitating, often resulting from the performance of evasive sides-stepping (Ssg) by team sport athletes. Previous laboratory based investigations of evasive Ssg have used generic visual stimuli to simulate realistic time and space constraints that athletes experience in the preparation and execution of the manoeuvre. However, the use of unrealistic visual stimuli to impose these constraints may not be accurately identifying the relationship between the perceptual demands and ACL loading during Ssg in actual game environments. We propose that stereoscopically filmed footage featuring sport specific opposing defender/s simulating a tackle on the viewer, when used as visual stimuli, could improve the ecological validity of laboratory based investigations of evasive Ssg. Due to the need for precision and not just the experience of viewing depth in these scenarios, a rigorous filming process built on key geometric considerations and equipment development to enable a separation of 6.5 cm between two commodity cameras had to be undertaken. Within safety limits, this could be an invaluable tool in enabling more accurate investigations of the associations between evasive Ssg and ACL injury risk.

We are engaged in international standardization activities for 3D displays. We consider that for a sound development of 3D displays' market, the standards should be based on not only mechanism of 3D displays, but also human factors for stereopsis. However, we think that there is no common understanding on what the 3D display should be and that the situation makes developing the standards difficult. In this paper, to understand the mechanism and human factors, we focus on a double image, which occurs in some conditions on an autostereoscopic display. Although the double image is generally considered as an unwanted effect, we consider that whether the double image is unwanted or not depends on the situation and that there are some allowable double images. We tried to classify the double images into the unwanted and the allowable in terms of the display mechanism and visual ergonomics for stereopsis. The issues associated with the double image are closely related to performance characteristics for the autostereoscopic display. We also propose performance characteristics, measurement and analysis methods to represent interocular crosstalk and motion parallax.

The Human Visual System is extremely powerful, but has characteristics that we can exploit to efficiently store and distribute 3D information. Similar to the basis for chroma subsampling, we can reduce the amount of video data needed for storage and distribution by taking advantage of our body's own limitations. In the late 1980's, researchers at the New York Institute of Technology analyzed how humans perceive video detail and determined that the human visual system does not detect fine detail equally in all directions. This paper will discuss this early research and outline how similar techniques can be used for the storage and distribution of 3D Stereoscopic video. Advantages and disadvantages compared to the other 3D formats are presented and tools that enable individual testing of this technique will be identified and discussed.

Common autostereoscopic 3D displays are based on multi-view projection. The diversity of resolutions and number of views of such displays implies a necessary flexibility of 3D content formats in order to make broadcasting efficient. Furthermore, distribution of content over a heterogeneous network should adapt to an available network capacity. Present scalable video coding provides the ability to adapt to network conditions; it allows for quality, temporal and spatial scaling of 2D video. Scalability for 3D data extends this list to the depth and the view domains. We have introduced scalability with respect to depth information. Our proposed scheme is based on the multi-view-plus-depth format where the center view data are preserved, and side views are extracted in enhancement layers depending on depth values. We investigate the performance of various layer assignment strategies: number of layers, and distribution of layers in depth, either based on equal number of pixels or histogram characteristics. We further consider the consequences to variable distortion due to encoder parameters. The results are evaluated considering their overall distortion verses bit rate, distortion per enhancement layer, as well as visual quality appearance. Scalability with respect to depth (and views) allows for an increased number of quality steps; the cost is a slight increase of required capacity for the whole sequence. The main advantage is, however, an improved quality for objects close to the viewer, even if overall quality is worse.

From the earliest stages of the Beauty and the Beast 3D conversion project, the advantages of accurate desk-side 3D viewing was evident. While designing and testing the 2D to 3D conversion process, the engineering team at Walt Disney Animation Studios proposed a 3D viewing configuration that not only allowed artists to "compose" stereoscopic 3D but also improved efficiency by allowing artists to instantly detect which image features were essential to the stereoscopic appeal of a shot and which features had minimal or even negative impact. At a time when few commercial 3D monitors were available and few software packages provided 3D desk-side output, the team designed their own prototype devices and collaborated with vendors to create a "3D composing" workstation. This paper outlines the display technologies explored, final choices made for Beauty and the Beast 3D, wish-lists for future development and a few rules of thumb for composing compelling 2D to 3D conversions.

Context: Stereoscopic 3D movies are gaining rapid acceptance commercially. In addition our previous experience with the short 3D movie "Cosmic Cookery" showed that there is great public interest in the presentation of cosmology research using this medium. Objective: The objective of the work reported in this paper was to create a three-dimensional stereoscopic movie describing the life of the Milky way galaxy. This was a technical and artistic exercise to take observed and simulated data from leading scientists and produce a short (six minute) movie that describes how the Milky Way was created and what happens in its future. The initial target audience was the visitors to the Royal Society's 2009 Summer Science Exhibition in central London, UK. The movie is also intended to become a presentation tool for scientists and educators following the exhibition. Apparatus: The presentation and playback systems used consisted of off-the shelf devices and software. The display platform for the Royal Society presentation was a RealD LP Pro switch used with a DLP projector to rear project a 4 metre diagonal image. The LP Pro enables the use of cheap disposable linearly polarising glasses so that the high turnover rate of the audience (every ten minutes at peak times) could be sustained without needing delays to clean the glasses. The playback system was a high speed PC with an external 8Tb RAID driving the projectors at 30Hz per eye, the Lightspeed DepthQ software was used to decode and generate the video stream. Results: A wide range of tools were used to render the image sequences, ranging from commercial to custom software. Each tool was able to produce a stream of 1080p images in stereo at 30fps. None of the rendering tools used allowed precise calibration of the stereo effect at render time and therefore all sequences were tuned extensively in a trial and error process until the stereo effect was acceptable and supported a comfortable viewing experience. Conclusion: We conclude that it is feasible to produce high quality 3D movies using off-the shelf equipment if care is taken to control the stereoscopic quality throughout the production process.

While there have been numerous studies concerning human perception in stereoscopic environments, rules of thumb for cinematography in stereoscopy have not yet been well-established. To that aim, we present experiments and results of subject testing in a stereoscopic environment, similar to that of a theater (i.e. large flat screen without head-tracking). In particular we wish to empirically identify thresholds at which different types of backgrounds, referred to in the computer animation industry as matte paintings, can be used while still maintaining the illusion of seamless perspective and depth for a particular scene and camera shot. In monoscopic synthetic imagery, any type of matte painting that maintains proper perspective lines, depth cues, and coherent lighting and textures saves in production costs while still maintaining the illusion of an alternate cinematic reality. However, in stereoscopic synthetic imagery, a 2D matte painting that worked in monoscopy may fail to provide the intended illusion of depth because the viewer has added depth information provided by stereopsis. We intend to observe two stereoscopic perceptual thresholds in this study which will provide practical guidelines indicating when to use each of three types of matte paintings. We ran subject tests in two virtual testing environments, each with varying conditions. Data were collected showing how the choices of the users matched the correct response, and the resulting perceptual threshold patterns are discussed below.

We measured the eye movements of participants who watched a 6-minute movie in stereoscopic and non-stereoscopic form. We analyzed four shots of the movie. The results indicate that in a 2D movie viewers tended to look at the actors, as most of the eye movements were clustered there. The significance of the actors started at the beginning of a shot, as the eyes of the viewer focused almost immediately to them. In S3D movie the eye movement patterns were more widely distributed to other targets. For example, complex stereoscopic structures and objects nearer than the actor captured the interest and eye movements of the participants. Also, the tendency to first look at the actors was diminished in the S3D shots. The results suggests that in a S3D movie there are more eye movements which are directed to wider array of objects than in a 2D movie.

Nowadays stereoscopic technology is being paid attention as a leading technology for the next generation film industry in many countries including Korea. In Korean stereoscopic film production, however, the quality but also the quantity of stereoscopic contents still leaves much to be desired, and know-how and skill of stereoscopic film production has been elevated in tardy progress. This paper shows a research on the correlation between stereoscopic cinematography and storytelling. Based on a casestudy of a documentary film about Ho Quyen, UNESCO World Heritage in Vietnam, we could deliver guidelines for the stereoscopic film production and storytelling. For this study, we analyzed scenes and shots of a documentary film script in pre-production stage. These analysis results were reflected on a storyboard. A stereographer grasped the idea of a storytelling that a director had meant through a script and storyboard. Then he applied suitable parameters for a stereoscopic cinematography to every shot with a beamsplitter rig. A researcher wrote major parameters like interaxial distance, convergence angle in every shot. Then average parameter values of scenes were calculated from the parameter database, and the relationship between stereoscopic cinematography and storytelling was derived by shot-by-shot analysis.

This paper breaks down the R&D and production challenges faced on a stereoscopic CG short film that was developed on a six month time budget. Using this film as an example, the manner in which technical innovation and story telling techniques can be used to emphasize the strengths and hide the weaknesses of a limited timeframe student production is discussed.

Interactive free-viewpoint selection applied to a 3D multi-view signal is a possible attractive feature of the rapidly developing 3D TV media. This paper explores a new rendering algorithm that computes a free-viewpoint based on depth image warping between two reference views from existing cameras. We have developed three quality enhancing techniques that specifically aim at solving the major artifacts. First, resampling artifacts are filled in by a combination of median filtering and inverse warping. Second, contour artifacts are processed while omitting warping of edges at high discontinuities. Third, we employ a depth signal for more accurate disocclusion inpainting. We obtain an average PSNR gain of 3 dB and 4.5 dB for the 'Breakdancers' and 'Ballet' sequences, respectively, compared to recently published results. While experimenting with synthetic data, we observe that the rendering quality is highly dependent on the complexity of the scene. Moreover, experiments are performed using compressed video from surrounding cameras. The overall system quality is dominated by the rendering quality and not by coding.

In this paper we address the estimation of scene range for multiview 3DTV applications. Based on their similar setup requirements, we propose a depth camera system combining the strengths of Structured Light and Active Stereo techniques to obtain a fast depth map recovery for a larger variety of scenes and from a wider view with less occlusion. The proposed system is composed by a stereo rig and a projector. A stripe-based one shot pattern is projected into the target scene. Using an efficient decoding technique, reliable correspondences can be found between each camera and the projected pattern. Decoded areas common to both cameras are triangulated and used to calibrate the projector on the fly in order to estimate depth in decoded areas only visible in one camera. Correspondences in undecoded areas are estimated via a stereo matching procedure. The different estimated data is then combined in a single depth map. We introduce a 3 projection method that improves the structured light identification in strong illumination conditions and more robust to color objects surfaces of the scene. We demonstrate the efficacy of the integration method experimental results with special emphasis on its performance in the context of the development and contents creation for depth image-based representation (DIBR) 3DTV.

The availability of multi-view images of a scene makes possible new and exciting applications, including Free-viewpoint TV (FTV). FTV allows us to change viewpoint freely in a 3D world, where the virtual viewpoint images are synthesized by Depth-Image-Based Rendering (DIBR). In this paper, we propose a new method of DIBR using multi-view images acquired in a linear camera arrangement. The proposed method improves virtual viewpoint images by predicting the residual errors. For virtual viewpoint image synthesis, it is necessary to estimate the depth maps with multi-view images. Some algorithms to estimate depth map were proposed, but it is difficult to estimate accurate depth map. As a result, rendered virtual viewpoint images have some errors due to the depth errors. Therefore, our proposed method takes into account those depth errors and improves the quality of the rendered virtual viewpoint images. In the proposed method, the virtual images of each camera position are generated using the real images from each other camera. Then, the residual errors can be calculated between the generated images and the real images acquired by the actual cameras. The residual errors are processed and fed back to predict the residual errors that can be happened to virtual viewpoint images generated by conventional method. In the experiments, PSNR could be improved for few decibels compared with the conventional method.

We present a general framework for the modeling and optimization of scalable large format 3-D displays using multiple projectors. Based on this framework, we derive algorithms that can robustly optimize the visual quality of an arbitrary combination of projectors (e.g. tiled, superimposed, combinations of the two) without manual adjustment. The framework creates for the first time a new unified paradigm that is agnostic to a particular configuration of projectors yet robustly optimizes for the brightness, contrast, and resolution of that configuration. In addition, we demonstrate that our algorithms support high resolution stereoscopic video at real-time interactive frame rates achieved on commodity graphics hardware. Through complementary polarization, the framework creates high quality multi-projector 3-D displays at low hardware and operational cost for a variety of applications including digital cinema, visualization, and command-and-control walls.

We describe a new iteration of the StereoJet process, which has been simplified by changes in materials and improved by the conversion from linear to circular polarization. A prototype StereoJet process for producing full color stereoscopic images, described several years ago by Scarpetti et al., was developed at the Rowland Institute for Science, now part of Harvard University. The system was based on the inkjet application of inks comprising dichroic dyes to Polaroid Vectograph sheet, a concept explored earlier by Walworth and Chiulli at the Polaroid Research Laboratories. Vectograph sheet comprised two oppositely oriented layers of stretched polyvinyl alcohol (PVA) laminated to opposite surfaces of a cellulose triacetate support sheet. The two PVA layers were oriented at +45 and -45 degrees, respectively, with respect to the running edge of the support sheet. A left-eye and right-eye stereoscopic image pair were printed sequentially on the respective surfaces, and the resulting stereoscopic image viewed with conventional linearly polarized glasses having +45 and -45 degree orientation. StereoJet, Inc. has developed new, simplified technology based on the use of PVA substrate of the type used in sheet polarizer manufacture with orientation parallel to the running edge of the support. Left- and right-eye images are printed at 0 and 90 degrees, then laminated in register. Addition of a thin layer of 1/4-wave retarder to the front surface converts the image pair's respective orientations to right- and left-circular polarization. The full color stereoscopic images are viewed with circularly polarized glasses.

A volumetric display system using a roof mirror grid array (RMGA) is proposed. The RMGA consists of a two-dimensional array of dihedral corner reflectors and forms a real image at a plane-symmetric position. A two-dimensional image formed with a RMGA is moved at thigh speed by a mirror scanner. Cross-sectional images of a three-dimensional object are displayed in accordance with the position of the image plane. A volumetric image can be observed as a stack of the cross-sectional images by high-speed scanning. Image formation by a RMGA is free from aberrations. Moreover, a compact optical system can be constructed because a RMGA doesn't have a focal length. An experimental volumetric display system using a galvanometer mirror and a digital micromirror device was constructed. The formation of a three-dimensional image consisting of 1024 × 768 × 400 voxels is confirmed by the experimental system.

We propose 2D/3D convertible display system with enhanced 3D viewing angle based on integral imaging. We use 3 LCD devices and a lens array in the proposed system. The LCD device 1is used for displaying an elemental image which turn into a 3D image through the lens array. The LCD device 2 is used as a polarization switcher to control the transparent and the opaque region of elemental image. The LCD device 3 is used as both mask pattern and 2D image displaying device. On the back glass of LCD device 3, we attach polarization-selective scattering film which makes LCD device 3 used as a mask pattern and a 2D display device at the same time. Behind of the LCD device 3, the lens array is located for displaying 3D image by the integral imaging method. The whole system is as compact as conventional integral imaging system. With the LCD device 2 and 3, the viewing angle of 3D image can be enhanced both vertically and horizontally according to the arrangement of elemental image. We prove the feasibility of proposed system by basic experiments.

Reproducing the light field of a 3D scene on a flat-panel display is an ultimate goal of autostereoscopic imaging. Currently, solutions using microlenses or parallax barriers are the most common for this purpose. However, while a light field is a 4D vector space, a flat-panel display is only a 2D entity. There is a fundamental mismatch between the device's capacity and the amount of information to be displayed. To address this problem, we consider a multilayer liquid crystal display (LCD), which is a display device constructed by stacking multiple LCDs on top of a light source. Since the total number of pixels in a multilayer LCD is larger than that of a single layer LCD, a multilayer LCD can store more information than a single layer LCD. Moreover, a multilayer LCD will exhibit varying images depending on the viewers' positions. This property is found to be useful for autostereoscopic 3D viewing, which is elaborated upon further in detail through simulation-based studies.

The Anaglyph 3D method of stereoscopic visualization is both cost effective and compatible with all full-color displays, however this method often suffers from poor 3D image quality due to poor color quality and ghosting (whereby each eye sees a small portion of the perspective image intended for the other eye). Ghosting, also known as crosstalk, limits the ability of the brain to successfully fuse the images perceived by each eye and thus reduces the perceived quality of the 3D image. This paper describes a research project which has simulated the spectral performance of a wide selection of anaglyph 3D glasses on CRT, LCD and plasma displays in order to predict ghosting levels. This analysis has included for the first time a comparison of crosstalk between different color-primary types of anaglyph glasses - green/magenta and blue/yellow as well as the more traditional red/cyan. Sixteen pairs of anaglyph 3D glasses were simulated (6 pairs of red/cyan glasses, 6 pairs of blue/yellow glasses and 4 pairs of green/magenta glasses). The spectral emission results for 13 LCDs, 15 plasma displays and one CRT Monitor were used for the analysis. A custom written Matlab program was developed to calculate the amount of crosstalk for all the combinations of different displays with different anaglyph glasses.

In this paper we propose a method to characterize polarization based stereoscopic 3D displays using multispectral Fourier optics viewing angle measurements. Full polarization analysis of the light emitted by the display in the full viewing cone is made at 31 wavelengths in the visible range. Vertical modulation of the polarization state is observed and explained by the position of the phase shift filter into the display structure. In addition, strong spectral dependence of the ellipticity and polarization degree is observed. These features come from the strong spectral dependence of the phase shift film and introduce some imperfections (color shifts and reduced contrast). Using the measured transmission properties of the two glasses filters, the resulting luminance across each filter is computed for left and right eye views. Monocular contrast for each eye and binocular contrasts are performed in the observer space, and Qualified Monocular and Binocular Viewing Spaces (QMVS and QBVS) can be deduced in the same way as auto-stereoscopic 3D displays allowing direct comparison of the performances.

A new computation technique is presented for calculating pixel colors in anaglyph images. The method depends upon knowing the RGB spectral distributions of the display device and the transmission functions of the filters in the viewing glasses. It requires the solution of a nonlinear least-squares program for each pixel in a stereo pair and is based on minimizing color distances in the CIEL*a*b* uniform color space. The method is compared with several techniques for computing anaglyphs including approximation in CIE space using the Euclidean and Uniform metrics, the Photoshop method and its variants, and a method proposed by Peter Wimmer. We also discuss the methods of desaturation and gamma correction for reducing retinal rivalry.

Display of stereo images is widely used to enhance the viewing experience of three-dimensional imaging and communication systems. In this paper, we propose a method for estimating the quality of stereoscopic images using segmented image features and disparity. This method is inspired by the human visual system. We believe the perceived distortion and disparity of any stereoscopic display is strongly dependent on local features, such as edge (non-plane) and non-edge (plane) areas. Therefore, a no-reference perceptual quality assessment is developed for JPEG coded stereoscopic images based on segmented local features of artifacts and disparity. Local feature information such as edge and non-edge area based relative disparity estimation, as well as the blockiness and the blur within the block of images are evaluated in this method. Two subjective stereo image databases are used to evaluate the performance of our method. The subjective experiments results indicate our model has sufficient prediction performance.

The ghost image induced by System-Crosstalk (SCT) of 3D display, due to optical hardware imperfections, is the major factor to jeopardize stereopsis. The system crosstalk can be measured by optical measuring instrument and describes the optical leakage from the neighboring viewing zones. The amount of crosstalk reduces the ability of the viewer to fuse the stereo-images into 3D images. The Viewer-Crosstalk (VCT), combined with hardware and content issues, is an overall evaluation of the ghost image and can be easily interpreted based on the principle of binocular 3D display. The examination of different SCT values was carried out with a seven-grade subjective evaluation test. In our previous study, it was shown that many other factors, such as contrast ratio, disparity and monocular cues of the images, play important roles in the stereopsis. In this paper, we study the factors of stereo-images with different crosstalk levels that may affect stereopsis. For simulate the interference between stereo-images, digital image processing are employed to assign different levels of crosstalk to each other at properly specified disparity between images. Results of this research can provide valuable reference to the content makers and for the optimized design of 3D displays with minimum System Crosstalk.

3D processing techniques are really promising. However, several hurdles have to be overcome. In this paper, two of them are examined. The first is related to the high disparity management. It is currently not well mastered and its impact is strong for viewing 3D scene on stereoscopic screens. The second concerns the salient regions of the scene. These areas are commonly called Region-Of-Interest (RoI) in the image processing domain. The problem appears when there are more than one region-of-interest in a video scene. Indeed, it is then complicated for the eyes to scan them and especially if the depth difference between them is high. In this contribution, the 3D experience is improved by applying some effects related to RoIs. The shift between the two views is adaptively adjusted in order to have a null disparity on a given area in the scene. In the proposed approach, these areas are the visually interesting areas. A constant disparity on the salient areas improves the viewing experience over the video sequence.

The interest in the production of stereoscopic contents is growing rapidly. Stereo material can be produced using different solutions, from high level devices to standard digital cameras suitably coupled. In the latter case, color correction in stereoscopic images is complex, due to possible different Color Filter Arrays or settings in the two acquisition devices: users must often tune each camera separately, and this can lead to visible color inter-differences in the stereo pair. The color correction methods often considered in the post-processing stage of stereoscopic production are mainly based on global transformations between the two views, but this approach can not completely recover relevant limits in the gamuts of each image due to color distortions. In this paper we evaluate the application of perceptually-based spatial color computational models, based or inspired by Retinex theory, to pre-filter the stereo pairs. Spatial color algorithms apply an unsupervised local color correction to each pixel, based on a simulation of color perception mechanisms, and were proven to effectively reduce color dominants and adjust local contrasts in images. We filtered different stereoscopic streams with visible color differences between right and left frames, using a GPU version of the Random Spray Retinex (RSR) algorithm, that applies in few seconds an unsupervised color correction, and the Automatic Color Equalization (ACE) algorithm, that considers both White Patch and Gray World equalization mechanisms. We analyse the effect of the computational models both by visual assessment and by considering the changes in the image gamuts before and after the filtering.

The use of tele-operated Unmanned Ground Vehicles (UGVs) for military uses has grown significantly in recent years with operations in both Iraq and Afghanistan. In both cases the safety of the Soldier or technician performing the mission is improved by the large standoff distances afforded by the use of the UGV, but the full performance capability of the robotic system is not utilized due to insufficient depth perception provided by the standard two dimensional video system, causing the operator to slow the mission to ensure the safety of the UGV given the uncertainty of the perceived scene using 2D. To address this Polaris Sensor Technologies has developed, in a series of developments funded by the Leonard Wood Institute at Ft. Leonard Wood, MO, a prototype Stereo Vision Upgrade (SVU) Kit for the Foster-Miller TALON IV robot which provides the operator with improved depth perception and situational awareness, allowing for shorter mission times and higher success rates. Because there are multiple 2D cameras being replaced by stereo camera systems in the SVU Kit, and because the needs of the camera systems for each phase of a mission vary, there are a number of tradeoffs and design choices that must be made in developing such a system for robotic tele-operation. Additionally, human factors design criteria drive optical parameters of the camera systems which must be matched to the display system being used. The problem space for such an upgrade kit will be defined, and the choices made in the development of this particular SVU Kit will be discussed.

Today, 3D viewing devices still need qualitative content; up to now, there is no real 3D video shooting system specifically designed to ensure a qualitative 3D experience on some pre-chosen 3D display. A fundamental element of multiscopic image production is the geometrical analysis of shooting and viewing conditions in order to obtain a qualitative 3D perception experience. Many autostereoscopic camera systems are proposed but none is designed with control of possible depth distortions in mind. This article introduces a patented autostereoscopic camera design scheme based upon this distortion control. Thanks to our scientific know-how, we based our work on the link between the shooting and rendering geometries, which enables to control the distortion of the perceived depth. Thus this design scheme provides camera systems producing qualitative 3D content complying with any pre-chosen distortion when rendered on any specific autostereoscopic display. Thanks to our technological expertise, we use this design scheme to product pre-industrial camera systems devoted to 3D live or pre-recorded shooting. These systems are compact, lightweight, easy to deploy and rather adaptable to other conditions (3D displays, depth distortion). We will introduce the associated software, which allows to control our 3D cameras and to display in real-time on the autostereoscopic display formerly specified. According to numerous spectators, both naive and expert, the 3D perception is really qualitative.

Autostereoscopic multiview 3D displays have been available for number of years, capable of producing a perception of depth in a 3D image without requiring user-worn glasses. Different approaches to compress these 3D images exist. Two compression schemes, and how they affect the 3D image with respect to induced distortion, is investigated in this paper: JPEG 2000 and H.264/AVC. The investigation is conducted in three parts: objective measurement, qualitative subjective evaluation, and a quantitative user test. The objective measurement shows that the Rate-Distortion (RD) characteristic of the two compression schemes differ in character as well as in level of PSNR. The qualitative evaluation is performed at bitrates where the two schemes have the same RD fraction and a number of distortion characteristics are found to be significantly different. However, the quantitative evaluation, performed using 14 non-expert viewers, indicates that the different distortion types do not significantly contribute to the overall perceived 3D quality. The used bitrate, and the content of the original 3D image, is the two factors that most significantly affect the perceived 3D image quality. In addition, the evaluation results suggest that viewers prefer less apparent depth and motion parallax when being exposed to compressed 3D images on an autostereoscopic multiview display.

Virtual views in 3D-TV and multi-view video systems are reconstructed images of the scene generated synthetically from the original views. In this paper, we analyze the performance of streaming virtual views over IP-networks with a limited and time-varying available bandwidth. We show that the average video quality perceived by the user can be improved with an adaptive streaming strategy aiming at maximizing the average video quality. Our adaptive 3D multi-view streaming can provide a quality improvement of 2 dB on the average - over non-adaptive streaming. We demonstrate that an optimized virtual view adaptation algorithm needs to be view-dependent and achieve an improvement of up to 0.7 dB. We analyze our adaptation strategies under dynamic available bandwidth in the network.

This paper proposes an electronic version of coarse integral volumetric imaging (CIVI) display with wide viewing angle. CIVI is a 3D display solution which combines multiview techniques based on integral imaging with volumetric techniques using multilayer panels. Though CIVI has solved most of the major problems of conventional 3D displays, it still has two shortcomings to be overcome. One is the difficulty in realizing electronic display due to unavailability of electronic color display panels transparent enough to be layered for volumetric imaging. The other is the limited viewing angle because of the aberration of lenses. As for the former problem the simplest way to attain electronic version of CIVI is to use half mirrors to merge multiple images from different depths. Though high quality 3D image can be attained with this method, the system size becomes large. To avoid bulky mirror system and realize compact system size, the authors propose layered use of a color panel and multiple monochrome panels to emulate color volumetric display. To expand viewing angle, the authors propose a display system where smaller CIVI display components, each of which has little aberration, are connected so that each display plane faces toward the center of the image optically generated.

This paper proposes a novel 2D-to-3D conversion system based on visual attention analysis. The system was able to generate stereoscopic video from monocular video in a robust manner with no human intervention. According to our experiment, visual attention information can be used to provide rich 3D experience even when depth cues from monocular view are not enough. Using the algorithm introduced in the paper, 3D display users can watch 2D media in 3D. In addition, the algorithm can be embedded into 3D displays in order to deliver better viewing experience with more immersive feeling. Using visual attention information to give a 3D effect is first tried in this research as far as we know.

This paper explores a novel metric which can check the consistency and correctness of a disparity map, and hence validate an interpolated view (or video frame for motion compensated frame interpolation) from the estimated correspondences between two or more input views. The proposed reprojection error metric (REM) is shown to be sufficient for the regions where the observed 3D scene has no occlusions. The metric is completely automatic requiring no human input. We also explain how the metric can be extended to be useful for 3D scenes (or videos) with occlusions. However, the proposed metric does not satisfy necessary conditions. We discuss the issues which arise during the design of a necessary metric, and argue that necessary metrics which work in finite time cannot be designed for checking the validity of a method which performs disparity estimation.

A vastly growing number of productions from the entertainment industry are aiming at 3D movie theaters. These productions use a two-view format, primarily intended for eye-wear assisted viewing in a well defined environment. To get this 3D content into the home environment, where a large variety of 3D viewing conditions exists (e.g. different display sizes, display types, viewing distances), we need a flexible 3D format that can adjust the depth effect. This can be provided by the image plus depth format, in which a video frame is enriched with depth information for all pixels in the video frame. This format can be extended with additional layers, such as an occlusion layer or a transparency layer. The occlusion layer contains information on the data that is behind objects, and is also referred to as occluded video. The transparency layer, on the other hand, contains information on the opacity of the foreground layer. This allows rendering of semi-transparencies such as haze, smoke, windows, etc., as well as transitions from foreground to background. These additional layers are only beneficial if the quality of the depth information is high. High quality depth information can currently only be achieved with user assistance. In this paper, we discuss an interactive method for depth map enhancement that allows adjustments during the propagation over time. Furthermore, we will elaborate on the automatic generation of the transparency layer, using the depth maps generated with an interactive depth map generation tool.

When an object is closer to an observer than the background, the small differences between right and left eye views are interpreted by the human brain as depth. This basic ability of the human visual system, called stereopsis, lies at the core of all binocular three-dimensional (3-D) perception and related technological display development. To achieve stereopsis, it is traditionally assumed that corresponding locations in the right and left eye's views must first be matched, then the relative differences between right and left eye locations are used to calculate depth. But this is not the whole story. At every object-background boundary, there are regions of the background that only one eye can see because, in the other eye's view, the foreground object occludes that region of background. Such monocular zones do not have a corresponding match in the other eye's view and can thus cause problems for depth extraction algorithms. In this paper I will discuss evidence, from our knowledge of human visual perception, illustrating that monocular zones do not pose problems for our human visual systems, rather, our visual systems can extract depth from such zones. I review the relevant human perception literature in this area, and show some recent data aimed at quantifying the perception of depth from monocular zones. The paper finishes with a discussion of the potential importance of considering monocular zones, for stereo display technology and depth compression algorithms.

Viewing 3D content on an autostereoscopic is an exciting experience. This is partly due to the fact that the 3D effect is seen without glasses. Nevertheless, it is an unnatural condition for the eyes as the depth effect is created by the disparity of the left and the right view on a flat screen instead of having a real object at the corresponding location. Thus, it may be more tiring to watch 3D than 2D. This question is investigated in this contribution by a subjective experiment. A search task experiment is conducted and the behavior of the participants is recorded with an eyetracker. Several indicators both for low level perception as well as for the task performance itself are evaluated. In addition two optometric tests are performed. A verification session with conventional 2D viewing is included. The results are discussed in detail and it can be concluded that the 3D viewing does not have a negative impact on the task performance used in the experiment.

Conventional stereoscopic displays present images at a fixed focal distance. Depth variations in the depicted scene therefore result in conflicts between the stimuli to vergence and to accommodation. The resulting decoupling of accommodation and vergence responses can cause adverse consequences, including reduced stereo performance, difficulty fusing binocular images, and fatigue and discomfort. These problems could be eliminated if stereo displays could present correct focus cues. A promising approach to achieving this is to present each eye with a sum of images presented at multiple focal planes, and to approximate continuous variations in focal distance by distributing light energy across image planes - a technique referred to as depth-filtering¹. Here we describe a novel multi-plane display in which we can measure accommodation and vergence responses. We report an experiment in which we compare these oculomotor responses to real stimuli and depth-filtered simulations of the same distance. Vergence responses were generally similar across conditions. Accommodation responses to depth-filtered images were inaccurate, however, showing an overshoot of the target, particularly in response to a small step-change in stimulus distance. This is surprising because we have previously shown that blur-driven accommodation to the same stimuli, viewed monocularly, is accurate and reliable. We speculate that an initial convergence-driven accommodation response, in combination with a weaker accommodative stimulus from depth-filtered images, leads to this overshoot. Our results suggest that stereoscopic multi-plane displays can be effective, but require smaller image-plane separations than monocular accommodation responses suggest.

In 3-D movies and virtual reality and augmented reality systems, stereoscopic images are used to improve perceptions of realism and depth. The distance (depth) that we perceive can be classified into absolute distance, which is the distance between the observer and the objects, and relative distance, which is the distance between the objects. It is known that in the real environment these two distances are independent. Previous studies have reported that we underestimate or overestimate the absolute distance in stereoscopic images under some circumstances. We examined perceptions of both absolute and relative distances in stereoscopic images to investigate the depth perception of virtual objects. The results of the experiments showed that (1) the perceived absolute distance from the observer to images in front of the screen was nearly accurate and that to images farther behind the screen was underestimated; (2) this underestimation tendency increased in short viewing distances; and (3) the relative distance from the screen to images in front of the screen was overestimated, whereas that to images farther behind the screen was underestimated.

We present a general concept of the proposed 3D imaging system called 3D-geometric camera (3D-gCam) to pick up pixel-wise 3D surface profile information along with color information. The 3D-gCam system includes two isotropic light sources placed at different geometric locations, an optical alignment system for aligning the light rays projected onto the scene, and a high precision camera. To determine the pixel-wise distance information, the system captures two images of the same scene during the stropping of the each light source. Then, the intensity of each pixel location in these two images along with the displacement between the light sources are utilized for calculating the distance information of object points corresponding to pixel locations in the image to generate a dense 3D point cloud. The approach is suitable for capturing of 3D and color information in high definition image format synchronously.

One of the critical issues to successful service of 3D video is how to compress huge amount of multi-view video data efficiently. In this paper, we described about geometric prediction structure for multi-view video coding. By exploiting the geometric relations between each camera pose, we can make prediction pair which maximizes the spatial correlation of each view. To analyze the relationship of each camera pose, we defined the mathematical view center and view distance in 3D space. We calculated virtual center pose by getting mean rotation matrix and mean translation vector. We proposed an algorithm for establishing the geometric prediction structure based on view center and view distance. Using this prediction structure, inter-view prediction is performed to camera pair of maximum spatial correlation. In our prediction structure, we also considered the scalability in coding and transmitting the multi-view videos. Experiments are done using JMVC (Joint Multiview Video Coding) software on MPEG-FTV test sequences. Overall performance of proposed prediction structure is measured in the PSNR and subjective image quality measure such as PSPNR.

Three-D (3-D) stereo video is becoming widely available and one need to consider depth effects when extending the 2-D video processing algorithms to 3-D stereo set-up. Depth is the additional attribute which contributes to overall visual quality of the 3-D stereo video. Sharpness enhancement algorithm is commonly applied in 2-D video processing chain; the effect of depth perception when sharpness enhancement algorithm is applied to the 3-D stereo video is studied. A subjective experiment is presented to study the relation between blur/sharpness and depth. A concept of just noticeable blur (JNB) at different depths is introduced for the stereo image pairs. Based on the results of the experiment an adaptive sharpness enhancement algorithm is proposed. The visual quality results, of the proposed depth aware sharpness enhancement algorithm, are presented.

Depth maps are important for generating images with new camera viewpoints from a single source image for stereoscopic applications. In this study we examined the usefulness of smoothing depth maps for reducing the cardboard effect that is sometimes observed in stereoscopic images with objects appearing flat like cardboard pieces. Six stereoscopic image pairs, manifesting different degrees of the cardboard effect, were tested. Depth maps for each scene were synthesized from the original left-eye images and then smoothed (low-pass filtered). The smoothed depth maps and the original left-eye images were then used to render new views to create new "processed" stereoscopic image pairs. Subjects were asked to assess the cardboard effect of the original stereoscopic images and the processed stereoscopic images on a continuous quality scale, using the doublestimulus method. In separate sessions, depth quality and visual comfort were also assessed. The results from 16 viewers indicated that the processed stereoscopic image pairs tended to exhibit a reduced cardboard effect, compared to the original stereoscopic image pairs. Although visual comfort was not compromised with the smoothing of the depth maps, depth quality was significantly reduced when compared to the original.

The 2D/3D switchable LCD stereoscopic display is based on the method of chromatic stereoscopic frame separation that occurs between the left and right eye with the use of two spectrally independent backlight sources with primary color sets RGBLeft and RGBRigth. Semiconductor lasers and/or LEDs with narrow band spectral filters could be used as backlight sources. In a 3D mode, the RGB sources illuminate alternatively and synchronously with the alternative displaying of stereo pair frames. Glasses with chromatic filters are used for watching in 3D mode and separate light from two RGB sources between the eyes of the viewer and in that way separate stereoscopic frames. In a 2D mode, any one of the RGB sources would work. In this case, the display looks like an ordinary display. Two RGB sources could be used for color enhancement in 2D mode. The quality of the images in 3D mode is the same as in 2D. Frame frequency is two times less in 3D mode as compared to 2D. The suggested method competes with two known methods for LCD that use glasses: "Shutter-Glasses Method" and "Polarization Method." It has certain advantages: image quality and stereo channel separation are independent from viewing angle or head tilt of viewer, and there are no active components in the glasses.

This research aims to develop an objective no-reference video quality evaluation method for MPEG-2 MP@ML coded (symmetric and asymmetric) stereoscopic videos. Our proposed method is based on segmented local features of spatial artifacts, disparity, and temporal activities of videos. Segmented local features information such as edge and non-edge areas of any stereoscopic pair frames (i.e., left and right views) have taken into consideration for blockiness and zero crossing. In this method, a temporal segmentation approach is considered and each temporal segment is evaluated for artifacts and disparity. Temporal features are calculated separately for left and right video sequences based on segmented local features and sub temporal segment. Different weighting factors are also applied to measure the spatial artifacts, disparity, and temporal features of the segment. In order to verify the performance, we conducted subjective experiment on different symmetric and asymmetric coded (Bit rates: 2, 3, 5, and 8 Mbps) stereo video pairs. An auto stereoscopic display was used for fifteen (15) reference stereo videos; each of the video was 15 seconds length and the total length of each test sequence was (15×15 sec = 3 min 45 sec). Seven video sequences were used to complete the experiment. The Single Stimulus Continuous Quality Evaluation (SSCQE) method was used to conduct our subjective experiment. The experiment result indicates that our proposed method has given sufficient prediction performance.

We present visual acuity metrics, human factors issues, and technical considerations in the construction of a stereorendered reality test in the spirit of the Turing test, Alan Turing's famous artificial intelligence test designed to explore the boundaries between human and machine interaction. The overall aim of this work is to provide guiding principles in the design of a stereoscopic reality test.

A novel 360-degree 3D image acquisition system that captures multi-view images with narrow view interval is proposed. The system consists of a scanning optics system and a high-speed camera. The scanning optics system is composed of a double-parabolic mirror shell and a rotating flat mirror tilted at 45 degrees to the horizontal plane. The mirror shell produces a real image of an object that is placed at the bottom of the shell. The mirror shell is modified from usual system which is used as 3D illusion toy so that the real image can be captured from right horizontal viewing direction. The rotating mirror in the real image reflects the image to the camera-axis direction. The reflected image observed from the camera varies according to the angle of the rotating mirror. This means that the camera can capture the object from various viewing directions that are determined by the angle of the rotating mirror. To acquire the time-varying reflected images, we use a high-speed camera that is synchronized with the angle of the rotating mirror. We have used a high-speed camera which resolution is 256×256 and the maximum frame rate is 10000fps at the resolution. Rotating speed of the tilted flat mirror is about 27 rev./sec. The number of views is 360. The focus length of parabolic mirrors is 73mm and diameter is 360mm. Objects which length is less than about 30mm can be acquired. Captured images are compensated rotation and distortion caused by double-parabolic mirror system, and reproduced as 3D moving images by Seelinder display.

The abnormal contraction of ciliary muscles due to the performance of a near visual task for several hours causes various vision problems such as asthenopia and visual loss. However, these problems can be resolved by activating the muscles by alternately repeating negative and positive accommodation. In this study, we have verified the effect of accommodation training that uses the strategy of presenting a stereoscopic movie to myopic youth and measuring the uncorrected distant visual acuity, spherical diopter (SPH), and subjective index of asthenopia obtained using a visual analog scale (VAS). Stereoscopic movies are prepared by using the POWER 3D method (Olympus Visual Communications Co., Ltd.), which reduces the inconsistency between the experienced and the actual senses. Thirty two myopic students aged 20 ± 1 years (16 males and 16 females) were chosen as the subjects. One group performed the accommodation training for 6 min, and the other group underwent a near visual task during the same period as the control group. We concluded the following from each item of verification: (a) The accommodation training using a stereoscopic movie had temporarily improved visual acuity. (b) This training led to a decrease in asthenopia. (c) The training improved the near-point accommodation function.

Live three-dimensional movies are acquired with stereoscopic cameras. Since there is a limitation of disparity for binocular fusion, it is necessary to restrict objects of shooting within an allowable range. This problem is more severe for stereoscopic LED display because of its large pitch. One of the answers to this problem is to control the depth of field. Objects outside the allowable range are blurred and objects inside the allowable range are in focus. In this study, we propose a method to control the depth of field of stereoscopic images continuously by lens-tilt imaging. There are two types of configurations in the lens-tilt stereoscopic imaging. One is a convergent configuration and the other is a divergent configuration. We analyze the relationships between the lens-tilt angle, the ratio of the lens-to-imaging device distance to focal length, and the plane-of-sharp-focus angle. Then the near and far limits of the depth of field of stereoscopic cameras in lens-tilt configurations are formulated geometrically. It is revealed that the depth of field in the divergent configuration is larger than that in the convergent configuration even if the distance between the two lenses is the same. The analyses and continuous control of the depth of field are also confirmed experimentally.

Stereoscopic movie production has been a topic in the making of films for a long time. However, it hasn't made it to the amateur sector. Commercially available stereoscopic cameras are too expensive for non professionals. When producing a stereo video with two separate standard cameras, synchronicity and spatial offset maintenance between the two views is a challenging non-trivial task. Even when this is done properly, the general lack of software-tools for stereo video post production definitely daunts any non-professional ambitions. In this work we present a tool for preprocessing stereoscopic videos. We describe how the input videos can be converted automatically to a stereo-movie, ready for displaying or further processing in standard video software.

In this paper we apply imaging polarimetry to characterize polarization based stereoscopic 3D displays. New imaging polarimeter is first described and its potentialities to characterize LCDs and their components are shown. Then one polarization based stereoscopic 3D display is measured with this new tool. 3D and standard contrast are evaluated on the entire surface of the display and compared to what is expected from viewing angle measurements at specific locations. Important inhomogeneities probably related to the quality of the phase shift film are detected. Color shifts are measured and interpreted with regards to polarization multispectral viewing angle measurements. The main sources of imperfections are detected and solutions to improve the characteristics are proposed.