Equipment to enjoy a 3D image, such as a movie theater, television and so on have been developed many. So 3D video are widely known as a familiar image of technology now. The display representing the 3D image are there such as eyewear, naked-eye, the HMD-type, etc. They has been used for different applications and location. But have not been widely studied for the transparent 3D display. If transparent large 3D display is realized, it is useful to display 3D image overlaid on real scene in some applications such as road sign, shop window, screen in the conference room etc. As a previous study, to produce a transparent 3D display by using a special transparent screen and number of projectors is proposed. However, for smooth motion parallax, many projectors are required. In this paper, we propose a display that has transparency and large display area by time multiplexing projection image in time-division from one or small number of projectors to active screen. The active screen is composed of a number of vertically-long small rotate mirrors. It is possible to realize the stereoscopic viewing by changing the image of the projector in synchronism with the scanning of the beam.3D vision can be realized by light is scanned. Also, the display has transparency, because it is possible to see through the display when the mirror becomes perpendicular to the viewer. We confirmed the validity of the proposed method by using simulation.
A super multi-view display provides three-dimensional images by emitting a lot of rays of different colors depending on the direction from each point on the display. It provides smooth motion parallax without special glasses, and it is expected that the observer is free from the visual fatigue caused by the accommodation-vergence conflict. However, a huge number of pixels are required on a display device because high-density rays are required for good quality images and each ray needs corresponding pixel. We proposed a new method to reduce the required number of pixels by limiting rays emitted to only around observer’s pupils. The display is based on the lenticular method. As stated above, the rays should be shot out to only around observer’s pupils. Therefore, the lenticular lens of which viewing zone angle is narrowed is used. But, due to the characteristics of the lenticular lens, the same image is seen repeatedly from different positions out of the viewing zone. It is called side lobe. Because of the side lobes, the rays for one eye enter the other eye. To suppress these side lobes, we proposed the lenticular lens is illuminated by the directional light. The direction of the directional light has to be changed to follow the observer’s eye. We implemented optical designs based on the technique as mentioned above, and we produced a prototype display. We experimented with consideration of change of viewing angle and viewing distance. We confirmed usefulness of the proposed display by these experiment result.
For safe driving, it is important for driver to check traffic conditions such as traffic lights, or traffic signs as early as soon. If on-vehicle camera takes image of important objects to understand traffic conditions from long distance and shows these to driver, driver can understand traffic conditions earlier. To take image of long distance objects clearly, the focal length of camera must be long. When the focal length is long, on-vehicle camera doesn’t have enough field of view to check traffic conditions. Therefore, in order to get necessary images from long distance, camera must have long-focal length and controllability of shooting direction. In previous study, driver indicates shooting direction on displayed image taken by a wide-angle camera, a direction controllable camera takes telescopic image, and displays these to driver. However, driver uses a touch panel to indicate the shooting direction in previous study. It is cause of disturb driving. So, we propose a telephoto camera system for driving support whose shooting direction is controlled by driver’s gaze to avoid disturbing drive. This proposed system is composed of a gaze detector and an active telephoto camera whose shooting direction is controlled. We adopt non-wear detecting method to avoid hindrance to drive. The gaze detector measures driver’s gaze by image processing. The shooting direction of the active telephoto camera is controlled by galvanometer scanners and the direction can be switched within a few milliseconds. We confirmed that the proposed system takes images of gazing straight ahead of subject by experiments.
A super multi-view display provides smooth motion parallax without special glasses, and it is expected that the observer is free from the visual fatigue caused by the accommodation-vergence conflict. However, a huge number of pixels are required on a display device because high-density rays are required for good quality images and each ray needs corresponding pixel. We have proposed a method to reduce the required number of pixels by limiting rays emitted to only around observer’s pupils. The display is based on the lenticular method. As stated above, the rays should be shot out to only around observer’s pupils. To do this, the lenticular lens of which viewing zone angle is narrowed is used and the lenticular lens is illuminated by directional light to suppress side lobes. The direction of directional light is changed to follow the observer’s pupil. In this paper, we constructed a prototype display and conducted an experiment. The experimental result confirmed that we could see image corresponding from each viewpoint by the change of the photographing images. In addition, it confirmed suppression of the side lobes by couldn’t see the image outside the viewing zone. By these results, we showed the effectiveness of the proposed method.
Head Up Display (HUD) is being applied to automobile. HUD displays information as far virtual image on the windshield. Existing HUD usually displays planar information. If the image corresponding to scenery on the road like Augmented Reality (AR) is displayed on the HUD, driver can efficiently get the information. To actualize this, HUD covering large viewing field is needed. However existing HUD cannot cover large viewing field. Therefore we have proposed system consisting of projector and many small diameter convex lenses. However observed virtual image has blurring and distortion . In this paper, we propose two methods to reduce blurring and distortion of images. First, to reduce blurring of images, distance between each of screen and lens comprised in lens array is adjusted. We inferred from the more distant the lens from center of the array is more blurred that the cause of blurring is curvature of field of lens in the array. Second, to avoid distortion of images, each lens in the array is curved spherically. We inferred from the more distant the lens from center of the array is more distorted that the cause of distortion is incident angle of ray. We confirmed effectiveness of both methods.
The Three-Dimensional (3D) vision became widely known as familiar imaging technique now. The 3D display has been put into practical use in various fields, such as entertainment and medical fields. Development of 3D display technology will play an important role in a wide range of fields. There are various ways to the method of displaying 3D image. There is one of the methods that showing 3D image method to use the ray reproduction and we focused on it. This method needs many viewpoint images when achieve a full-parallax because this method display different viewpoint image depending on the viewpoint. We proposed to reduce wasteful rays by limiting projector’s ray emitted to around only viewer using a spinning mirror, and to increase effectiveness of display device to achieve a full-parallax 3D display. We propose a method by using a tracking viewer’s eye, a high-speed projector, a rotating mirror that tracking viewer (a spinning mirror), a concave mirror array having the different vertical slope arranged circumferentially (a concave mirror array), a cylindrical mirror. About proposed method in simulation, we confirmed the scanning range and the locus of the movement in the horizontal direction of the ray. In addition, we confirmed the switching of the viewpoints and convergence performance in the vertical direction of rays. Therefore, we confirmed that it is possible to realize a full-parallax.
This paper focuses on a road-to-vehicle visible light communication (VLC) system using LED traffic light as the transmitter and camera as the receiver. The traffic light is composed of a hundred of LEDs on two dimensional plain. In this system, data is sent as two dimensional brightness patterns by controlling each LED of the traffic light individually, and they are received as images by the camera. Here, there are problems that neighboring LEDs on the received image are merged due to less number of pixels in case that the receiver is distant from the transmitter, and/or due to blurring by defocus of the camera. Because of that, bit error rate (BER) increases due to recognition error of intensity of LEDs To solve the problem, we propose a method that estimates the intensity of LEDs by solving the inverse problem of communication channel characteristic from the transmitter to the receiver. The proposed method is evaluated by BER characteristics which are obtained by computer simulation and experiments. In the result, the proposed method can estimate with better accuracy than the conventional methods, especially in case that the received image is blurred a lot, and the number of pixels is small.
Super multi-view display provides smooth motion parallax without special glasses, and it is expected that the observer is free from the visual fatigue caused by the accommodation-vergence conflict. However, there is a problem that a huge number of pixels are required on a display device such as liquid crystal display panel because high-density rays are required for good quality images and each ray needs corresponding pixel. In this paper, we propose a new three-dimensional display based on lenticular method to reduce the required number of pixels. The rays are shot out to only
around observer’s pupils. To do this, the lenticular lens of which viewing zone angle is narrowed is used and the
lenticular lens is illuminated by parallel light made by cylindrical lenses and LEDs to suppress sidelobes. The direction
of the parallel light is changed to follow the observer’s eye. We designed the optics system and confirmed the
availability of the proposed method by computer simulation. Moreover, we constructed a prototype display and
demonstrated. As a result, the parallax pitch and the viewing zone nearly equaled to designed values when the display
was observed from the front, but these values were increased with the increasing viewing angle. The result of analysis,
the reason why the parallax pitch and the viewing zone were expanded is thought as the curvature of field of the lenticular lens.
In driving a vehicle, if the information which is needed to drive can be displayed like Augmented Reality (AR), driver operates the vehicle effectively. For example, it is efficient to display the images indicating an intersection where the vehicle should turn next. Unlike conventional AR system such as using See Through Head Mounted Display, Head Up Display (HUD) which is currently used on vehicle for displaying speed meter, tachometer and so on, can display the
optical virtual images for long distance view and covers large viewing field. But it is difficult to apply HUD into AR
because of narrow viewing field. To optimize this, we propose a system in which HUD is divided up many small optical
systems. A convex lens array and elemental images which are similarly to Integral Photography (IP) were applied. One
elemental image has been corresponding in front of each lens which generates the virtual image. In this paper, a
theoretical formula of the position relation among the elemental images was solved to create the continuous virtual
images. Moreover, we simulated the system with ray tracing method.
In this paper, we discuss a multiview video and depth coding system for Multiview video applications such as 3DTV
and Free View-point Television (FTV) 1. We target an appropriate multiview and depth compression method. And then
we investigate the effect on free view synthesis quality by changing the transmission rates between multiview and depth
sequences. In the simulations, we employ MVC in parallel to compress the multiview video and depth sequences at
different bitrates, and compare the virtual view sequences generated by decoded data with the original video sequences
taken in the same viewpoint. Our experimental results show that bitrates of multi depth stream has less effect on the view
synthesis quality compared with the multi view stream.
Visible Light Communication (VLC) is a wireless communication method using LEDs. LEDs can respond in high-speed
and VLC uses this characteristics. In VLC researches, there are two types of receivers mainly, one is photodiode receiver
and the other is high-speed camera. A photodiode receiver can communicate in high-speed and has high transmission
rate because of its high-speed response. A high-speed camera can detect and track the transmitter easily because it is not
necessary to move the camera. In this paper, we use a hybrid sensor designed for VLC which has advantages of both
photodiode and high-speed camera, that is, high transmission rate and easy detecting of the transmitter. The light
receiving section of the hybrid sensor consists of communication pixels and video pixels, which realizes the advantages.
This hybrid sensor can communicate in static environment in previous research. However in dynamic environment, high-speed
tracking of the transmitter is essential for communication. So, we realize the high-speed tracking of the transmitter
by using the information of the communication pixels. Experimental results show the possibility of communication in
In general, free-viewpoint image is generated by captured images by a camera array aligned on a straight line or
circle. A camera array is able to capture synchronized dynamic scene. However, camera array is expensive and
requires great care to be aligned exactly. In contrast to camera array, a handy camera is easily available and can
capture a static scene easily. We propose a method that generates free-viewpoint images from a video captured by
a handheld camera in a static scene. To generate free-viewpoint images, view images from several viewpoints and
information of camera pose/positions of these viewpoints are needed. In a previous work, a checkerboard pattern
has to be captured in every frame to calculate these parameters. And in another work, a pseudo perspective
projection is assumed to estimate parameters. This assumption limits a camera movement. However, in this
paper, we can calculate these parameters by "Structure From Motion". Additionally, we propose a selection
method for reference images from many captured frames. And we propose a method that uses projective block
matching and graph-cuts algorithm with reconstructed feature points to estimate a depth map of a virtual
We are developing technologies for FTV in which the viewer can freely change the viewpoint. The free-viewpoint image
can be generated by using images captured by an static multi-camera system. However, it is hard to render an object that
moves widely in the scene. In this paper, we address this problem by proposing moving camera array and the free-viewpoint
image synthesis algorithm. In our synthesis method, we use the temporal and spatial information together, in
order to further improve the view generation quality. Experiments using a sequence captured by simulated moving multi-camera
systems demonstrate the improvement of view synthesis quality in comparison with conventional view synthesis
In this paper, we propose a method for high efficiency acquisition of Ray-Space for FTV (Free viewpoint TV). In this
research, incomplete data is directly captured by a novel device, i.e. photodiode/lens array, and transformed to full
information by Radon transform. We must capture the large amount of data in conventional acquisition of Ray-Space
using multiple cameras. However Ray-space has redundancy because it consists of set of lines which depend on depth of
objects. We use the Radon transform to exploit this redundancy. The Radon transform is set of projection data along
different directions. Thus Ray-space can be reconstructed from projection data in limited range by the inverse Radon
transform. Capturing the part of projection data correspond to capturing sums of several rays by 1 pixel. We have
simulated reconstruction of Ray-space projection data which was computed by computer simulation of capturing device.
As a result, by using fewer pixels than rays, we could reduce the information to reconstruct Ray-space.
In this paper, we present a new image acquisition system for FTV (Free-viewpoint TV). The proposed system can
capture the dynamic scene from all-around views. The proposed system consists of two ellipsoidal mirrors, a high-speed
camera, and a rotating aslope mirror. As for two ellipsoidal mirrors, the size and the ellipticity are mutually different.
The object is set in the focus of ellipsoidal mirror. The size of this system is smaller than that of early system since
ellipsoidal mirrors can reduce virtual images. High-speed camera can acquire multi viewpoint images by mirror
scanning. Here, we simulated this system with ray tracing and confirmed the principle.
The availability of multi-view images of a scene makes possible new and exciting applications, including Free-viewpoint
TV (FTV). FTV allows us to change viewpoint freely in a 3D world, where the virtual viewpoint images are synthesized
by Depth-Image-Based Rendering (DIBR). In this paper, we propose a new method of DIBR using multi-view images
acquired in a linear camera arrangement. The proposed method improves virtual viewpoint images by predicting the
residual errors. For virtual viewpoint image synthesis, it is necessary to estimate the depth maps with multi-view images.
Some algorithms to estimate depth map were proposed, but it is difficult to estimate accurate depth map. As a result,
rendered virtual viewpoint images have some errors due to the depth errors. Therefore, our proposed method takes into
account those depth errors and improves the quality of the rendered virtual viewpoint images. In the proposed method,
the virtual images of each camera position are generated using the real images from each other camera. Then, the
residual errors can be calculated between the generated images and the real images acquired by the actual cameras. The
residual errors are processed and fed back to predict the residual errors that can be happened to virtual viewpoint images
generated by conventional method. In the experiments, PSNR could be improved for few decibels compared with the
A novel 360-degree 3D image acquisition system that captures multi-view images with narrow view interval is proposed.
The system consists of a scanning optics system and a high-speed camera. The scanning optics system is composed of a
double-parabolic mirror shell and a rotating flat mirror tilted at 45 degrees to the horizontal plane. The mirror shell
produces a real image of an object that is placed at the bottom of the shell. The mirror shell is modified from usual
system which is used as 3D illusion toy so that the real image can be captured from right horizontal viewing direction.
The rotating mirror in the real image reflects the image to the camera-axis direction. The reflected image observed from
the camera varies according to the angle of the rotating mirror. This means that the camera can capture the object from
various viewing directions that are determined by the angle of the rotating mirror. To acquire the time-varying reflected
images, we use a high-speed camera that is synchronized with the angle of the rotating mirror. We have used a high-speed
camera which resolution is 256×256 and the maximum frame rate is 10000fps at the resolution. Rotating speed of
the tilted flat mirror is about 27 rev./sec. The number of views is 360. The focus length of parabolic mirrors is 73mm and
diameter is 360mm. Objects which length is less than about 30mm can be acquired. Captured images are compensated
rotation and distortion caused by double-parabolic mirror system, and reproduced as 3D moving images by Seelinder
In this paper, we propose a method for compressive acquisition of Ray-Space. Briefly speaking, incomplete data which
directly captured by a specific device is transformed to full information by Radon transform. Ray-Space, which
represents 3D images, describes position and direction of rays on reference plane in real space. Ray-Space has
information of many rays. In conventional acquisition of Ray-Space, multiple cameras are used and 1 pixel on a camera
captures 1 ray. Thus we need many pixels and we must capture the large amount of data. However Ray-Space has
redundancy because Ray-Space consists of set of lines which depend on the depth of objects. We use the Radon
transform to exploit this redundancy. The Radon transform is set of projection data along different directions. The Radon
transform of Ray-Space show uneven distribution. Thus Ray-Space can be reconstructed from projection data in limited
range by the inverse Radon transform. Capturing the part of projection data correspond to capturing sums of several rays
by 1 pixel. A sum of several rays means a sum of brightness of rays. In this paper, we have simulated reconstruction of
Ray-Space projection data which was computed by the Radon Transform of Ray-Space. This experiment showed that
Ray-Space could be reconstructed from the parts of projection data. As a result, using fewer pixels than rays, we could
reduce the amount of data to reconstruct Ray-Space.
This paper presents a novel 3D display using a new principle which has the features of both Integral Imaging (II) and volumetric display. The display we propose consists of two lens arrays, a convex lens array and a concave lens array, and one 2D display moving back and forth. The two lens arrays are placed between the 2D display and observer. The concave lens array forms elemental images, and the convex lens array and the formed elemental images reproduce a depth division image like the II method. When the observer watches the 2D display through the two lens arrays, he feels that the image displayed by the 2D display is reproduced not at the position of 2D display but at a certain depth according to the position of the 2D display. So when the 2D display is moved, the reproduced image also moves to another depth position. Therefore various depth images can be reproduced by the movement of the 2D display. This is how the proposed display reconstructs 3D space. This time we introduce the optics system which can reconstruct a wireframe cube by oscillating the 2D display only a few centimeters. We also show the result of simulation of the proposing display with a ray tracing method to confirm the moving parallax.
In this paper, we introduce a new Ray-Space acquisition system that we developed. The Ray-Space method records the
position and direction of rays that are transmitted in the space as ray data. The composition of arbitrary viewpoint images
using the Ray-Space method enables the generation of realistic arbitrary viewpoint picture. However, acquisition of a
dense Ray-Space is necessary to apply the Ray-Space method. The conventional method of acquiring the ray data uses a
camera array. This method enables capturing a dynamic scene. To acquire a dense Ray-Space by this method, however,
interpolation is necessary. There is another common method for ray data acquisition, which uses a rotating stage. This
method enables capturing images without requiring interpolation. However, only static scenes can be captured by this
method. Therefore, we developed a new Ray-Space acquisition system. This system uses two parabolic mirrors. Incident
rays that are parallel to the axis of a parabolic mirror gather at the focus of the parabolic mirror. Hence, rays that come out
of an object that is placed at the focus of the lower parabolic mirror gather at the focus of the upper parabolic mirror. Then,
the real image of the object is generated at the focus of the upper parabolic mirror, and a rotating aslope mirror scans rays
at the focus of the upper parabolic mirror. Finally, the image from the aslope mirror is captured by a camera. By using this
system, we were able to acquire an all-around image of an object.
The 3D display using light beam reconstruction method has some great advantages. Special glasses are not needed. The observation point is not fixed. Some researcher claims that a viewer may be able to focus on 3D images under the super multi-view condition. However, the 3D display needs to reconstruct a great number of light beams. Usually, the number of light beams is limited by the resolution of a flat-panel display because only the space-division method is used. Therefore, improving the performance of the flat-panel display as a 3D display is difficult. Thus, using the time-multiplexing method is important.
In this paper, we discussed the 3D display using light beam reconstruction method that uses a fast light shutter as the 3D display with the time-multiplexing method. We consider the relationsip between the performance of the 3D display and that of the devices that comprise the 3D display. The simulation results of the super multi-view condition suggest that the number of light beams that enter the pupil of the viewer's eye and the width of the slit are important for the accommodation function.
This paper presents a novel 3D display using a new principle which has the features of both Integral Imaging (II) and
volumetric display. The proposed display consists of one 2D display and two lens arrays, a convex lens array and a
concave lens array. The two lens arrays are placed between the 2D display and the observer. When the observer watches
the 2D display through the two lens arrays, he feels that the image displayed by 2D display is reproduced at the position
which is different from the position of the 2D display. Furthermore, by changing the position of the 2D display, the
image is reproduced at the different position than before. Therefore the various depth images are reproduced by moving
2D display. This is how the proposed display reconstructs 3D space. Here, we simulated this display with ray tracing and
checked its validity.
Ray-Space is categorized by Image-Based Rendering (IBR), thus generated views have photo-realistic quality.
While this method has the performance of high quality imaging, this needs a lot of images or cameras. The reason
why that is Ray-Space requires various direction's and position's views instead of 3D depth information. In this
paper, we reduce that flood of information using view-centered ray interpolation. View-centered interpolation
means estimating view dependent depth value (or disparity map) at generating view-point and interpolating
that of pixel values using multi-view images and depth information. The combination of depth estimation and
interpolation realizes the rendering photo-realistic images effectively. Unfortunately, however, if depth estimation
is week or mistake, a lot of artifacts appear in creating images. Thus powerful depth estimation method is
required. When we render the free viewpoint images video, we perform the depth estimation at every frame.
Thus we want to keep a lid on computing cost. Our depth estimation method is based on dynamic programming
(DP). This method optimizes and solves depth images at the weak matching area with high-speed performance.
But scan-line noises become appeared because of the limit of DP. So, we perform the DP multi-direction pass and
sum-up the result of multi-passed DPs. Our method fulfills the low computation cost and high depth estimation
In this paper, we analyze the distortion of images acquired with a novel Ray-Space acquisition system. In case
an arbitrary viewpoint picture is generated using the Ray-Space method, it is necessary to acquire dense ray
data. Conventional methods for acquiring the Ray-Space data consist of using rotating stages or a camera
array. We developed a system consisting of two parabolic mirrors, a synchronized galvanometric mirror and
a high-speed camera. The principle is as follows; if an object is put in the bottom of the parabolic mirror,
the ray which comes out of the object is imaged in the upper part, and form a real image. The galvanometer
mirror is put on the position of a real image, and is made to scan horizontally. Images of the object of different
angles (directions) are then possible to generate and are captured by the high-speed camera. By capturing
many images at each scan, Ray-Space is therefore acquirable. However, distortion arises in the real image
of the object formed. Consequently, distortion appears in the captured image. Therefore, it is necessary to
correct the captured image to the right image. Here, we examine a method to generate corrected images from
the acquired Ray-Space.
We propose a technique of Imaged-Based Rendering(IBR) using a circular camera array. By the result of having recorded the scene as surrounding the surroundings, we can synthesize a more dynamic arbitrary viewpoint images and a wide angle images like a panorama . This method is based on Ray- Space, one of the image-based rendering,
like Light Field. Ray-Space is described by the position (x, y) and a direction (θ, φ) of the ray's parameter which passes a reference plane. All over this space, when the camera has been arranged circularly, the orbit of the point equivalent to an Epipor Plane Image(EPI) at the time of straight line arrangement draws a sin curve. Although described in a very clear form, in case a rendering is performed, pixel of which position of which camera being used and the work for which it asks become complicated. Therefore, the position (u, v) of the position (s, t) pixel of a camera like Light Filed redescribes space expression. It makes the position of a camera a polar-coordinates system (r, theta), and is making it close to description of Ray-Space. Thereby, although the orbit of a point
serves as a complicated periodic function of periodic 2pi, the handling of a rendering becomes easy. From such space, the same as straight line arrangement, arbitrary viewpoint picture synthesizing is performed only due to a geometric relationship between cameras. Moreover, taking advantage of the characteristic of concentrating
on one circular point, we propose the technique of generating a wide-angle picture like a panorama. When synthesizing a viewpoint, since it is overlapped and is recording the ray of all the directions of the same position, this becomes possible. Having stated until now is the case where it is a time of the camera fully having been
arranged and a plenoptic sampling being filled. The discrete thing which does not fill a sampling is described from here. When arranging a camera in a straight line and compounding a picture, in spite of assuming the pinhole camera model, an effect like a focus shows up. This is an effect peculiar to Light Field when a sampling is not
fully performed, and is called a synthetic aperture. We have compounded all focal images by processing called an "Adaptive Filter" to such a phenomenon. An adaptive filter is the method of making the parallax difference map of perfect viewpoint dependence centering on a viewpoint to make. This is a phenomenon produced even when it has arranged circularly. Then, in circular camera arrangement, this adaptive filter is extended, and all focal pictures are compounded. Although there is a problem that an epipor line is not parallel etc. when it has arranged circularly, extension obtains enough, it comes only out of geometric information, and a certain thing is clarified By taking such a method, it succeeded in performing a wide angle and arbitrary viewpoint image synthesis also from discrete space also from the fully sampled space.
A ray-based cylindrical display is proposed that allows multiple viewers to see 3D images from a 360-degree horizontal arc without wearing 3-D glasses. This technique uses a cylindrical parallax barrier and a one-dimensional light source array constructed from such semiconductor light sources as LEDs aligned in a vertical line. The light source array rotates along the inside of the cylindrical parallax barrier, and the intensity of each light is synchronously modulated with the rotation.
Since this technique is based on the parallax panoramagram, the density of rays is limited by the diffraction at the parallax barrier. In order to solve this problem, we employed revolving parallax barrier. We have developed two protype displays and they showed high presence 3D image. Especially the newer one is capable of displaying color images whose diameter is 200mm, it is suitable for displaying real object like a human head.
Therefore we acquired ray-space data using a video camera rotating around an object and reconstructed the object using the prototype display successfully. In this paper, we describe details of the system and discuss about ray control method to reconstruct object from ray-space data.
We propose a 3D live video system that generates arbitrary viewpoints in real-time based on the ray-space, one of the image-based rendering. With this system, a remote user can freely change the viewpoint, not only according to the captured camera position, but also can synthesize views where a camera is not physical present using the ray-space interpolation. The basic idea of ray-space rendering is collecting and rearranging the partition of simultaneously captured images according to an arbitrarily specified virtual-view. If hundreds of cameras were arranged in significant density, synthesizing a free viewpoint away from the camera baseline require only camera geometric information. Since we cannot obtain such full information of ray according to plenoptic sampling, arbitrarily view generation necessitate interpolation of slightly missed rays. However, such view interpolation's cost is particularly huge. Therefore, we introduce three novel techniques of view interpolation: first, view centered interpolation framework, second, estimating disparity with smoothing, third, hierarchical searching of correspondences for fast computation. Moreover, we implement the experimental system with those algorithms. This free-view generating system includes sixteen cameras arranged straightforward. All cameras are connected with the consumer computers one by one. Whole the computers connect a server computer via Ethernet categorized star network. This system carries out four processes in real time: capture images, correct position of cameras with projective transformation, interpolate images on baseline, rendering arbitrary viewpoint.
The experimental result shows that this system is rendering arbitrary viewpoint at 12fps (frames per second) set image resolution set to "320x240". We succeeded in synthesizing highly photo-realistic images.
This paper proposes a novel multiple-image coding technique using Ray-Space interpolation. Ray-Space, an image-based rendering technique to generate arbitrary views from multiple cameras, describes three- dimensional space based on only ray information from a large number of cameras. Therefore, data compression is needed. We leverage the correlation of time and space aiming for high compression. H.264/AVC is employed for dynamic image coding, and studies have been conducted on using the AVC in time domain. Here we propose a novel algorithm that uses view-interpolation for coding in space domain. Interpolation is a method to generate the middle view in a stereoscopic setup. By generating interpolated images from coded images as reference ones, coding performance should give better results. Therefore, interpolation accuracy is important for coding performance. In this paper, we propose an interpolation technique using geometric information in a linear camera arrangement. By calculating the trace of each point considering camera arrangement, and obtaining its corresponding point, the middle image is generated. In so doing, the interpolation method is an intensity-based scheme, constrained by smoothness in disparity domain. Experiment of coding using interpolation outperforms the standard AVC by 1~2 dB in all bitrates. Moreover, we deal with occlusion regions by means of extrapolation using four images. To detect occlusion regions, we use two criteria, one is minimum error, second is ratio of minimum error between four images. In occlusion region, the intensity of middle image is generated using extrapolated images. This method gives up to 1~3 dB improvement compared to occlusion-ignored algorithm.