We present a new solution to the known problem of video keying in a natural environment. We segment foreground objects from background objects using their relative distance from the camera, which makes it possible to do away with the use of color for keying.
To do so, we developed and built a novel depth video camera, capable of producing RGB and D signals, where D stands for the distance to each pixel. The new RGBD camera enables the creation of a whole new gallery of effects and applications such as multi-layer background substitutions. This new modality makes the production of real time mixed reality video possible, as well as post-production manipulation of recorded video.
We address the problem of color spill -- in which the color of the foreground object is mixed, along its boundary, with the background color. This problem prevents an accurate separation of the foreground object from its background, and it is most visible when compositing the foreground objects to a new background.
Most existing techniques are limited to the use of a constant background color. We offer a novel general approach to the problem with enabling the use of the natural background, based upon the D channel generated by the camera.
At 3DV Systems Ltd. we developed and built a true 3D video camera (Zcam), capable of producing RGB and D signals where D stands for distance or depth to each pixel.
The new RGBD camera makes it possible to do away with color based background substitution known as chroma-key as well as creating a whole gallery of new effects and applications such as multilayer foreground as well as background substitutions and manipulations.
The new multilayerd modality makes possible the production of mixed reality real time video as well as post- production manipulation of recorded video.
The new RGBD camera is scannerless and uses low power laser illumination to create the D channel. Prototypes have been in use for more than 2 years and, are capable of sub-centimeter depth resolution at any desired distance up to 10 m. on the present model. Additional potential applications as well as low cost versions are currently being explored.