Spatial video segmentation is usually performed based on motion between two frames. Some recent approaches extend this to joint segmentation in space-time; the resulting 3-D segmentation can be interpreted as a volume "carved out" by a moving object in the image sequence domain, or the so-called "object tunnel". In this paper, we extend this concept to explicit modeling of occlusion events in
space-time. In addition to the modeling of object evolution, we also explicitly model occluded and newly-exposed areas in the background by means of "occlusion volume", a new space-time concept. A voxel belongs to occlusion volume if its intensity is consistent with past intensities along its motion trajectory but inconsistent with future intensities (reversed for "exposed volume"). We propose a variational formulation of the problem that we solve using the multiphase level set method. We show encouraging experimental results for synthetic and natural image sequences.
Spatial segmentation of image sequences is usually performed based on motion between two frames, and then followed by tracking. Some recent approaches extend this to joint segmentation in space-time; the resulting 3-D segmentation (in x-y-t space) can be interpreted as a volume 'carved out by a moving object in the image sequence domain. We call such volumes 'object tunnels'. In this paper, we propose a new approach to occlusion analysis and characterization that is based on object tunnels. It results from the observation that object-tunnel wall for a fully visible object has different shape than that for an object undergoing occlusion or exposure. Walls of tunnels associated with moving objects have tangent planes that are, in general, non-parallel to the time axis. When an object gets occluded or exposed by a static feature, part of the object tunnel wall stops evolving freely; its spatial coordinates remain fixed (static occlusion boundary) while the temporal coordinate increases linearly (time evolution). This forces part of the wall to be comprised of lines parallel to the time axis, each line defined by a single point on the occlusion boundary. In case this boundary is a straight-line edge, the occluding part of the wall becomes planar. We propose to detect occlusions by searching for such characteristic surfaces of object tunnel walls. We formulate the problem for planar occlusion walls based on a robust distance metric, and we show experimental results for various occlusion types on synthetic and camera-acquired image