We describe an efficient system for recovering the 3-D motion and structure of entire polyhedra from an evolving image sequence. The suggested technique utilizes the image flow velocities in order to recover the 3-D parameters. We develop a method for estimating the image flow velocities and an algorithm for computing the 3-D parameters given two successive image frames. The solution is then improved by using a large number of image frames and exploiting the temporal coherence of 3-D motion. We use the ordinary differential which describe the parametric evolution in terms of the current motion/structure and the measurements in the image plane. The extended Kalman filter is then used to update the solution. The process is started by segmenting the entire scene into a number of planar surfaces and then applying the above technique to each surface under consideration the probable inconsistencies are then resolved.