A new source coding algorithm is proposed which delivers an encoded bit stream that is scalable in both spatial resolution and frame rate. Motion compensated temporal filtering is combined with a spatial subband/wavelet pyramid to provide an efficient 3D multiresolution representation that yields perceptually acceptable low frame rate sequences. Error feedback hierarchical coding is used to eliminate the propagation of coding errors between spatio temporal resolutions in order to achieve near optimal results for each subvideo. The proposed order in of refinement addressees the non-commutative property of motion compensated temporal filtering and spatial subband/wavelet analysis. A new algorithm based on adaptive conditional arithmetic coding of quantizer significance maps is introduced and is shown to increase coding efficiency. Experimental results demonstrate a significant improvement in performance over other published algorithms. The algorithm is found to be more robust with respect to variations in the type of motion present in video data. The code-and-refine nature of the hierarchical algorithm makes it possible to use the algorithm as a scalable extension to other source coding algorithms. Finally, the complexity of the algorithm is modest and is well suited to a parallel implementation.