A segmentation-based video coding technique is presented. A change detector is used to find the regions of interest, i.e., moving areas, and then the moving areas are segmented based on the motion vector information. For the regions where the prediction error, defined by the difference between the original image and the motion-compensated image reconstructed by the motion vector, is large, the frame difference signal is used to supplement the segmentation results. Depending on the region characteristics, each region is represented by the motion vector and the frame difference signal. After postprocessing, the region information and the region boundaries are transmitted. Computer simulation with video sequences shows that the proposed algorithm reduces the trailing effect and gives better performance than the conventional method.