In this paper, we propose a new segmentation method aimed at separating the moving objects from the background in a generic video sequence using Cellular Neural Networks (CNN). This task may be accomplished to support the functionalities foreseen by new multimedia scenarios, and in particular the content-based functionalities focused by the MPEG-4 activity. Extraction of motion information from video series is very power consuming, the proposed scheme extracts moving objects based on both motion and spatial information. Initially, a symmetrical inter-frame difference is performed on a group of gray image, so the approximate area of the video object was presented, then this area can be divided into some flat zones with uninterrupted grey scale information. Finally some zones are merged and forming the object according to a certain rule, others are discarded. It is the case of stationary background hereinbefore, in the case of moving, we will do some motion estimation at first. For the good of laborsaving, some work will be realized by CNN,. At the end of this paper, some typical results obtained on MPEG-4 sequences are here shown, in order to illustrate the segmentation algorithm performance using Aladdin V1.3 simulator system.