This paper presents a method to segment moving human bodies. A self-adaptive background model is used to update the background image(so-called reference image). By calculating the Euclidean distance of corresponding points in the current and background image, we can check out the foreground objects. And the shadow can be detected and removed according to the characteristics of the shadow regions shown in HSV space. Finally, target tracking is implemented by calculating the relativity of color histogram between the moving areas in two succeeding images.