This paper presents the algorithm research of real time human tracking in video surveillance. This system consist of shadow detection, moving human identify and false object detection. We can identify the shadow from the foreground according to the changes of the illumination and the color changes when a point covered by a shadow gets darker. Through the step of the moving human identify, we use the codebook to classify the human from other objects. This algorithm is to find a code vector in codebook with the minimum distortion to the feature vector of object. If the minimum distortion is less than a threshold, this object is human. The based assumption of false object detection method is that object boundaries coincide with color boundaries. We find the false object which burst upon in the scene, and then use motion analysis to verify whether or not the tracked subject is indeed a human. The experiment result proved that the algorithms for real time moving human tracking are effective and robust.