Mean shift is a traditional moving target tracking algorithm, which has some deficiencies such as: A tracking window of a target needs to be initialed manually in the first frame; the window size cannot be adaptively changed according to a moving object in the process of tracking; if a target is obscured, it might be lost in the tracking window. In order to solve these problems, a method combining Kalman filter and Scale and Orientation Adaptive Mean Shift Tracking (SOAMST) is proposed. Firstly we use Kalman filter to locate a moving target at the beginning. Then the ratio of the first order moment to the zero order moment is used to estimate its center, and the second order center moment is used to estimate its size and orientation. Meanwhile, whether the target is obscured is determined by the Bhattacharyya coefficient based on the target model and a candidate one. A candidate model is more similar to the target and the estimation result of the target is more reliable when the Bhattacharyya coefficient is closer to 1. On the contrary, if the Bhattacharyya coefficient decreases to 0, the target will be lost for being totally obscured. If the target is partially obscured or not obscured, SOAMST is used directly to track the target; if totally obscured, Kalman filter is imposed to estimate the location of the target in the next frame before SOAMST. The experiments show that the proposed algorithm can track a moving target automatically at the initial frame without prior knowledge. It can also track a completely obscured target accurately by Kalman filtering based location estimation.