Aerial surveillance is a main functionality of UAV, which is realized via video camera. During the operations, the mission assigned targets always are the kinetic objects, such as people or vehicles. Therefore, object tracking is taken as the key techniques for UAV sensor payload. Two difficulties for UAV object tracking are dynamic background and hardly predicting target’s motion. To solve the problems, it employed the particle filter in the research. Modeling the target by its characteristics, for instance, color features, it approximates the possibility density of target state with weighting sample sets, and the state vector contains position, motion vector and region parameters. The experiments demonstrate the effectiveness and robustness of the proposed method in UAV video tracking.