A new infrared small target tracking method is proposed to track objects. First, the two-scale flux density calculating method based on the infrared orientation gradient feature of IR image is presented to extract feature of tracker, which is an effective method to solve the problem in feature extraction of tracking infrared small targets. Second, the least-square trajectory prediction algorithm is applied to deal with the difficulty of target loss in the tracking process. This algorithm makes full use of the continuity and direction of the target motion, avoids the interference of noise and achieves the prediction and discrimination of the target trajectory. The experimental results indicate that the proposed tracking algorithm is superior in precision and has less processing time compared with the contrast algorithms, so it is an efficient method for IR small target tracking in a complex background.