We present a novel algorithm based on nonnegative least square (LS) estimation for visual object tracking. Most existing algorithms, such as template matching, only consider each candidate separately. We offer another, new perspective. The template model is approximated by the linear combination of candidates in the space spanned by the feature vectors associated with particles. The coefficient of the linear combination is achieved through solving a nonnegative LS problem, and then it is used to evaluate the similarity between the template and candidates. Experimental results demonstrate that the proposed algorithm has better tracking accuracy than the compared algorithms and strong robustness against noise.