Object tracking is widely used in many applications such as intelligent surveillance, scene understanding, and behavior analysis. Graph-based semisupervised learning has been introduced to deal with specific tracking problems. However, existing algorithms following this idea solely focus on the pairwise relationship between samples and hence could decrease the classification accuracy for unlabeled samples. On the contrary, we regard tracking as a one-class classification issue and present a novel graph-based semisupervised tracker. The proposed tracker uses linear neighborhood propagation, which aims to exploit the local information around each data point. Moreover, the manifold structure embedded in the whole sample set is discovered to allow the tracker to better model the target appearance, which is crucial to resisting the appearance variations of the object. Experiments on some public-domain sequences show that the proposed tracker can exhibit reliable tracking performance in the presence of partial occlusions, complicated background, and appearance changes, etc.