Feature extraction plays an important role in the tracking process. However, most attention is paid to the performance of the tracker without considering the sensitivity of feature to the environment. In this paper, the tracker based on the Kernelized Correlation Filter (KCF) is chosen to compare the tracking performance of different features such as the grayscale, the Histogram of Oriented Gradients (HOG) and the Color Names on scenarios with different attributes. The tracking accuracies corresponding to different features on sequences with various attributes are compared and validated through the OTB-100 dataset and the ALOV300++ dataset. The results show that the HOG gets the best tracking performance on image sequences with attributes such as background clutters, illumination variation, out-of-plane rotation, occlusion, deformation compared with grayscale and the Color Names. And the grayscale gets the best performance for motion blur, in-plane rotation. The Color Names obtains the best result with scale variation. And the reasons for performance differences between the three features are analyzed. It can be concluded that the accuracy of a tracker can be improved by choosing a proper feature according to the attributes of scenes.