Correlation Filters (CFs) based trackers have recently attracted many researchers’ attention because of their high efficiency and robustness. Nevertheless, CFs trackers usually require a cosine window on account of the boundary effects. This allows trackers to distinguish targets in small background areas. In this paper, we propose an online learning algorithm that employs the global context to alleviate the problems. It is based on Passive-Aggressive algorithm that incorporates context information within CFs trackers. In addition, we train an SVM classifier to redetect objects in case of the model drift caused by occlusion and fast motion etc. The results of extensive experiments on a large-scale benchmark dataset show that the proposed tracker outperform the state-of-the-art trackers.