The basic computational module of the technique is an old pattern recognition procedure: the mean-shift. In case of gray level feature domain, the spatial information of the target is lost when the background brightness histogram is the same as the target histogram. In this paper, we propose a new algorithm that is independent of background contrast by changing features from a conventional brightness based histogram to a temperature- based histogram. The proposed algorithm can track targets robustly regardless to target-background contrast. The experiment results demonstrate that the temperature-based Mean-Shift outperforms comparing with the brightness-based Mean-Shift when track a object with successive background variations.