For the purpose of coloring the night-vision images captured by low-light image intensifiers or infrared thermal imagers, color transfer algorithms were used to transfer natural colors to these gray images. Most of the color transfer algorithms can be divided into two classes: global color transfer and point color transfer. In global color transfer algorithms, the means and variances of the initial false color image were adjusted according to those of the reference color image. In point color transfer algorithms, the matching points were determined between the grayscale image and the reference color image. These two kinds of algorithms are always carried out in two common color spaces: YUV color space and Lab color space. The color space influences the performance of the color transfer algorithms. In this paper, several typical color transfer algorithms, including basic ones and multi-resolution ones, were carried out in different color spaces. The results show that global color transfer algorithms perform better in the YUV color space and the Lab space is more suitable for point color transfer algorithms. The biggest difference between these two color spaces is that the correlation between the channels of Lab space is much lower than that of YUV space. The global color transfer algorithms adjust the color components of the initial false color image with a uniform conversion, linear or non-linear ways. This process can benefit form the correlation between the channels, which is much higher in YUV space. However, the coloring process of the point color transfer algorithms is independent from the points matching process based on grayscale. This is the reason why the point color transfer algorithms should be implemented in the Lab space.