In recent years, domestic and foreign scholars have done a lot of research on the natural color image fusion technology; in which, color transfer method and the color lookup table method have been widely studied. The color transfer method needs a larger amount of calculation and the color consistency of fusion images is poorer, but the reference color images are obtained more easily. The color lookup table method is simple, real-time and can obtain better natural colors of fusion images. The image fusion method based on color lookup table and used dual-spectrum images’ gray value as the color index of fusion image, so an undesired influence on fusion image’s colors was caused by the instability of images’ gray value. A kind of improved color lookup table fusion method based on histogram matching was proposed. Firstly, mapping from dual-spectrum target images’ gray value to dual-spectrum reference images’ gray value was accomplished by utilizing histogram matching, thereby reduced the instability of gray value. Secondly, the color lookup table was complemented according to the principle that the closer the Euclidean distance is, the more similar the color is, in the range of dual-spectrum reference images’ gray scales. Finally, the fusion image was obtained from the histogram matched dual-spectrum target images by indexing the complemented color lookup table. The experimental results show that, the proposed method can effectively weaken the unexpected effect on fusion image’s colors caused by the instability of gray value and significantly improve the color fusion image’s visual effect. At the same time, the proposed method reduced portion which needs to be complemented of look-up table, so the improved method can decrease the lookup table’s data size, and increase efficiency.