In this paper, a color holographic display system based on the effective utilization of the spatial light modulator (SLM) is proposed. Color reconstruction is achieved by using a single SLM. The effective viewing area of three color reconstructed images is analyzed based on the geometrical optics principle and the holographic diffraction theory. Then the effective holograms of three colors are calculated based on the calculation of the effective area of the SLM. When three color parallel light sources are used to illuminate the SLM, color chromatic aberration is compensated by adjusting the blazing grating that loaded on the SLM. At last, three color reconstructed images can be coincided in the same position with a fast speed. The experimental results verify the feasibility of the proposed system.