Color asymmetry is a common phenomenon in stereoscopic display system, which can cause visual fatigue or visual discomfort. When the color difference between the left and right eyes exceeds a threshold value, named binocular color fusion limit, color rivalry is said to occur. The most important information brought by stereoscopic displays is the depth perception produced by the disparity. As the stereo pair stimuli are presented separately to both eyes with disparities and those two monocular stimuli differ in color but share an iso-luminance polarity, it is possible for stereopsis and color rivalry to coexist. In this paper, we conducted an experiment to measure the color fusion limit for different disparity levels. In particular, it examines how the magnitude and sign of disparity affect the binocular color fusion limit that yields a fused, stable stereoscopic percept. The binocular color fusion limit was measured at five levels of disparities: 0, ±60, ±120 arc minutes for a sample color point which was selected from the 1976 CIE u'v' chromaticity diagram. The experimental results showed that fusion limit for the sample point varied with the level and sign of disparity. It was an interesting result that the fusion limit increased as the disparity decreases at crossed disparity direction (sign −), but there is almost no big change at uncrossed disparity direction (sign +). We found that color fusion was more difficult to achieve at the crossed disparity direction than at the uncrossed disparity direction.