The realization of color constancy on computer vision is important to recognize objects in varying light sources. This paper proposes a method to estimate the illuminant under the “Minimum Brightness Variance Assumption” which states that the variation of the brightness of the objects is as small as possible. In this method, the illuminant is estimated to be red when the red part of the object in the scene is bright. In detail, we define an evaluation function to calculate the variance of the brightness in the scene and we minimize the evaluation function to estimate the color of the illuminant and the color of the object. We conducted experiments with synthetic images and confirmed that the proposed method works well to reduce the influence of the illuminant for the objects in the scene.