A real time machine vision system for fruit quality inspection was developed, which consists of rollers, an encoder, a lighting chamber, a TMS-7DSP CCD camera (PULNIX Inc.), a computer (P4 1.8G, 128M) and a set of grading controller. The system was made for size detecting of fruit, and then sorting fruits into 3 groups by the skin color: red group, yellow group, and green group which was immaturity. Color model for segmenting fruits from background and classing fruits into different groups was discussed. RGB color model was used to segment fruits from background, an equation of red component and blue component was used to segment the figure of relationship between red and blue component into two zones, which represent background and a fruit respectively. And then HIS color model was introduced to class fruits into three groups, Hue component was used as the optimum feature for this objective because that there were less overlap on this component of the three groups.100 navel orange was used to class by their skin color, total error was 2.1%.