We usually recognize color by two kinds of processes. In the first, the color is recognized continually and a small difference in color is recognized. In the second, the color is recognized discretely. This process recognizes a similar color of a certain range as being in the same color category. The small difference in color is ignored. Recognition by using the color category is important for communication using color. It is known that a color vision defect confuses colors on the confusion locus of color. However, the color category of a color vision defect has not been thoroughly researched. If the color category of the color vision defect is clarified, it will become an important key for color universal design. In this research, we classified color stimuli into four categories to check the shape and the border of the color categories of varied color vision. The experimental result was as follows. The border of protanopia is the following three on the CIE 1931 (x, y) chromaticity diagram: y = -0.3068x + 0.4795, y = -0.1906x + 0.4021, y = -0.2624x + 0.3896. The border of deuteranopia is the following three on the CIE 1931 (x, y) chromaticity diagram: y = -0.7931x + 0.7036, y = -0.718x + 0.5966, y = -0.6667x + 0.5061.