We have demonstrated a color recognition system composed by a novel, self-organized optical neural network system that includes genetic algorithms along with back propagation schemes, which make it possible that the system avoids the local minimum problems and make the learning processes faster and better. Our system is composed of a color liquid crystal display panel (LCD), bistable semiconductor lasers, photo-diodes, and liquid crystal light projector (LCP). The LCD weighs the intensities of light that passes through it and works as synapses in the neural network. The optical bistable semiconductor lasers originate the optical sigmoid functions and serve as threshold processing units. Using these devices can simplify the configuration of the optical neural network system. The color of light emitted from the LCP will be recognized by the neural network system. A monochromatic light beam generated by the LCP is illuminated on all over the LCD surface displaying the colored boxes in the three primary components. Thus, the light beam is weighed when it passes through the boxes on the LCD. As a consequence, we have achieved a novel, simplified color recognition system using the genetic algorithms for self- organization of the optical neural network. The unique feature of this system is to make use of the genetic algorithms and the back propagation at the same time to derive selectively the merits from these two methods. By this system, more naturalized color recognition like human will be performed, being able to distinguish the colors under different conditions of environment, e.g. lightening conditions, surface conditions of colored material, etc.