We propose a technique for the segmentation of color map images by means of an algorithm based on fuzzy clustering and prototype optimization. Its purpose is to facilitate the extraction of lines and characters from a wide variety of geographical map images. In this method, segmentation is considered to be a process of pixel classification. The fuzzy c-means clustering algorithm is applied to a number of training areas taken from a selection of different color map images. Prototypes, generated from the clustered pixels, that satisfy a set of validation criteria are then optimized using a neural network with supervised learning. The image is segmented using the optimized prototypes according to the nearest neighbor rule. The method has been verified to work efficiently with real geographical map data.