This paper describes a new spatial-spectral feature extraction and selection technique for analysis and segmentation of the color images of natural scenes. It is a statistical-structural method which is developed by modeling several basic scene patterns such as uniformly colored object regions, textured image areas, and shadowed or highlighted image segments. The uniformly colored object parts are characterized by their spatial continuity property. The textured areas are identified by their spatial placement rules and spectral primitives. The shadowed and highlighted image segments are determined by their constant chromaticity and gradually varying lightness property. The basic rule used in this modelling process is the Julesz conjecture. The features are extracted by a simple averaging process in the local areas of these visual patterns, which are determined by the algorithm. This averaging process implicitly incorporates the spatial and spectral information contained in the local areas. Thus the extracted feature set is expected to provide better clustering in the feature space than the sensory data alone.