Hyperspectral image (HSI) classification is one of the significant research topics in the remote sensing community. The high dimensionality of the hyperspectral data, the high correlation among pixels, and the availability of fewer numbers of training samples affect the HSI classification accuracy. We propose an approach to extract the best representative bands from the high-dimensional imagery for better classification. Initially, the spectral bands are extracted by re-representing the traditional principal component analysis in terms of Hebbian learning, formulated and solved as a fuzzy optimization problem. Next, a spatial filter is applied to these spectral bands to obtain the smoothed image that preserves the spatial details. Finally, the spectral and spatial features are trained with the nonlinear support vector machine with the radial basis function kernel to obtain the classification map. Performance of the proposed approach is tested by varying different values of the parameters used in our model. The classification accuracy of the proposed approach is compared with the state-of-the-art techniques, which proves the effectiveness of the proposed methodology. The proposed approach can be applied in real-world applications, such as food quality, environment change detection, mineralogy, and pharmaceutical drug design.