Image intensity value is determined by both the albedo component and the shading component. The albedo component describes the physical nature of different objects at the surface of the earth, and land-cover classes are different from each other because of their intrinsic physical materials. We, therefore, recover the intrinsic albedo feature of the hyperspectral image to exploit the spatial semantic information. Then, we use the support vector machine (SVM) to classify the recovered intrinsic albedo hyperspectral image. The SVM tries to maximize the minimum margin to achieve good generalization performance. Experimental results show that the SVM with the intrinsic albedo feature method achieves a better classification performance than the state-of-the-art methods in terms of visual quality and three quantitative metrics.