With the development of hyperspectral sensors over the last few decades, hyperspectral images (HSIs) face new challenges in the field of data analysis. Due to those high-dimensional data, the most challenging issue is to select an effective yet minimal subset from a mass of bands. This paper proposes a class-oriented model to solve the task of classification by incorporating spectral prior of the target, since different targets have different characteristics in spectral correlation. This model operates feature selection after a partition of hyperspectral data into groups along the spectral dimension. In the process of spectral partition, we group the raw data into several subsets by a hierarchy tree structure. In each group, band selection is performed via a recursive support vector machine (R-SVM) learning, which reduces the computational cost as well as preserves the accuracy of classification. To ensure the robustness of the result, we also present a weight-voting strategy for result merging, in which the spectral independency and the classification effectivity are both considered. Extensive experiments show that our model achieves better performance than the existing methods in task-dependent classifications, such as target detection and identification.