Creating a visual codebook is an important problem in object recognition. Using a compact visual codebook can boost computational efficiency and reduce memory cost. A simple and effective method is proposed for visual feature codebook construction. On the basis of a feedforward hierarchical model, a robust local descriptor is proposed and an a priori statistical scheme is applied to the class-specific feature-learning stage. The experiments show that the proposed approach achieves reliable performance with shorter codebook length, and incremental learning can be easily enabled.