This paper describes a novel feature representation and selection approach for classification problems, especially for visual object detection within the framework of AdaBoost. This work is distinguished by two contributions. The first contribution is the introduction of a new feature generation and representation method called the spatial dependence matrix feature, which not only provides information related to the first-order statistics distribution of the object, but also gives some information about the relative positions within the object, more importantly, it can provide different degrees of importance for different discriminative parts within the object. It is flexible, extendable, and compatible with Haar-like features. The second contribution is an improved feature selection algorithm, which introduces a novel weighted features redundancy elimination rule that eliminates the irrelevant and redundant features from the candidate feature pool at every boosting stage when gradually training detector, and thus two advantages can be obtained: leading to selecting features with more discrimination and the final detector having a higher accuracy, and also increasing the learning convergence and achieving high training rates. Extensive experiments with synthetic and real scene data sets show that our works outperform conventional AdaBoost and are better than or at least equivalent to previously published results.