We studied the statistical learning methods with imbalanced training data sets. Imbalanced training sets are very common in industrial machine vision applications. The minority class contains the defects or anomalies we try to catch. The majority class contains the "regular" objects. We need a method that performs well at both false positive and false negative error rates. Traditional methods such as classification tree yield unsatisfactory results. We propose a two-stage classification scheme. We first use a subset selection method to remove redundant examples from the majority class. As a result, the training sample becomes more balanced without losing critical boundary information. The computation-intensive methods such as boosted classification trees are then applied to further improve both error rates.