Feature classification plays an important role in computer-aided diagnosis (CADx) of suspicious lesions or polyps in this concerned study. As one of the simplest machine learning algorithms, the k-nearest neighbor (k-NN) classifier has been widely used in many classification problems. However, the k-NN classifier has a drawback that the majority classes will dominate the prediction of a new sample. To mitigate this drawback, efforts have been devoted to set weight on each neighbor to avoid the influence of the “majority” classes. As a result, various weighted or wk-NN strategies have been explored. In this paper, we explored an alternative strategy, called “distance weighted inside disc” (DWID) classifier, which is different from the k-NN and wk-NN by such a way that it classifies the test point by assigning a corresponding label (instead a weight) with consideration of only those points inside the disc whose center is the test point instead of the k-nearest points. We evaluated this new DWID classifier with comparison to the k-NN, wk-NN, support vector machine (SVM) and random forest (RF) classifiers by experiments on a database of 153 polyps, including 116 neoplastic (malignance) polyps and 37 hyperplastic (benign) polyps, in terms of CADx or differentiation of benign from malignancy. The evaluation outcomes were documented quantitatively by the Receiver Operating Characteristics (ROC) analysis and the merit of area under the ROC curve (AUC), which is a well-established evaluation criterion to various classifiers. The results showed noticeable gain on the polyp differentiation by this new classifier according to the AUC values, as compared to the k-NN and wk-NN, as well as the SVM and RF. In the meantime, this new classifier also showed a noticeable reduction of computing time.