A new approach for scaling, rotation, and translation invariant object recognition is proposed. The geometric discrepancy between the test pattern and the related reference pattern can be reduced by applying affine transform to the test pattern with the parameters estimated from the affine relation between the control points of both parties. The automatically paired control points are the centroids of patterns’ desired regions with specific gray level distributions. By means of the analysis of intrinsic invariance properties, a small number of reference templates are required to deal with the residual geometric variation of the transformed test pattern. Consequently, a simple similarity measure can be employed directly without any scale and rotation invariant feature extraction.