The typical probability based point pattern matching method is coherent point drift (CPD) algorithm, which treats one point set as centroids of a Gaussian mixture model, and then fits it to the other. It uses the expectation maximization framework, where the point correspondences and transformation parameters are updated alternately. However, the anti-outlier performance of CPD is not robust enough as outliers have always been involved in the operation until the CPD converges. Hence, an automatic outlier suppression (AOS) mechanism is proposed. First, outliers are judged by a matching probability matrix. Then, transformation parameters are fitted using accurate matching point sets. Finally, the Gaussian centroids are forced to move coherently by this transformation model. AOS-CPD can efficiently improve the anti-outlier performance of rigid CPD. Furthermore, CPD is applied to image matching. A new local changing information descriptor-relative phase histogram (RPH) is designed and RPH-AOS-CPD is proposed to embed RPH measurement into AOS-CPD as a constraint condition. RPH-AOS-CPD makes full use of grayscale information besides having an excellent anti-outlier performance. The experimental results based on both synthetic and real data indicate that compared with other algorithms, AOS-CPD is more robust to outliers and RPH-AOS-CPD offers a good practicability and accuracy in image matching applications.