Image registration has been a hot research spot in the computer vision technology and image processing. Image registration is one of the key technologies in image mosaic. In order to improve the accuracy of matching feature points, this paper put forward the least square optimization in image mosaic based on the algorithm of matching similarity of matrices. The correlation coefficient method of matrix is used for matching the module points in the overlap region of images and calculating the error between matrices. The error of feature points can be further minimized by using the method of least square optimization. Finally, image mosaic can be achieved by the two pair of feature points with minimized residual sum of squares. The experimental results demonstrate that the least square optimization in image mosaic can mosaic images with overlap region and improve the accuracy of matching feature points.