To solve the problem of the small field of the view caused by CCD in the process of the hologram record, the hologram mosaic algorithm based on the Harris corners is proposed. The Harris corners in multi-scale are extracted and the mismatching points are removed. The final homography is calculated by using the improved RANSAC algorithm based on L-M algorithm. Finally, the stitched hologram with high quality can be obtained based on the weighted average fusion algorithm. It can overcome the influence to the hologram that the incident angles of the object beam are not consistent. Two experiments carried out with different reconstructed distance demonstrate that the proposed algorithm can realize the measurement of the big object by using the hologram method. Furthermore, it has high accuracy and strong robustness.