Digital speckle correlation (DSC) solves the problem of searching corresponding points between two images, and it shows great application potential in pattern-projection based fast 3D shape measurement, because only one shot is enough to retrieve the 3D structure. As DSC relies on analyzing the spatial intensity distribution of a subset in image with a given point, it is likely to get false correspondences in low quality DSC area such as the background, because the searching range is hard to locate. So it is still hard to use DSC to realize fast 3D shape measurement. To solve this problem, the gray standard deviation of the subset is designed to recognize and remove the low-quality DSC area, and the principle of epipolar geometry and disparity constraint are utilized to determine the searching range, so the correspondences can be obtained. Moreover, in order to enhance the robustness of this method, a connected region method based on neighboring pixels possessing similar disparity is proposed to remove mismatched points after establishing initial disparity map by correspondences. Once the disparity map is obtained 3D structure can be retrieved based on the triangulation principle. The experiment on reconstructing Gorky plaster statue is performed, verifying that the proposed method can substantially reduce mismatched points and achieve robust single frame 3D shape measurement.