Three-dimensional digital image correlation (3D DIC), which combines binocular stereo vision and digital image cross-correlation matching technology, can be used to restore the three-dimensional and deformation information of the object under test. 3D DIC can be accomplished by matching the subset in the left(right) image with that in the right(left). The size of the matching window is found to be critical to the measurement accuracy. Nevertheless, when the subset is small, the measurement accuracy and resolution are high but very sensitive to noise. In contrast, for large subset the measurement accuracy and resolution are lower, while the measurement is more robust to noise. To combine the advantages of high precision and robustness, the Spatio-temporal cross-correlation method is proposed in this paper. A set of speckle patterns are projected onto the objects under test. Instead of the way constructing subsets from spatial neighbor points, the way used in conventional DIC, both spatial and temporal neighbor points are utilized to construct subsets with rich information and strong characteristics. To implement the proposed scheme, we use a mechanical galvanometer to realize the projection of sequential speckle patterns and construct a stereo vision system to realize the three-dimensional reconstruction. The sub-pixel matching algorithm is used to improve the accuracy of stereo matching and 3D reconstruction. Simulations and experiments are carried out to verify the feasibility and success of the proposed scheme and system.