In recent years, fringe projection profilometry (FPP), as a kind of three-dimensional shape measurement technology, has achieved the great breakthrough, due to the rapid development of the high-speed camera and high-speed projection equipment. The number-theoretical approach, as a classical method for the temporal phase unwrapping algorithm, is suitable for the binary defocusing FPP since it can avoid the acquiring of low frequency fringes. However, in order to ensure the stability of phase unwrapping, the period of fringe is generally around 20, which leads to the limited accuracy of 3D measurement. In this paper, we propose a bi-frequency number-theoretical phase unwrapping method with depth constraint. Using the principle of depth constraint, we will eliminate the period ambiguities of each pixel within a pixel-variant local period range so that the method only requires the coprime of two fringe frequencies within the local period range instead of the conventional global range. In this way, the requirement of stability of the traditional number-theoretical phase unwrapping can be adjusted from global range to local range. The stability is higher in the local period range due to containing less period ambiguities. As a result, we can realize phase unwrapping of higher frequency fringes with the same stability. Several experiments on various scenes are performed, verifying that our method can achieve high-speed and high-precision 3D measurement.