A phase-recovery method based on iterative least-square fitting is proposed to overcome off-axis phase-shifting errors of high numerical-aperture (NA) surfaces in tilt phase-shift interferometry. Vibration of test optics having optical powers makes nonuniform off-axis phase-shift errors over the aperture, which is severe when testing optics with high NA. The proposed method calculates the test wavefront, off-axis phase shift, and tilt phase shifts in x- and y-directions within four separate least-square fitting steps iteratively. Parabolic and linear regressions are used for extracting off-axis and tilt phase-shift errors, respectively. The proposed method is verified with simulations and experiments with a Fizeau interferometer having F/0.75 transmission sphere. This method can be used with a temporal phase-shifting interferometers for testing high-NA optics in the presence of vibration.