From the actual implementation of displacement measurement in the laser synthetic wavelength interferometer, a nonlinear error analysis model was proposed by detecting the simultaneous zero-crossing position errors of two interference signals. By using this model, periodic nonlinear errors resulting from the polarizing leakage induced by the imperfection or misalignment of the polarizing beamsplitter in the basic configuration of the interferometer were analyzed. In order to reduce periodic nonlinear errors, two optimized optical configurations of the interferometer were presented. The simulations and experiments for verifying the performance of three configurations of the interferometer were performed. The experimental results demonstrate that the optimized third configuration has good stability, with a standard deviation of about 0.7 nm.