A simple but effective filtering scheme in phase shifting interferometry is proposed. Conventionally, either the phase-shifted fringe patterns, or the resulting wrapped phase field, is chosen for filtering to remove noise. In this letter, we propose filtering the complex field resulting from the combination of the phase-shifted fringe patterns, as a third choice. The filtering can be realized by Fourier transform or windowed Fourier transform. Processing results of speckle fringes are given to show its effectiveness.