The concept of exponential averaging of a reconstructed wavefront is extended by considering the full process of image formation in digital holographic microscopy (DHM), including object illumination, optical imaging, reference wave, and hologram sampling. Phase filtering by exponential averaging uses the whole capability of DHM to retrieve both the amplitude and phase of a wavefront. The aim is to apply the exponential filtering to the spatial distribution of the complex reflection coefficient of the object surface rather than to the whole orthoscopically reconstructed wavefront. To identify possible contributions to errors in the weighting of the exponential averaging, the phase measurement process of DHM is described as a signal transmission path. Accordingly, the orthoscopic wavefront needs to be compensated for both amplitude and phase of the illuminating wave and of the reference wave as closely as possible. As hologram sampling can introduce spectral amplitude attenuation, its numerical compensation is also proposed. Finally, nonlinear amplitude weighting is proposed in exponential averaging. For a better understanding of the physical meaning of weighting and its particular importance for measuring rough object surfaces, the effect of object roughness on an imaged object wavefront is presented by the concept of optical convolution.