In this paper, we present an adaptive algorithm to improve the quality of millimeter-wave video sequence by separating
each video frame into foreground region and background region, and handle them differently. We separate the
foreground from background area by using an adaptive Kalman filter. The background is then denoised by both spatial
and temporal algorithms. The foreground is denoised by the block-based motion compensated averaging, and enhanced
by wavelet-based multi-scale edge representation. Finally further adaptive contrast enhancement is applied to the
reconstructed foreground. The experimental results show that our algorithm is able to produce a sequence with smoother
background, more reduced noise, more enhanced foreground and higher contrast of the region of interest.