In this paper an algorithm is presented to extract the valid depth data and correct the values of flying pixels by using depth information and confidence image. An adaptive segmentation for the measured depth image is executed based on kernel density estimation and one-pass connected component labeling. Then a modified structure tensor is used to detect the invalid pixels and the flying pixels contained in the depth image. Finally these pixels are corrected with the bi-cubic interpolation method or selectively removed by voting operation. And also, the erroneous pixels are excluded with augmented confidence. Experimental results have demonstrated the effectiveness of our algorithm.