This paper proposes and experimentally demonstrates a new denoising and hole-filling algorithm through discrete points removal and bilinear interpolation based on the bi-material cantilever FPA infrared imaging system. In practice, because of the limitation of FPA manufacturing process and optical readout system, the quality of obtained images is always not satisfying. A lot of noise and holes appear in the images, which restrict the application of the infrared imaging system. After analyzing the causes of noise and holes, an algorithm is presented to improve the quality of infrared images. Firstly, the statistic characteristics such as probability histograms of images with noise are analyzed in great detail. Then, IR images are denoised by the method of discrete points removal. Second, the holes are filled by bilinear interpolation. In this step, the reference points are found through partial derivative method instead of using the edge points of the holes simply. It can detect the real points effectively and enable the holes much closer to the true values. Finally, the algorithm is applied to different infrared images successfully. Experimental results show that the IR images can be denoised effectively and the SNRs are improved substantially. Meanwhile, the filling ratios of target holes reach as high as 95% and the visual quality is achieved well. It proves that the algorithm has the advantages of high speed, great precision and easy implement. It is a highly efficient real-time image processing algorithm for bi-material micro-cantilever FPA infrared imaging system.