To remove noise from infrared thermal images captured in underground mining working face under low luminance and dusty environment, a nonreference infrared thermal image denoising method based on heuristic dual-tree wavelet thresholding is proposed. The threshold is optimized through estimating noise variance in wavelet domain using an improved chaotic drosophila algorithm (CDOA), which is promoted by a spatial–spectral entropy based metric. The basic DOA, genetic algorithm, particle swarm optimization algorithm, and virus colony search algorithm are implemented to compare the convergence rate and optimization ability of improved CDOA. Moreover, other representative denoising methods, such as BM3D, BLS-GSM, fast translation invariant, and nonlocal Bayes, are also applied for comparison. Comparison result proves effectiveness and superiority of the proposed method. Finally, the proposed method is applied in infrared thermograph-based visual surveillance system, and the denoising results also prove the state-of-art performance.