The existence of self-generated thermal radiation in infrared optical systems exhibits a great impact to the extraction of target signal and further degrades the signal-to-noise ratio (SNR), thus making the self-generated thermal radiation one of the important factors affecting the detective property. In this paper, a refraction-reflection optical system has been taken as an example and the three-dimensional simulation model has been built up using the ASAP optical analysis software. On this basis, the influence of the surface roughness, the level of the optics contaminated by the particles with the uniform and non-uniform distributions, the treatment of the mechanical surface (such as blacking, polishing, roughening) on the self-generated thermal radiation have been focused on discussion. Moreover, the thermal radiation of the system has been evaluated by the effective emissivity. The results indicate that the effective emissivity varies with different surface treatment. The self-generated thermal radiation is more and more serious with the increasing of the effective emissivity, resulting in great difficulty in obtaining and analyzing the target signal. It follows that the surface treatment of components exhibits a significant effect on the stray radiation performance in infrared optical systems. Consequently, appropriate treatments should be taken to diminish the self-generated thermal radiation in order to meet the requirements of the stray radiation performance in practical applications.