In the view of space remote sensing，such as satellite detection，space debris detection etc.，visible band is usually used，in order to have the all-weather detection capability, long wavelength infrared (LWIR) detection is also an important supplement. However, in the tow wave band, the earth can be a very strong interference source, especially in the dim target detecting. When the target is close to the earth, especially the LEO target, the background radiation of the earth will also enter into the baffle, and became the stray light through reflection, the stray light can reduce the signal to clutter ratio (SCR) of the target and make it difficult to be detected. In the visible band, the solar albedo by the earth is the main clutter source while in the LWIR band the radiation of the earth is the main clutter source. So, in this paper, we establish the energy transformation from the earth background radiation to the detection system to assess the effects of the stray light. Firstly, we discretize the surface of the earth to different unit, and using MODTRAN to calculate the radiation of the discrete point in different light and climate conditions, then, we integral all the radiation which can reach the baffle in the same observation angles to get the energy distribution, finally, according the target energy and the non-uniformity of the detector, we can calculate the design requirement of the system stray light suppression, which provides the design basis for the optical system.