Fiber-optic image guides in confocal reflectance endomicroscopes introduce background backscatter that limits the achievable contrast in these devices. We show the dominant source of backscatter from the image guide is due to Rayleigh scattering at short wavelengths and terminal reflections of the fibers at long wavelengths. The effective Rayleigh scattering coefficient and the wavelength-independent reflectivity due terminal reflections are measured experimentally in a commercial image guide. The Rayleigh scattering component of backscatter can be accurately predicted using the fractional refractive-index difference and length of the fibers in the image guide. We also presented a simple model that can be used to predict signal-to-background ratio in a fiber-optic confocal reflectance endomicroscope for biologically relevant tissues and contrast agents that cover a wide range of reflectivity.