Diffuse optical imaging can be used to probe highly scattering media like biological tissue down to a depth of few centimeters, with spatial resolution limited by light scattering. Its combination with ultrasound imaging can potentially lead to medical imaging systems with, for instance, high specificity in the examination of tumors. However, the presence of the ultrasound coupling gel between probe and tissue can have detrimental effects on the accuracy of optical imaging techniques. Here we present an experimental study on the effect of ultrasound coupling fluids on diffuse optical spectroscopy (DOS) and diffuse correlation spectroscopy (DCS). We demonstrate on tissue-mimicking phantoms that the use of standard water-clear gels, providing a direct path for the light from the source to the detection point, can distort optical measurements generating strong underestimation of both the absorption and the reduced scattering coefficients in DOS measurements, as well as underestimation of the Brownian diffusion coefficient in DCS measurements. On the contrary, various turbid fluids demonstrate excellent performance in preventing this issue.