We report on the characterization of oxyfluoride glasses and glass ceramics for their application in optical refrigeration. Oxide glasses are chemically and mechanically stable and relatively ease to handle and fabricate, but their high maximum phonon energy leads to a nonradiative decay rate which is unacceptable for optical refrigeration. On the other hand, low-maximum phonon energy hosts such as fluorides lack the desirable mechanical and chemical stabilities to make them widely used. The combination of the high chemical and mechanical stability of oxides and the low maximum phonon energy of fluorides make oxyfluorides strong potential candidates for wide-spread use in optical refrigeration. Glasses and ultra-transparent glass-ceramics of molar composition 30SiO2-15Al2O3-(27-x)CdF2-22PbF2-4YF3-xYbF3, with x = (2, 5, 8, 12, 16 and 20) mol % are investigated. The absorption and photoluminescence spectra, as well as the lifetime and the external quantum efficiency of the photoluminescence for these samples using an integrating sphere are reported. The effects of reabsorption on the measured mean fluorescence wavelength are also reported. The cooling efficiencies of the samples were measured as a function of the pump wavelength using a calorimetric method with a Ti:Sapphire laser pump source and a fiber Bragg grating sensor for a direct temperature measurement. Impurities and background absorption are also investigated using different pump sources and the calorimetric method. From a comparison of the cooling/heating performance of the oxyfluoride glasses and glass-ceramics containing various Yb3+ amounts, we developed a strategy to realize and enhance optical refrigeration in this class of material.