A comprehensive study was performed to quantify anti-Stokes-fluorescence (ASF) cooling in fibers of various host compositions (telluride, fluorozirconates, fluorophosphates, phosphates, and chalcogenides) doped with Yb3+ or Er3+. Published expressions were used to calculate the maximum heat that can be extracted per unit length and time from a single-mode fiber in the limit of negligible absorptive loss, and the associated cooling efficiency. These expressions consider host- and ion-dependent parameters, namely the absorption and emission cross-section spectra, the radiative and nonradiative lifetimes, and the critical concentration for quenching. Using these expressions with published values for these parameters, the maximum extractable heat was calculated for a large-mode-area fiber (NA = 0.05) doped with either Yb3+ or Er3+ in a variety of hosts. The results show that for a given ion, the maximum heat that can be extracted depends strongly on the host due to the strong dependence of quenching on host composition. In contrast, the cooling efficiency (ratio of extracted heat to pump power absorbed) depends very weakly on the host. The cooling efficiency is also almost twice as high for Er3+ (average of 3.8%) than for Yb3+ (average of 2.2%) due to the larger gap between the pump and mean fluorescence energy in Er3+. Of the limited number of materials for which a full set of data was found in the literature, the highest extractable heat for Yb3+ is in phosphate (-51.5 mW/m), and for Er3+ is in chalcogenide (-10.3 mW/m). This work provides a simple methodology to evaluate the quantitative cooling performance of these and other rare-earth ions in any amorphous host, a procedure that should guide researchers in the selection of optimum materials for ASF cooling of fibers.