In breast imaging by ultrasound CT, ultrasound is refracted owing to the difference of the sound speed between the breast and background water. The sound speed of a dense breast is higher than that of the water, while that of a fatty breast is lower than that of the water. In this study, we developed an oil-gel-based phantom for mimicking the wave refraction from the fatty breast to the dense breast. An oil gel was generated by adding SEBS (Styrene-Ethylene/Butylenes-Styrene, 10 wt%) to paraffin oil. The oil-gel-based phantom has a cylindrical shape and contains rod shaped inclusions which can be filled with salty water (3.5%). When temperature increases, the sound speed of water increases, while that of the oil gel decreases; the sound speeds of the oil gel were higher than those of the water at less than 20°C, while the sound speeds of the oil gel were lower than those of water at higher than 20°C. By controlling the temperature, the oil-gel-based phantom was able to simulate the refraction from the fatty breast (1476 [m/s]) to the dense breast (1559 [m/s]). For 43 days, the variation of the sound speed and attenuation of the oil gel in the reconstructed images were 0.7[m/s] and 0.03[dB/MHz/cm], respectively. This phantom with high temporal stability is suitable for multi-center distribution and may be used for standardization of data acquisition and image reconstruction across centers.