A spectral-domain interferometric technique using the interference of polarization modes of a highly birefringent (HB) elliptical-core fiber to measure the temperature is presented. The method is based on the wavelength interrogation, i.e., the position of a chosen spectral interference maximum as a function of temperature is measured. Temperature sensing is carried out in a range from 300 to 370 K in an experimental setup comprising a white-light source, a polarizer, a delay line, a sensing HB fiber, an analyzer and a spectrometer. As the delay line, a birefringent quartz crystal of a suitable thickness is utilized to resolve a channeled spectrum in a range as wide as possible. A part of the sensing HB fiber, which is placed in a chamber, is exposed to temperature changes, and first, the polarimetric sensitivity to temperature is measured. It is revealed that the HB fiber is suitable for temperature sensing at a wavelength of 600 nm. Second, the shift of the wavelength position of the chosen spectral interference maximum with temperature is measured. It is revealed that the temperature sensitivity is higher at shorter wavelengths.