Long photon confinement and high optical fields require good optical resonators. Some of the best optical resonators with a small footprint are whispering gallery mode (WGM) resonators. Their principle is based on continuous total internal reflection at the interface of a round dielectric. Currently most WGM resonators are fabricated fully symmetric to their rotational axis. In WGM resonators fabricated from uniaxial crystals this symmetry axis then coincides with the optic axis, such that the modes are either parallel or perpendicular polarized. If the optic axis is however tilted with respect to the symmetry axis the polarization of the modes changes dramatically. We report on high Q resonances in a slightly birefringent MgF2 WGM resonator, cut at an angle of 20° with respect to the optic axis. A novel type of mode is observed that can be fully coupled (decoupled) with a right (left) hand circular polarized beam of light. Furthermore, the polarization properties at different outcoupling positions, determined via full Stokes measurements, are recorded and show a continuous complex change in ellipticity. We present the experimental results. Understanding the polarization behavior in an off-axis, birefringent WGM resonator may offer a new way for phase-matching in non-linear χ(2) materials.