TERS has emerged over the past decade as a powerful tool for Raman spectroscopy that shows high sensitivity and capability of nano-scale imaging with high spatial resolution. TERS utilizes a metallic nano-tip, which confines and enhances the propagating light into near-field in the close vicinity of the apex. Besides the nano-scale spatial resolution, polarization analysis in TERS is of tremendous advantage, as it allows one to study highly directional intrinsic properties of a sample at the nanoscale. In this study, we have developed a method to analyze the polarization of near-field light in TERS from the scattering pattern produced by the induced dipole in the metallic tip. Under dipole approximation, we measured the image of the dipole at a plane away from the focal plane, where the information about the direction of the dipole oscillation was intact. The direction of the dipole oscillation was determined from the defocused pattern, and then the polarization of near-field light was evaluated from the oscillation direction by calculating the intensity distribution of near-field light We used those evaluated tips to measure nano-images from single-walled carbon nanotubes and confirmed that the contrast of the TERS image depended on the oscillation direction of the dipole, which were also found in excellent agreement with the calculated TERS images, verifying that the polarization of the near-field was quantitatively estimated by our technique.