In Single Molecule Localization Microscopy (SMLM) emission spots are fitted with a Point Spread Function (PSF) model in order to find the position of the molecules. Recently Franke et al. [Nature Methods 2017] found that the use of a Gaussian PSF model can underestimate the photon count by up to 30%. In the presentation we elucidate the reasons for this underestimate. We show that it can be traced back to differences between the simplified Gaussian and the exact vectorial PSF, that takes all effects of high-NA, polarization, and interfaces between media into account. Especially spots captured under total internal reflection conditions show major deviations from the Gaussian spot shape. Deficiencies of other simplified PSF-models such as the low-NA scalar diffraction Airy distribution or the Gibson-Lanni model will be discussed too. Furthermore, we show a simulation study of the effects of aberrations on the photon count estimation. In particular, we will discuss the impact of spherical aberration due to refractive index mismatch. Finally, we show implementation issues and the impact on the fitting outcome of the use of the exact vectorial PSF model in combination with Maximum-Likelihood Estimation, building on the treatment of Smith et al. [Optics Express 2016].