Indium gallium nitride (InGaN) semiconductor quantum dots are an attractive candidate for scalable room temperature quantum photonics applications owing to their large exciton binding energy and large oscillation strength. Previously, we reported single photon emission from site-controlled InGaN quantum dot structures. However, large homogeneous linewidth and significant non-radiative recombination were thought to be linked to the nearby surface charge centers. These charge centers can lead to spectral diffusion and excessive non-radiative recombinations at high temperature. In this work, ammonium sulfide passivation was investigated. Nitrogen vacancies were successfully passivated by ammonium sulfide ((NH4)2Sx) treatment, and the emission linewidth of a single quantum dot was reduced by 5 meV. Furthermore, the linewidth broadening with an increasing temperature was suppressed in the temperature range from 9 K to 95 K in this study. Satellite emission peak believed to be associated with the nitrogen vacancy was observed for un-passivated quantum dots. The satellite peak was 55 ~ 80 meV away from the main InGaN emission peak and was eliminated after sulfide passivation.