Effectively size-dependent refractive index of ultrathin gold film deposited by chemical vapour deposition (CVD) is experimentally investigated at infrared wavelength. By coating the gold film on tilted fiber Bragg grating (TFBG), the wavelength and amplitude of the TFBG cladding modes are modulated by the interaction between their evanescent fields and the gold film. Then, the complex refractive index of the gold film in in-plane and out-of-plane directions can be calculated from the effective indices of the cladding modes with azimuthally and radically polarized electric fields at cladding boundary, respectively. The obtained real parts of the complex refractive indices are about 10 times higher than that of bulk gold, for the gold films with thickness from 6 to 65 nm, while the imaginary parts are 2 orders of magnitude lower than the bulk value in the both of directions. Based on the Atomic force microscope and scanning electron microscope images of the gold films with different thicknesses, the aggregation of gold nanoparticles (NPs) caused by high substrate temperature and low deposition rate is considered as the main contribution to the anomalous refractive indices.