Single-nanoantenna has intrigued vast interest due to its exceptional properties such as light harvesting and field enhancement, which provide the opportunities for strengthening light-matter interaction and efficient photon manipulation in nano-scale, as well as boosting nonlinear response. On the other hand, materials with structural or electronic phase transition have been employed to achieve large optical modulation contrast and order-unity switching, making them promising building blocks for high-performance optical circuits and devices with ultra-small footprint. In this context we demonstrate nano-scale all-optical modulation with single Au antennas fabricated on phase-transition material vanadium dioxide (VO2) substrate. VO2 films are deposited on boroaluminosilicate glass coated with a 30-nm layer of fluorine-doped tin oxide. The inclusion of this intermediate layer allows the production of VO2 films with low surface roughness and suitable thermochromic transition temperature. Then the nanoantennas are fabricated by e-beam lithography and subsequent 45-nm-thick gold deposition on the VO2 substrate. A 5-nm-thick Ti layer is used to improve the adhesion of the gold to the VO2. We use a pump-probe spectroscopy to characterize the modulation feature of the antenna/VO2. The pump beam at 1060 nm wavelength is used to introduce a local heating for VO2's phase transition and the probe beam from 1100 nm to 2000 nm wavelength is for readout of the modulated local transmission of antenna/VO2 hybrid owing to the dielectric environment change. A spatial modulation technique is also used to extract the differential transmission (ΔT/T) around the antennas. As a result, with pump pulse energy increasing to less than 1 nJ, the measured ΔT/T of single-antenna//VO2 hybrid exhibits substantial change that crossing the zero line and significant blue shift. As reported the ΔT/T obtained from spatial modulation spectroscopy is supposed to be proportional to the antenna’s extinction cross section. However, with the obtained negative values which lead to unphysical extinction cross sections less than 0, we believe the VO2 substrate beneath the antennas is highly involved as its optical property has been modified considerately. In addition, we observe that the pump-modulated differential transmission of the antenna/VO2 hybrid evidently depends on the polarisation of the pump when it is below a certain level. In this regime, the parallel pumping excites the longitudinal resonant mode while the perpendicular one only induces non-resonant absorption of antenna’s transverse mode. Going beyond this regime, the stronger pump transits the VO2 substrate from insulating phase into metallic phase completely, which dominates the dielectric environment change of the antenna, leading to nearly polarisation-independent modulation. The time for fully switch-on obtained from the pump-probe measurement is less than 50 ps. We also investigate the time response of the differential transmission dependent on the pulse repetition rate and substrate temperature, respectively. Less modulation depth with repetition rate over 2 MHz or base temperature higher than 40 °C suggest that the heat accumulation from adjacent pulses and thermal equilibrium time plays important roles in the achievable modulation speed. The single-antenna/VO2 structure may find applications in nano-scale optoelectronics for multiple functionalities including modulation, memory and so on.