In the field of exoplanetary sciences, high contrast imaging is crucial for the direct detection of, and answering questions about habitability of exoplanets. For the direct imaging of habitable exoplanets, it is important to employ low inner working angle (IWA) coronagraphs, which can image exoplanets close to the PSF. To achieve the full performance of such coronagraphs, it is crucial to correct for atmospheric dispersion to the highest degree, as any leakage will limit the contrast. To achieve the highest contrast with the state-of-the-art coronagraphs in the SCExAO instrument, the spread in the point-spread function due to residual atmospheric dispersion should not be more than 1 mas in the science band. In a traditional approach, atmospheric dispersion is compensated by an atmospheric dispersion compensator (ADC), which is simply based on model which only takes into account the elevation of telescope and hence results in imperfect correction of dispersion. In this paper we present the first on-sky closed-loop measurement and correction of residual atmospheric dispersion. Exploiting the elongated nature of chromatic speckles, we can precisely measure the presence of atmospheric dispersion and by driving the ADC, we can do real-time correction. With the above approach, in broadband operation (y-H band) we achieved a residual of 4.2 mas from an initial 18.8 mas and as low as 1.4 mas in H-band only after correction, which is close to our science requirement. This work will be valuable in the field of high contrast imaging of habitable exoplanets in the era of the ELTs.