Metal nanoparticles (NPs) have wide applications in various fields due to their unique properties. The accurate and fast characterization of metal NP concentration is highly demanded in the synthesis, metrology, and applications of NPs. The commonly used inductively coupled plasma mass spectrometry (ICP-MS) is a standard method for measuring the mass concentration (MC) of NPs, even though it is time-consuming, expensive, and destructive. While for the number concentration (NC) characterization of NPs, the method is less explored. Here, we present an improved optical extinction-scattering spectroscopic method for the fast, non-destructive characterization of the MC and NC of poly-disperse metal NP colloid simultaneously. By measuring the extinction spectrum and the 90° scattering spectrum of the nanorod (NR) colloid, we can solve an inverse scattering problem to retrieve the two dimensional joint probability density function (2D-JPDF) with respect to the width and the aspect ratio of NR sample accurately, based on which the NC and MC of the colloidal NPs can be calculated. This method is powerful to characterize both the geometric parameters and the concentrations, including the MC and NC, of poly-disperse metal NPs simultaneously. It is very useful for the non-destructive, non-contact, and in-situ comprehensive measurement of colloidal NPs. This method also has the potential to characterize NPs of other shapes or made of other materials.