A large range multi-functional metrological atomic force microscope based on optical beam deflection method has been set up at NIM one year ago. Being designed intended to make a traceable measurement of standard samples, the machine uses three axes stacked piezoceramic actuators, each axis with a pair of push-pull piezo operated at opposite phases to make orthogonal scanning with maximized dimensional up to 50×50×2mm3. The stage displacement is measured by homodyne interferometer framework in x,y,z direction, from which beams are aligned to intersect at cantilever tip to avoid Abbe error, an eight times optical path multiplier interferometer mirror is researched to enhance fringe resolution. There is also a new compact AFM head integrated with LD, quadrant PD, cantilever, optical path and microscope, the head uses special track lens group to guarantee laser spot focused and static on the back of the cantilever, no matter whether or not the cantilever have lateral movements; similarly, reflect beam also focused and static in the center of quadrant detector through convergence lens group, assumed no cantilever bending on vertical direction. Attribute to above design, the AFM have a resolution up to 0.5nm. In the paper, further improvement is described to reduce the influence of parasitic interference caused by reflection from sample surface using laser multimode modulation, the results shows metrological AFM have a better performance in measuring step, lateral pitch, line width, nanoroughness and other nanoscale structures.