The resolution of a new subdivision technique proposed in our previous studies, directly subdividing grating stripes by using the combination of CMOS microscopic imaging and image processing, was theoretically discussed and experimentally tested. The relations of resolution to the parameters, such as subdivision number, grating period, magnifying power, included angle, etc., were discussed in theory on the basis of resolution equations derived from subdivision principle and the object - image relation. And it was concluded that the resolution is higher as the subdivision number is more, the grating period is shorter, the magnifying power is higher or the tilt angle is smaller. Five subdivision systems with various resolutions were constructed with two kinds of metrological gratings and CMOS arrays and their performances were tested on an Abbe comparator with a minimum scale mark of 1 μm in the distance range of one grating period. The conclusion that subdivision system performs better under the higher resolution can be drawn from experimental results. At present, the maximum displacement error is 0.2 μm for the subdivision system with a resolution of 0.04 μm.