Metrological feasibilities of the prototype of a high-resolution displacement encoder with a laser diode light beam focused on the scale surface are investigated. Resolution of 0.01 µm has been obtained. The repeatability of the system ranges from 0.02 to 0.03 µm. A new simple reference point detection method that uses a scale grating edge as a reference point has been proposed. We suggest two possible ways of implementation of this method: analog by dc signal component extraction and digital by signal amplitude extraction. The proposed reference point detection technique allows significant reduction of nonlinearity errors of the measuring system. Nonlinearity of the prototype device does not exceed 0.1 µm in the 48-mm measuring range. Due to the application of a focused laser beam, the proposed optical system is extremely insensitive to disturbing shifts and tilts of the scale. The acceptable tilts of the encoder detection head relative to the grating in the prototype device reach about 0.5 arcdeg in all directions. The tolerance of the distance between detection head and the grating is approximately equal to 0.1 mm.