We present a portable non-contact displacement sensor (NCDS) based on astigmatic method for micron displacement measurement. The NCDS are composed of a collimated laser, a polarized beam splitter, a 1/4 wave plate, an aspheric objective lens, an astigmatic lens and a four-quadrant photodiode. A visible laser source is adopted for easier alignment and usage. The dimension of the sensor is limited to 115 mm x 36 mm x 56 mm, and a control box is used for dealing with signal and power control between the sensor and computer. The NCDS performs micron-accuracy with ±30 μm working range and the working distance is constrained in few millimeters. We also demonstrate the application of the NCDS for lens centration error measurement, which is similar to the total indicator runout (TIR) or edge thickness difference (ETD) of a lens measurement using contact dial indicator. This application has advantage for measuring lens made in soft materials that would be starched by using contact dial indicator.