Laser trigonometric displacement sensor has characteristics of high efficiency, non-contact and large-scale measurement range, when coupled with scanning system, it can be widely used in profile measurement of complex workpiece surface. But for large steep workpieces, the axis of incident light emitted from the sensor can’t be perpendicular to its surface, accuracy will be largely degraded by this inclination angle. Also, the relationship between error and influencing factors dominated by inclination angle is a nonlinear function. If the influence of measuring distances is taken into account, the relationship becomes a multivariate mapping. So an improved multi-layer BP neural network is proposed to compensate for errors. This paper uses the genetic algorithm to optimize the initialization parameters of the network, while using adaptive training method to optimize the convergence process and adjusting the learning rate, increasing the momentum item to avoid falling into local extreme points. Besides, the laser displacement sensor of Keyence LK-H020 is used to obtain the measurement data and the error was obtained by comparing with a grating ruler with a precision of 10 nm. Based on the simulation and experimental results, the method can reduce the error from 3.8 μm to 0.5 μm when inclination range is from 0° to 8°, and from 7μm to 3 μm when the angle is from 0° to 50°. The results prove effectiveness, generalization and robustness of the algorithm.