In the structured light three-dimensional measurement system, calibration is crucial for measurement accuracy. However, conventional projector calibration methods involve complicated procedures. Therefore, a novel method is developed for projector calibration. The key concept is to establish the relationship between projector coordinates and the world coordinates using the projection checkerboard, which enables the calibration of the projector to be the same as that of a camera. In addition, the edges of the checkerboard form different degrees of high-light regions when the projection checkerboard is collected. These high-light regions affect the extraction of corner points and cause low accuracy of the projector calibration. Therefore, the projection checkerboard is collected by a CCD camera equipped with a polarizer at the optimal polarization angle. Then the subpixel corner extraction algorithm based on the homography matrix mapping is used to extract the subpixel coordinates of the corners. Finally, the projector can be calibrated with the same method as the camera. In the experimental part, the validity of the method is verified by the reprojection system.