To self-calibrate the intrinsic camera parameters, we designed a calibration pattern and proposed a method to estimate camera parameters involving distortion coefficients from orthogonal vanishing points. We first obtained four pairs of vanishing points from an image of the calibration pattern and selected the two pairs with the smallest error. Then, after analyzing the effect of the intrinsic parameter error from the noise of the vanishing points, we employed an optimization algorithm to obtain the best vanishing points. Using two images with different orientations, we could obtain the intrinsic parameters by using a linear method. Finally, according to the preliminary calibrated intrinsic parameters and a criterion that the airline is a static straight line in the pictorial plane, the arithmetic mean of the camera distortion, which is based on the common characteristic points, could be obtained. Simulated and experimental results indicated that the two-dimensional reconstruction error is 0.32 pixels. Although this method has the same error level as the traditional method, it has a better flexibility and can achieve real-time camera self-calibration.