A theoretical analysis of the x-ray phase imaging in the method of in-line holography setup with a finite size source is presented. Based on the transport of intensity equation (TIE) with a point source, we given an algorithm to quantitatively restore the phase from x-ray phase imaging with an incoherent finite size source. We show that the image intensity is a convolution of the source intensity distribution and the intensity got from the x-ray phase imaging with a point source. The algorithm needs the intensity distribution at the source plane, the plane just after the object and the image plane. Performing deconvolution and solving the TIE, the phase can be retrieved. Analytical investigation of a simple model suggests that our method is mainly applicable to the differential phase contrast case. Numerical examples are also presented.