We analyze the ultimate accuracy of interference phase neasrement that can be achieved by using heterodyne technique in conjunction with interfering speckle fields. We show that the mean square fluctuation of the measured phase depends not only on the traditionally introduced scintillation noise but also on the spatially random arrangement of the speckles over the detector aperture (speckle grain noise). Scintillation noise arises because the two speckle fields are partially decorrelated, in which case the interference pattern does not become completely dark even when the two speckle fields are out-of-phase since destructive interference is not complete. However, even in the case of perfect correlation, intrinsic random noise is to be expected, since the position of the speckles over the detector is random and therefore the total signal exhibits statistical fluctuations, which we call speckle grain noise. Our formula show, as expected, that scintillation noise can be reduced by enlarging the detector aperture which, however, cannot be larger than half of an interfringe spacing. On the contrary speckle grain noise cannot be reduced by enlarging the detect& aperture since the effect of increased sampling is cancelled by the increased range of phase variation.