The continual deformation of an object will lead to a temporal speckle pattern. By analyzing this time-dependent speckle pattern the temporal deformations of the object can be obtained. We propose a method for measuring the displacement components caused by deformations using the temporal speckle pattern interferometry (TSPI). By capturing a series of speckle interference patterns related to the object deformations, we can get the fluctuations in the intensity of the interference patterns. Further, the phase maps for whole-field object displacements are calculated with the inversion of the temporal interference intensities by estimating both the average intensity and the modulation. In this way, one can quantitatively measure temporal displacements simply using a conventional electronic speckle pattern interferometry (ESPI) system without phase shifting or a carrier. An elaboration of the TSPI is presented and out-of-plane displacements caused by pressure and thermal deformations are measured.