With the rapid development of sensor and communication technology, the volume and the resolution of the data became increasingly high. Compressive Sensing theory allows signal compressed at a rate much lower than the Nyquist rate, which is promising to deal with big data acquisition and transmission. Compressive sensing has been applied in a variety of fields such as clutter suppression, image/video reconstruction, and real time processing. Most of the conventional algorithms for the estimation of the original signal, for instance, Total Variation (TV), consist of consistency error and constraint terms, the latter of which is quite influential on the quality of reconstructed image. The results subject to different constraints may vary greatly, for example, the conventional TV constraint suffers from the step effect, while the Higher Degree Total Variation (HDTV) may have the defect of edge blur. Besides, the computational cost is another problem, which needs to be considered. In this paper, a constraint refinement based algorithm for compressive sensing image reconstruction is proposed. Firstly, the construction of the constraint term is studied. For images that show different characteristics (for example, the richness of texture, etc.), the appropriate constraints for different cases are discussed. Secondly, a modified constraint is introduced to overcome the defect of the aforementioned algorithms. Moreover, a fast approximation algorithm to enhance the calculation efficiency is proposed based on the introducing of an auxiliary function to cross update. The visual and quantitative assessment both prove the superiority of the proposed constraint refinement method in terms of SNR, SSIM, and PSNR.