Computer models of real world objects and scenes are essential in a large and rapidly growing number of applications, hence motivating the automatic generation of models from images. While the completeness and accuracy of extracted models may be essential in some cases, in applications such as image-based view synthesis in which the goal is to produce new views of a scene, partial models with limited accuracy may produce satisfactory results. In this paper a method is described for partial image based-modeling which relies on a sparse set of matching points between several views. While a sparse set of matching points may be obtained more reliably, it provides only partial information on the reconstructed scene and uses only a small subset of the information contained in the images. Consequently, in the proposed approach, correlation constraints are used in order to test hypotheses in projective space so as to improve the correctness of the reconstructed model. The correlation constraints are based on all the image pixels belonging to the convex hull of the matched point set, thus utilizing a large amount of the information contained in the images. The same constraints are then used to modify the reconstructed model by detecting zones in which the model should be broken into several parts in order to accommodate occlusions in the scene and in order to smooth planar surfaces composed of several polygons. The paper provides demonstration of the application of the proposed approach to image-based view synthesis and geometric distortion correction in document images.