An image frame in image sequences, in general, suffers from degradation due to spatially varying motions. In this paper, we propose a new image degradation model for space-variant motion blur and a spatially adaptive image restoration algorithm to remove such motion blur. For the proposed image degradation model, we mathematically analyze boundary effect which arises on the border of two image segments with different motions. We extend the already proposed model for a moving object in an arbitrary direction. In order to represent the point spread function (PSF) for motion blur in an arbitrary direction, we develop a method which distributes energy of samples in the PSF into the neighboring integer grids. In order to remove the above mentioned motion blur, we propose an object-based adaptive regularized image restoration algorithm. Both in synthetically and naturally motion blurred images, the proposed image restoration algorithm gives acceptable performance in removing motion blur and, as a result, in restoring important features, such as numbers and characters, which cannot be recognized in the input blurred image.