The dominant image transformation used in the existing fractal coding schemes is the affine function. Although an affine transformation is easy to compute and understand, its linear approximation ability limits the employment of larger range blocks, that is, it limits further improvement in compression efficiency. We generalize the image transformation from the usual affine form to the more general form, e.g., quadratic form, and provide theoretical requirements for the generalized transformation to be contractive. Based on the self-transformation system (STS) model, an image sequence coding scheme-fractal-based image sequence coding-is proposed. In this coding scheme, our generalized transformation is used to model the self-transformation from the domain block to its range blocks. Experimental results on a real image sequence show that for the same size of blocks, the SNR can be improved by 10 dB, or, for the same SNR of the decoded image sequence, the compression ratio is raised twofold when the new generalized transformation is used to replace the usual affine transformation. In addition, due to the utilization of the STS model, the computational complexity is only linearly related to the size of the 3-D blocks. This provides for fast encoding and decoding.