An image processing method based on fractal dimension is proposed in this paper. This method uses fractal dimension to process the character images firstly, and rises the analysis of each grid to the analysis of interrelation between the grids to eliminate interference. Box-counting method is commonly used for calculating fractal dimension of fractal, which uses small box whose side length is r ,that is the topological dimension of the box is d, to cover up the image. Because there are various levels of cavities and cracks, some small boxes are empty and some small boxes cover a part of fractal image which is called non-empty box (here refers to the average gray of the part that contained in the small box is larger than a certain threshold). We note down the number of non-empty boxes, analyze and calculate them. The method is used to image process the polluted characters, which can remove ink and scratches around the contour of the characters and remain basic contour, then the characters can be recognized by using template matching. In computer simulation experiment for polluted character recognition, this method can recognize the polluted characters quickly, which improve the accuracy of the recognition of the polluted characters.