Based on microscopic imaging and sub-aperture stitching technology, Surface defect detection system realizes the automatic quantitative detection of submicron defects on the macroscopic surface of optical components, and solves quality control problems of numerous large- aperture precision optical components in ICF (Inertial Confinement Fusion) system. In order to improve the testing efficiency and reduce the number of sub-aperture images, the large format CCD (charged-coupled device) camera is employed to expand the field of view of the system. Large format CCD cameras are usually mosaicked by multi-channel CCD chips, but the differences among the intensity-grayscale functions of different channels will lead to the obvious gray gap among different regions of image. It may cause the shortening and fracture of defects in the process of the image binarization , and thereby lead to the misjudgment of defects. This paper analyzes the different gray characteristics in unbalance images, establishes gray matrix mode of image pixels, and finally proposes a new method to correct the gray gap of CCD self-adaptively. Firstly, by solving the inflection point of the pixel level curve in the gray histogram of the original image, the background threshold is set, and then the background of the image is obtained; Secondly, pixels are sampled from the background and calculated to get the gray gap among different regions of the image; Ultimately, the gray gap is compensated. With this method, an experiment is carried out to adjust 96 dual-channel images from testing a fused silica sample with aperture 180mm×120mm. The results show that the gray gap of the images on different channel is reduced from 3.64 to 0.70 grayscale on average. This method can be also applied to other CCD mosaic camera.