The quality of the image is crucial for the performance of a visual surface inspection system. The final image quality is affected by the properties of the imaging unit. In order to evaluate the quality of the imaging unit a designer should be able to characterize the imaging performance using quantitative figures of merit. The suitability of the modulation transfer function (MTF) for the figure of merit of imaging in visual web inspection is discussed. A straightforward setup for the determination of the system MTF is suggested. Three exemplary cases are analyzed: (1) the performance of an imaging system as a function of the system f number, (2) the performance of an imaging system against the depth-of-field requirement, and (3) the performance of a TDI imager against speed mismatching. In order to relate the system MTFs to image quality, a set of defective samples from a paper manufacturing process was collected and imaged. Quantitative image quality criteria are proposed, and the sample images are analyzed according to these criteria. The final image quality is compared with the corresponding system MTF values.