As semiconductor processing becomes more complicated and pattern sizes shrink, the overlay metrology has become one of the most important issues in the semiconductor industry. Therefore, in order to obtain correct, reliable overlay values in semiconductor fabrication facilities (fab), quantization methods for the efficient management and implementation of a measurement algorithm are required, as well as an understanding of the target structures in the semiconductor device. We implemented correct, reliable overlay values in the pattern using the image processing method. The quantization method, through correlation analysis and a new algorithm for target structures, were able to improve the sensitivity to misalignment in the pattern and enable more stable and credible in-line measurement by decreasing the distribution of the residuals in overlay values. Since overlay values of the pattern in the fab were measured and managed more reliably and quickly, it is expected that our study will be able to contribute to the yield enhancement of semiconductor companies.