Recent advances in machine learning and deep learning have provided an opportunity for improvement in the field of lithography. Compared with the numerical simulation, machine learning/deep learning may provide much faster and more efficient performance, but they provide less accurate solution due to the limitation of the statistical approach. Typical machine learning models cannot take into account complex multiple processes such as UV exposure, photoreaction and photo-resist development in lithography. In this work, we developed a newly designed deep learning algorithm not only to improve the model accuracy but also to overcome data limitation. By combining a physics-driven machine learning model and a complex-valued neural network (CVNN), we designed a novel machine learning model structure. Applying CVNN to phase shift mask analysis could improve the model performance dramatically. As such, this work opens up a new class of photo-lithography analysis by using a novel neural network model.