Source and mask optimization (SMO) has emerged as a key resolution enhancement technique for advanced optical lithography. Current SMO, however, keeps the polarization state fixed, thus limiting the degrees of freedom during the optimization procedure. To overcome this limitation, pixelated gradient-based joint source polarization mask optimization (SPMO) approaches, which effectively extend the solution space of the SMO problem by introducing polarization variables, are developed. First, the SPMO framework is formulated using an integrative and analytic vector imaging model that is capable of explicitly incorporating the polarization angles. Subsequently, two optimization methods, namely simultaneous SPMO (SISPMO) and sequential SPMO (SESPMO) are developed, both of which exploit gradient-based algorithms to solve for the optimization problem. In addition, a postprocessing method is applied to reduce the complexity of the optimized polarization angle pattern for improving its manufacturability. Illustrative simulations are presented to validate the effectiveness of the proposed algorithms. The simulations also demonstrate the superiority of the SESPMO over SISPMO in computational efficiency and improvement of image fidelity.