Dense wavelength division multiplexing (DWDM) system is the tendency of optical fiber communication systems because of its high speeds and capacities. DWDM systems with channel data rates of 40 Gbit/s bring us both advantages and challenges. With broader spectrum, the signal suffers more from chromatic dispersion, optical fiber nonlinear effects and polarization mode dispersion (PMD). Simultaneously, the combination of PMD and nonlinear effects results in more complexity. In this paper, amended nonlinear Schrodinger equations, which include group velocity dispersion (GVD), third order dispersion (TOD), self-phase modulation (SPM), cross-phase modulation (XPM), four-wave mixing (FWM) and PMD synthetically, are derived, then the system degradation induced by PMD and nonlinear effects is investigated via numerical simulation. The results show that when the power of input signal is low, PMD has more affection on system comparing with nonlinear effects, and nonlinear effects become dominating with power increasing, but mild PMD may mitigate their impact to some extent. The results in this paper are valid for design and analysis of long haul DWDM systems with high bit rate.