This paper focuses on a method to decrease the scattering loss induced by surface roughness through waveguide structure optimization. First, the concept of roughness is discussed briefly, and the diagram of waveguide surface roughness tested by optical profiler is given. Then, this report mainly analyzes the influence on scattering loss coefficient and total loss coefficient induced by surface roughness under different waveguide parameters. The study finds that the scattering loss coefficient and the total loss coefficient increase as roughness increasing. Last, the part produces a method to decrease scattering loss induced by roughness through waveguide structure optimization importantly. It is found that the total scattering loss coefficient can be decreased greatly if waveguide core size is in range from 60 μm to 80 μm or the parameter Δ is smaller than 0.016. When surface roughness is 200 nm, the correlation length is 4 μm, waveguide length is 100 cm, and core width (height, a=b) is from 30 μm to 70 μm, the total scattering loss coefficient can decrease from 3.37×10-2 dB/cm to 1.65×10-2 dB/cm.