The ultrahigh resolution of unmanned aerial vehicle (UAV) remote sensing images and tilting photography with multiple perspectives provide complete and detailed ground observation data for various engineering applications. However, noise and interference information make learning the typical features of ground objects difficult for current deep learning semantic segmentation networks. The hierarchical cognitive structure of human vision and the information transmission modes of retinal cone and rod cells were used to design a two-pathway anti-interference network for retinal perception mechanism simulation (RPMS). In the first pathway, the hierarchical cognition of cone cells was simulated by a one-to-one connected multiscale dilated convolution structure. In the second pathway, the hierarchical cognition of rod cells was simulated by a multiscale pyramid structure with many-to-one connections. With the one-to-one connection, the ability of RPMS to recognize detailed edges was strengthened. Furthermore, the many-to-one connection helped RPMS resist the disturbance from noise and interference. By combining the feature maps of the two paths, RPMS exhibited stronger noise resistance, better texture recognition, and better detail recognition compared with other semantic segmentation networks in the classification experiments. Thus this technique is suitable for UAV remote sensing image classification and has a broad application potential.