Road information is a crucial type of geographic information. The extraction of road information from remote sensing images has been widely applied in various fields such as mapping, transportation, and navigation. However, due to the obstruction of buildings, trees, and shadows, or the spectral similarity between roads and buildings, road extraction remains a challenging research topic. Most current methods focus only on the spatial domain, neglecting the information contained in the image frequency domain. Therefore, this work proposes a remote sensing image road extraction model, frequency domain attention encoder-decoder network (FDAENet). This model mainly consists of three parts. First, the encoder is composed of frequency domain transformer modules (FDTMs). The gnConv in the FDTM includes depthwise separable convolution and phase and magnitude (PM) filters, where the PM filter contains a global filter and phase and amplitude filters located in two parallel layers, used to extract feature information of road remote sensing images from the frequency domain. Then, a multi-scale context extraction module is proposed, which introduces appropriate road context information to enhance inference capability. Finally, a stripe convolution module is introduced to capture long-distance context information from four different directions. Experiments on public road datasets show that FDAENet performs excellently in terms of F1, intersection over union, average path length similarity, and other indicators. Visualization results show that FDAENet performs better in extracting complex roads and can effectively extract roads from high-resolution remote sensing images.