In recent years, transformers have been introduced into the field of remote sensing image change detection (CD) due to their excellent global context modeling capabilities. However, the global nature of the self-attention used by transformers is not sensitive to local high-frequency information, making it challenging to address complex CD problems. To address this issue, some methods have considered combining convolutional neural networks and transformers to jointly harvest local-global features. Nevertheless, these methods have not paid much attention to the interactions between the features extracted by the two components. Therefore, to address the challenges faced by existing CD methods in balancing local and global features, as well as their inadequacy in handling complex scenarios, we propose a frequency-driven transformer network (FDTNet) that improves self-attention and the overall architecture. In the overall framework, the network first extracts features to obtain primary and deep features and then utilizes the transformer encoder–decoder network to obtain context embeddings with spatiotemporal information from these primary features to guide the subsequent processing of deep features. In the transformer encoding part, we introduce a frequency-driven attention module, comprising low-frequency attention (LFA) branch, high-frequency attention (HFA) branch, and local window self-attention, where LFA captures global dependencies, HFA handles important high-frequency information, and local window self-attention supplements detailed local information loss. In the transformer decoding part, an interactive attention module is utilized to integrate context information from the transformer encoder into deep features. In addition, we propose an edge enhancement module and gate-controlled channel exchange operation, where the former enhances boundary features using the Sobel operator and the latter swaps channels to obtain richer perspective information. The experimental results show that FDTNet achieved an F1 score of 90.95% on LEVIR-CD, 82.70% on NJDS, and 79.84% on SYSU, outperforming several state-of-the-art CD methods.