In light of recent advances in deep learning and high-resolution remote sensing imaging technology, there has been a growing adoption of remote sensing ship classification models that are based on deep learning methodologies. However, the efficiency of remote sensing ship classification models is affected by complex backgrounds, shooting conditions, high inter-class similarity of ship targets, and sample diversity. To tackle the challenges above, we propose a multi-scale attention-based adaptive feature fusion (AFF) network for fine-grained ship classification in remote sensing scenarios to improve the fine-grained classification ability of the model from local details. First, using the idea of information complementarity, the multi-scale feature interaction module is constructed in the multi-scale attention module. It employs bidirectional feature interaction paths to concurrently capture intricate details within both deep and shallow ship features, enhancing the interplay among different levels of information. Second, the hybrid attention module is part of the multi-scale attention module. It is designed to enhance the cross-dimensional interaction of spatial domain and channel domain information to amplify the importance of crucial feature regions and feature channels. This allows the network to pay more attention to specific areas and extract distinctive features. Finally, the AFF module is designed to automatically calibrate and fuse different levels of saliency features to obtain features with more fine-grained discrimination for model classification. In this approach, these modules synergistically collaborate and mutually reinforce each other, ultimately increasing the accuracy of ship classification tasks. We evaluated our method on three large-scale fine-grained classification benchmarks; the experimental results show that the proposed method had better fine-grained classification than other methods.