Deep convolutional neural networks (CNNs) have contributed to the significant progress of the single image super resolution (SISR) field. However, most of existing CNN-based SR models require high computing power, which is not conducive to daily use. In addition, these algorithms need to use a large number of CNN to obtain global features. Therefore, this paper proposes an image super-resolution framework based on adaptive residual neural network, using the adaptive framework to switch between global and local reasoning for internal features in a flexible way, it can extract a large number of global features without neglecting key information, which is conducive to the comprehensiveness of residual images. After the adaptive block, SENet is added to conduct channel modeling for the extracted features, and the importance of each feature channel is automatically acquired by learning method. Then, according to this importance, useful features are promoted and those that are not useful for the current task are suppressed. In this way, with more nonlinearity, the complex correlation between channels can be better fitted, and the number of parameters and computation can be reduced, which can improve the performance of super resolution to a certain extent.