The rapid, accurate, and intelligent extraction of vegetation areas is of great significance for conducting research on forest resource inventory, climate change, and the greenhouse effect. Currently, existing semantic segmentation models suffer from limitations such as insufficient extraction accuracy (ACC) and unbalanced positive and negative categories in datasets. Therefore, we propose the Attention U-Net model for vegetation extraction from Landsat8 operational land imager remote sensing images. By combining the convolutional block attention module, Visual Geometry Group 16 backbone network, and Dice loss, the model alleviates the phenomenon of omission and misclassification of the fragmented vegetation areas and the imbalance of positive and negative classes. In addition, to test the influence of remote sensing images with different band combinations on the ACC of vegetation extraction, we introduce near-infrared (NIR) and short-wave infrared (SWIR) spectral information to conduct band combination operations, thus forming three datasets, namely, the 432 dataset (R, G, B), 543 dataset (NIR, R, G), and 654 dataset (SWIR, NIR, R). In addition, to validate the effectiveness of the proposed model, it was compared with three classic semantic segmentation models, namely, PSP-Net, DeepLabv3+, and U-Net. Experimental results demonstrate that all models exhibit improved extraction performance on false color datasets compared with the true color dataset, particularly on the 654 dataset where vegetation extraction performance is optimal. Moreover, the proposed Attention U-Net achieves the highest overall ACC with mean intersection over union, mean pixel ACC, and ACC reaching 0.877, 0.940, and 0.946, respectively, providing substantial evidence for the effectiveness of the proposed model. Furthermore, the model demonstrates good generalizability and transferability when tested in other regions, indicating its potential for further application and promotion.