Deep learning has been widely used in hyperspectral image (HSI) classification tasks and has achieved good results in various fields. However, the large gap between a large number of parameters to be adjusted and the limited labeled samples of HSI will greatly increase the risk of model overfitting and inevitably consume a large amount of time and memory. A lightweight hybrid channel expansion and squeeze network is proposed to solve the small sample set problem. First, a channel expansion and squeeze module (CESM) based on depthwise convolution is designed to obtain higher fine-grained features by increasing the width of network channels. At the same time, the introduction of efficient channel attention module can emphasize the importance of each channel and effectively improve the representation of CESM compression output. Second, multidimensional CESM is used to reconstruct the entire feature extraction process and further reduce the model complexity while maintaining the intrinsic physical characteristics of HSI. Finally, the extracted high-level abstract features are imported into the global average pooling classifier to obtain the prediction probability of each category. Experimental results on Indian Pines, Pavia University, and Salinas Scene datasets show that the accuracy of the proposed method can reach 98.66%, 99.61%, and 99.84%, respectively, under limited labeled samples, and the computational cost is significantly lower than other advanced methods.