Vehicle classification is vital to an intelligent transport system. To obtain a high accuracy, it is the most crucial process to extract reliable and distinguishable features of vehicles. A feature extraction method using a lightweight convolutional network for vehicle classification is proposed. The main contributions are threefold: (1) a lightweight network named LWNet with two convolution layers is proposed to extract the features of the vehicles; (2) Hu moment is integrated with spatial location information to improve its own describing and distinguishing ability; and (3) histogram of oriented gradient (HOG) feature is extracted from the complete image, and then the above two kinds of features are combined with HOG to form the vector. And then, a support vector machine is trained to obtain the classification model. Vehicles are classified into six categories, i.e., large bus, car, motorcycle, minibus, truck, and van. The experimental results have demonstrated that the classification accuracy can achieve 97.39%, which is 3.81% higher than that obtained from the conventional methods. In addition, for this vehicle classification task, the proposed lightweight convolutional network can achieve comparable or even higher performance compared to the deep convolutional neural networks, while the proposed method does not need the support of a graphics processing unit and has much lower complexity without the training process.