Image classification has become an important topic in multimedia processing. Recently, neural network-based methods have been proposed to solve the classification problem. Among them, the radial basis function neural network (RBFNN) is the most popular architecture, because it has good learning and approximation capabilities. However, traditional RBFNNs are sensitive to center initialization. To obtain appropriate centers, it needs to find significant features for further RBF clustering. In addition, the training procedure of a traditional RBFNN is time consuming. Therefore, in this work, a combination of a self-organizing map (SOM) and learning vector quantization (LVQ) neural networks is proposed to select more appropriate centers for an RBFNN, and a modular RBF neural network (MRBFNN) is proposed to improve the classification rate and to speed up the training time. Experimental results show that the proposed MRBFNN has better performance than those of the traditional RBFNN, the discrete wavelength transform (DWT)-based method, the tree structured wavelet (TWS), the discrete wavelet frame (DWF), the rotated wavelet filter (RWF), and the wavelet neural network based on adaptive norm entropy (WNN-ANE) methods.