This paper presents a convolutional neural networks (CNN) based on sparse coding for human postures recognition. It’s an unsupervised approach for color multi-channel processing. The improvement of the method is mainly reflected in two aspects. We transform sample images into patches and make a decorrelation between input patches and reconstructed patches. In addition, we use the convolution kernels extracted by sparse coding to replace the initialization of the convolution kernels for human postures recognition. The proposed method is tested in the public KTH pedestrian behavior dataset and HUMAN-V2 self-test dataset. Compared with the traditional way, our approach shortens the training time a lot and also improves the recognition rate. Our experimental results verifies the effectiveness.