Convolutional neural networks have achieved excellent results in pedestrian detection state of the art. They are able to learn features from raw images which makes them easy, practical and robust for multiple visual classification tasks. In this paper, we propose a further improvement of convolutional neural networks using saliency detection. First, we use contourlet transform for saliency detection to generate a region of interest (ROI). The generated saliency maps are then used to feed the convolutional network which will be used for both feature extraction and classification. The paper contribution is two fold : (1) We use saliency detection as a filter to remove the noisy information in the background, which allow the network to converge faster during the training process. (2) Saliency reduced complexity of the road scene which improve significantly the CNN classification performance. Experiments conducted on INRIA and Pascal VOC datasets achieves state-of-the-art performance.