Speckle noise exerts a noticeable impact on the quality of synthetic aperture radar (SAR) images. It could harm their applications in the remote sensing field, for example, the land cover classification. This paper presents a SAR image despeckling method by using the variance constrained convolutional neural network (CNN). We exploit the significant distinction between speckle noise and ground truth from the viewpoint of their statistical characteristics. The estimated noise variance as well as a weighting factor is introduced into the loss function. It can drive the learning of network to produce the result with more dispersion. After the model training, the variance constrained CNN could generate the despeckled SAR image by means of noise matrix estimation from an input contaminated by strong speckle. Finally, experiments on synthetic SAR images are conducted to demonstrate its effectiveness. It indicates that the proposed method is not only independent of image background in training, but also outperforms the classical SAR despeckling CNN.