Extracting character information from complex images has always been a research hotspot and a difficult topic in the field of computer vision. Natural scene number is severely distorted due to blurred image, uneven illumination, weak illumination, which makes it difficult to achieve ideal results for character recognition, especially identifying characters of arbitrary length. In this paper, we use the convolutional network to automatically extract the advantages of features, and construct a convolutional neural network that recognizes single digits. In order to highlight important features, we also use grayscale methods to weaken the background information in natural scenes and apply certain Proportional Dropout strategy to prevent overfitting. We use a cyclic network to generate character sequences and construct a deep convolutional neural network that recognizes sequence numbers and without split character characters. We construct a deep convolutional neural network that uses convolutional networks and cyclic network fusion to simultaneously identify multiple digits. We verify on the SVHN data set, we achieve better results in accuracy, we get the recognition rate of single digital house number is 95.72%, better than most algorithms in existing articles and the recognition rate of serial digital house number is 89.14%.