Based on the “as low as reasonably practicable” principle, the amount of radiation exposure of CT should be decreased without impairing image quality. However, the excessive reduction of radiation exposure in CT results in the degraded images with noise and artifacts. The purpose of this study was to construct virtual normal dose CT images from ultra-low dose CT images by using dilated residual networks (DRN) that can expand the receptive field. Our database consisted of 1,860 pairs of normal dose and ultra-dose chest CT images obtained from 12 patients. In our proposed method, the DRN which consisted of seven dilated convolutional layers were trained the relationship of signal patterns between ultra-low dose CT patches (small region) and the corresponding normal dose CT patches. The trained DRN was employed to construct the virtual normal dose CT images from the ultra-low dose CT images. The root mean squared error, peak signal to noise ratio, and structural similarity index for the ultra-low dose CT images to the normal dose CT images were 59.3, 32.6dB, and 0.897, whereas those for the constructed images from ultra-low dose CT with a model-based iterative reconstruction (MBIR) were 49.1, 34.1dB, and 0.956. Those indices for virtual normal dose CT images were 44.4, 35.0dB, and 0.964, showing a significant improvement when compared with the ultra-low dose CT images and the constructed images with MBIR. The virtual normal dose CT images achieved higher image quality as compared with the ultra-low dose CT images and the constructed images with MBIR.