Fusing the pertinence of natural scene statistics-based methods and the ubiquity of convolutional neural network-based methods, a no-reference image quality assessment (IQA) method fusing deep learning and statistical visual features for no-reference image quality assessment (FDSVIQA) is proposed. For the statistical visual features, a local normalized luminance map and a local normalized local binary pattern (LBP) map of the image are constructed, and the local normalized luminance features and the gradient-weighted local normalized LBP features are extracted on the two maps, respectively. These two kinds of features are concatenated to build the image statistical visual features. For deep learning, the local normalized luminance block and the localized normalized LBP block are input into a double-path deep learning network, and the statistical visual features are input into the deep learning network to be integrated with the depth features. After learning and training, IQA is achieved. The performance of the proposed FDSVIQA algorithm is tested on the Laboratory for Image and Video Engineering (LIVE), LIVE Multiply Distorted Image Quality Database, and Multiply Distortion Optics Remote Sensing Image databases. Experimental results show that the FDSVIQA algorithm has excellent subjective and objective consistency and good robustness for both distorted natural images and distorted remote sensing images. In addition, the FDSVIQA has database independence.