Nowadays, breast cancer has increasingly threatened the health of human, especially females. However, breast cancer is still hard to detect in the early stage, and the diagnostic procedure can be time-consuming with abundant expertise needed. In this paper, the main research is the application of deep learning method in the diagnosis of photoacoustic breast cancer and the comparison of the performance of the traditional machine learning classification algorithm and deep learning method in the actual scenario of the photoacoustic imaging breast cancer diagnosis. The traditional supervised learning method firstly obtains the photoacoustic images of breast cancer through preprocessing, extracts the SIFT features, and uses K-means clustering to obtain the feature dictionary. The histogram of the feature dictionary was used as the final feature of the image. Support vector machine (SVM) was used to classify the final features, achieving an accuracy of 82.14%. In the deep learning method, AlexNet and GoogLeNet were used to perform the transfer learning, achieving 88.23%, 89.23%, and 91.18% accuracy, respectively. Finally, by comparing the AUC, sensitivity, and specificity of SVM with AlexNet and GoogLeNet, it can be concluded that the combination of deep learning and photoacoustic imaging obtain a profound and important impact on clinical applications.