Network output visualization to uncover limitations of deep learning detection of pneumothorax

Jennie Crosby; Sophia Chen; Feng Li; Heber MacMahon; Maryellen Giger

doi:10.1117/12.2550066

Translator Disclaimer

16 March 2020 Network output visualization to uncover limitations of deep learning detection of pneumothorax

Jennie Crosby, Sophia Chen, Feng Li, Heber MacMahon, Maryellen Giger

Author Affiliations +

Proceedings Volume 11316, Medical Imaging 2020: Image Perception, Observer Performance, and Technology Assessment; 113160O (2020) https://doi.org/10.1117/12.2550066
Event: SPIE Medical Imaging, 2020, Houston, Texas, United States

Abstract

The overlapping structures in a chest radiograph can make the detection of pneumothorax difficult. In addition, the visual signs of a pneumothorax, including a fine line at the edge of the lung and a change in texture outside the lung, can be subtle. Some published studies have reported high performances using deep learning for the detection of pneumothorax in chest radiographs using the publicly available ChestX-ray8 dataset. However, at the image input sizes these studies used, 256 x 256 or 224 x 224 pixels, the visual signs of a pneumothorax are typically not visible. In this study, radiographs labeled as pneumothorax in the ChestX-ray8 dataset were interpreted by a radiologist and then confirmed using the radiologist-defined truth from a pneumothorax challenge database. In addition, chest radiographs with and without pneumothorax were obtained from our institution and verified. Therefore, the entire dataset of 5,346 radiographs had truth confirmed by two radiologists. The dataset was used for fine-tuning a VGG19 neural network for the task of detecting pneumothorax in chest radiographs. After fine-tuning was complete, network visualization was performed using Grad-CAM to determine the most influential aspects of the radiograph for the network’s classification. It was found that 67% of Grad-CAM heatmaps for correctly classified pneumothorax cases did not have regions of high influence that overlapped with the actual location of the pneumothorax. Overall, the independent test set yielded an AUC of 0.78 (95% confidence interval: 0.74, 0.82) in the task of distinguishing between radiographs with and without pneumothorax.

Conference Presentation

Citation Download Citation

Jennie Crosby, Sophia Chen, Feng Li, Heber MacMahon, and Maryellen Giger "Network output visualization to uncover limitations of deep learning detection of pneumothorax", Proc. SPIE 11316, Medical Imaging 2020: Image Perception, Observer Performance, and Technology Assessment, 113160O (16 March 2020); https://doi.org/10.1117/12.2550066

ACCESS THE FULL ARTICLE