Visualizing and enhancing a deep learning framework using patients age and gender for chest x-ray image retrieval

Yaron Anavi; Ilya Kogan; Elad Gelbart; Ofer Geva; Hayit Greenspan

doi:10.1117/12.2217587

7 July 2016 Visualizing and enhancing a deep learning framework using patients age and gender for chest x-ray image retrieval

Yaron Anavi, Ilya Kogan, Elad Gelbart, Ofer Geva, Hayit Greenspan

Author Affiliations +

Proceedings Volume 9785, Medical Imaging 2016: Computer-Aided Diagnosis; 978510 (2016) https://doi.org/10.1117/12.2217587
Event: SPIE Medical Imaging, 2016, San Diego, California, United States

Abstract

We explore the combination of text metadata, such as patients’ age and gender, with image-based features, for X-ray chest pathology image retrieval. We focus on a feature set extracted from a pre-trained deep convolutional network shown in earlier work to achieve state-of-the-art results. Two distance measures are explored: a descriptor-based measure, which computes the distance between image descriptors, and a classification-based measure, which performed by a comparison of the corresponding SVM classification probabilities. We show that retrieval results increase once the age and gender information combined with the features extracted from the last layers of the network, with best results using the classification-based scheme. Visualization of the X-ray data is presented by embedding the high dimensional deep learning features in a 2-D dimensional space while preserving the pairwise distances using the t-SNE algorithm. The 2-D visualization gives the unique ability to find groups of X-ray images that are similar to the query image and among themselves, which is a characteristic we do not see in a 1-D traditional ranking.

Citation Download Citation

Yaron Anavi, Ilya Kogan, Elad Gelbart, Ofer Geva, and Hayit Greenspan "Visualizing and enhancing a deep learning framework using patients age and gender for chest x-ray image retrieval", Proc. SPIE 9785, Medical Imaging 2016: Computer-Aided Diagnosis, 978510 (7 July 2016); https://doi.org/10.1117/12.2217587

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available