Automatic multi-label annotation of abdominal CT images using CBIR

Zhiyun Xue; Sameer Antani; L. Rodney Long; George R. Thoma

doi:10.1117/12.2254368

Translator Disclaimer

13 March 2017 Automatic multi-label annotation of abdominal CT images using CBIR

Zhiyun Xue, Sameer Antani, L. Rodney Long, George R. Thoma

Author Affiliations +

Proceedings Volume 10138, Medical Imaging 2017: Imaging Informatics for Healthcare, Research, and Applications; 1013807 (2017) https://doi.org/10.1117/12.2254368
Event: SPIE Medical Imaging, 2017, Orlando, Florida, United States

Abstract

We present a technique to annotate multiple organs shown in 2-D abdominal/pelvic CT images using CBIR. This annotation task is motivated by our research interests in visual question-answering (VQA). We aim to apply results from this effort in Open-i^SM, a multimodal biomedical search engine developed by the National Library of Medicine (NLM). Understanding visual content of biomedical images is a necessary step for VQA. Though sufficient annotational information about an image may be available in related textual metadata, not all may be useful as descriptive tags, particularly for anatomy on the image. In this paper, we develop and evaluate a multi-label image annotation method using CBIR. We evaluate our method on two 2-D CT image datasets we generated from 3-D volumetric data obtained from a multi-organ segmentation challenge hosted in MICCAI 2015. Shape and spatial layout information is used to encode visual characteristics of the anatomy. We adapt a weighted voting scheme to assign multiple labels to the query image by combining the labels of the images identified as similar by the method. Key parameters that may affect the annotation performance, such as the number of images used in the label voting and the threshold for excluding labels that have low weights, are studied. The method proposes a coarse-to-fine retrieval strategy which integrates the classification with the nearest-neighbor search. Results from our evaluation (using the MICCAI CT image datasets as well as figures from Open-i) are presented.

Citation Download Citation

Zhiyun Xue, Sameer Antani, L. Rodney Long, and George R. Thoma "Automatic multi-label annotation of abdominal CT images using CBIR", Proc. SPIE 10138, Medical Imaging 2017: Imaging Informatics for Healthcare, Research, and Applications, 1013807 (13 March 2017); https://doi.org/10.1117/12.2254368

ACCESS THE FULL ARTICLE