A method of ROC analysis by applying item response theory (IRT) to results of 1/0 judgments on the presence or absence of abnormal findings in CT image readings

Toru Matsumoto; Akira Furukawa; Kanae Nisizawa; Kohei Murao; Kozo Hanai; Shinji Yamamoto; Shinichi Wada; Shusuke Sone

doi:10.1117/12.771105

Translator Disclaimer

6 March 2008 A method of ROC analysis by applying item response theory (IRT) to results of 1/0 judgments on the presence or absence of abnormal findings in CT image readings

Toru Matsumoto, Akira Furukawa, Kanae Nisizawa, Kohei Murao, Kozo Hanai, Shinji Yamamoto, Shinichi Wada, Shusuke Sone

Author Affiliations +

Proceedings Volume 6917, Medical Imaging 2008: Image Perception, Observer Performance, and Technology Assessment; 69170Y (2008) https://doi.org/10.1117/12.771105
Event: Medical Imaging, 2008, San Diego, California, United States

Abstract

The purpose of this study is to develop a method of ROC analysis to evaluate both the ability of individual readers to detect abnormal findings and the detectability of abnormal findings in individual cases by applying item response theory to the results of 1/0 judgments on presence of abnormal findings in CT image readings. The validity of the method was verified by the following data and methods. Twenty-four readers searched for abnormal findings in 25 cases for which there were chest CT images with defined abnormal findings. From the 1/0 judgment data for the 25 cases with CT images (column) read by the 24 readers (row), each reader's potential ability to detect the abnormal findings (θ), the rate of "1" judgment by each reader, i.e. confidence level for TP and FP, P(θ), and the individual image response characteristic curves with the image as the item were calculated, from which ROC curves that represent the ability of each reader to detect abnormal findings were created. In addition, from the 1/0 judgment data for the 25 cases with CT images (row) read by the 24 readers (column), the potential detectability of abnormal findings for each CT image (θ) and the rate of "1" judgment for the image by readers, i.e. confidence level for TP and FP, P(θ), were calculated, from which ROC curves that represent the detectability of the abnormal finding in each case were created.

Citation Download Citation

Toru Matsumoto, Akira Furukawa, Kanae Nisizawa, Kohei Murao, Kozo Hanai, Shinji Yamamoto, Shinichi Wada, and Shusuke Sone "A method of ROC analysis by applying item response theory (IRT) to results of 1/0 judgments on the presence or absence of abnormal findings in CT image readings", Proc. SPIE 6917, Medical Imaging 2008: Image Perception, Observer Performance, and Technology Assessment, 69170Y (6 March 2008); https://doi.org/10.1117/12.771105

ACCESS THE FULL ARTICLE