Independent versus sequential reading in ROC studies of computer-assist modalities

Sergey V. Beiden; Robert F. Wagner; Kunio Doi; Robert M. Nishikawa; Matthew T. Freedman M.D.; Shih-Chung Benedict Lo; Xin-Wei Xu

doi:10.1117/12.462678

Translator Disclaimer

12 April 2002 Independent versus sequential reading in ROC studies of computer-assist modalities

Sergey V. Beiden, Robert F. Wagner, Kunio Doi, Robert M. Nishikawa, Matthew T. Freedman M.D., Shih-Chung Benedict Lo, Xin-Wei Xu

Author Affiliations +

Proceedings Volume 4686, Medical Imaging 2002: Image Perception, Observer Performance, and Technology Assessment; (2002) https://doi.org/10.1117/12.462678
Event: Medical Imaging 2002, 2002, San Diego, California, United States

Abstract

This paper provides results of a statistical analysis of two methods for arranging the temporal sequencing of the unaided vs computer-assisted reading in multiple-reader, multiple- case (MRMC) receiver operating characteristic (ROC) studies of computer-aided detection of solitary pulmonary nodules (SPNs) on chest radiographs. The modes are the Independent mode, in which the readings are separated by a time on the order of one month, and the Sequential mode, in which the CAD-assisted reading immediately follows the unassisted reading. The method of Beiden, Wagner, Campbell (BWC) was used to decompose the variance of the ROC area summary accuracy measure into the components that are correlated across unaided and aided reading conditions and the components that are uncorrelated across these reading conditions. The latter are the only components of variability that contribute to the uncertainty in a measurement of the difference in reader performance between reading conditions. These uncorrelated components were dramatically reduced in the Sequential reading mode compared to the Independent reading mode-while the total reader variability remained almost constant. The results were remarkably similar across two independent studies analyzed. This may have important practical consequences because the Sequential mode is the least demanding on reader logistics and time.

Citation Download Citation

Sergey V. Beiden, Robert F. Wagner, Kunio Doi, Robert M. Nishikawa, Matthew T. Freedman M.D., Shih-Chung Benedict Lo, and Xin-Wei Xu "Independent versus sequential reading in ROC studies of computer-assist modalities", Proc. SPIE 4686, Medical Imaging 2002: Image Perception, Observer Performance, and Technology Assessment, (12 April 2002); https://doi.org/10.1117/12.462678

ACCESS THE FULL ARTICLE