Translator Disclaimer
Paper
3 March 2011 Analysis of the number of distinct findings obtained by multiple readers in an MRMC study: When do findings obtained from the addition of new readers become redundant, or otherwise negligible?
Author Affiliations +
Abstract
The ultimate goal of this project is to investigate whether the effect of a computer-aided detection (CAD) system on readers' performance (especially, in situation of an upgrade of the CAD system, or between two different CAD systems with similar design) can be accurately predicted without having to perform a multi-reader multi-case (MRMC) observer study and, if such prediction is possible, to establish the underlying methodology. Our current study is intended to provide evidence that would substantiate efforts toward such investigation. The objectives of this study were 1) to investigate the relationship between the number of radiologists reading a dataset of thoracic computed tomography (CT) images to identify lung nodules and the number of distinct findings and 2) to determine the number of readers needed to identify almost all clinically distinct findings in a dataset. We used data from a multi-reader multi-case (MRMC) observer study that consisted of six radiologists interpreting 85 thoracic CT examinations. To further illustrate our approach, we also utilized simulated data consisting of twelve readers interpreting 198 samples equally distributed between three levels of detection difficulty. For each possible reader grouping, the number of distinct findings identified by the readers in the group was calculated. Five types of regression models used to describe the relationship between the average number of distinct findings per case and the number of readers needed were compared. The result showed that the logistic model best fitted both the thoracic CT data and the simulated data. Our assumption is that adding more readers after a certain reader set size would mostly add redundant findings and, therefore, the benefit would be negligible. Using this model, the predicted number of readers was found to depend on the type of findings considered. Our study showed that the number of clinically distinct findings that can be identified by radiologists on CT lung examinations without the use of a CAD system may be limited and that identifying almost all of these findings may only require a limited number of readers.
© (2011) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Sophie Paquerault, Berkman Sahiner, Anna Kettermann, Laura M. Yarusso, Lubomir M. Hadjiiski, and Heang-Ping Chan "Analysis of the number of distinct findings obtained by multiple readers in an MRMC study: When do findings obtained from the addition of new readers become redundant, or otherwise negligible?", Proc. SPIE 7966, Medical Imaging 2011: Image Perception, Observer Performance, and Technology Assessment, 79661M (3 March 2011); https://doi.org/10.1117/12.878103
PROCEEDINGS
10 PAGES


SHARE
Advertisement
Advertisement
Back to Top