28 March 2013 A preliminary comparison of different methods for observer performance estimation
Author Affiliations +
Abstract
In medical imaging, image quality is assessed by the degree to which a human observer can correctly perform a given diagnostic task. Therefore the image quality is typically quantified by using performance measurements from decision/detection theory like the receiver operation characteristic (ROC) curve and the area under ROC curve (AUC). In this paper we compare five different AUC estimation techniques, widely used in the literature, including parametric and non-parametric methods. We compared each method by equivalence hypothesis testing using a model observer as well as data sets from a previously published human observer study. The main conclusions of this work are 1) if a small number of images are scored, one cannot tell apart different AUC estimation methods due to large variability in AUC estimates, regardless whether image scores are reported on a continuous or quantized scale. 2) If the number of scored images is large and image scores are reported on a continuous scale, all tested AUC estimation methods are statistically equivalent.
© (2013) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Francesc Massanes, Francesc Massanes, Jovan G. Brankov, Jovan G. Brankov, } "A preliminary comparison of different methods for observer performance estimation", Proc. SPIE 8673, Medical Imaging 2013: Image Perception, Observer Performance, and Technology Assessment, 86730Z (28 March 2013); doi: 10.1117/12.2007229; https://doi.org/10.1117/12.2007229
PROCEEDINGS
9 PAGES


SHARE
Back to Top