24 March 2016 Ranking inconsistencies in the assessment of digital breast tomosynthesis (DBT) reconstruction algorithms using a location-known task and a search task
Author Affiliations +
Abstract
In this work, we validated a task-based performance figure-of-merit (FOM) by investigating ranking inconsistencies due to lurking variable/factors. We applied a falsifiable search assessment theory to assessing digital breast tomosynthesis (DBT) image quality using a scanning channelized Hotelling observer (CHO) on a simulated DBT dataset. We compared the performance of five reconstruction algorithms: filter back projection (FBP), maximum likelihood (ML), simultaneous algebraic reconstruction technique (SART), total-variation regularized least square estimator (TVLS) with strong and mild regularization settings. The results showed that the location-known-exactly (LKE) detection performance was almost identical for the five reconstruction algorithms. However the search characteristic as described by effective set size (M*) and search AUC value, ranked them differently. To falsify/corroborate our evaluations on search characteristic and performance, we conducted an image-size test. This test demonstrated an agreement between theoretical predictions and empirically measured observer performance in absolute performance levels, except for the ML algorithm. We concluded that evidence corroborated our evaluations, except that for the ML algorithm where our evaluation was wrong. Further investigation of the wrong evaluation in the ML case revealed a lurking variable that affected system performance ranking in search when AUC value was used as the FOM. This further confirmed that our evaluation in its current form for the ML algorithm was indeed wrong. We also noted that the ranking inconsistencies exist even when the AUC value was used as the FOM, and the falsifiable nature of M* allowed such inconsistencies to be identified.
© (2016) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Xin He, Xin He, Rongping Zeng, Rongping Zeng, Frank Samuelson, Frank Samuelson, Berkman Sahiner, Berkman Sahiner, } "Ranking inconsistencies in the assessment of digital breast tomosynthesis (DBT) reconstruction algorithms using a location-known task and a search task", Proc. SPIE 9787, Medical Imaging 2016: Image Perception, Observer Performance, and Technology Assessment, 97870R (24 March 2016); doi: 10.1117/12.2217613; https://doi.org/10.1117/12.2217613
PROCEEDINGS
7 PAGES


SHARE
Back to Top