28 March 2013 The impact of using a JAFROC or ROC approach on the conclusions of a typical observer performance study
Author Affiliations +
Abstract
Purpose: The current study aims to compare ROC with JAFROC methodologies to investigate how the choice of available analytical approaches in observer studies can impact upon study conclusions.
Methods and materials: A total of 129 readers independently reviewed 60 mammographic cases, 20 of which were biopsy proven cases (abnormal) and 40 were normal. Each case consisted of the four standard cranio-caudal (CC) and medio-lateral oblique (MLO) projections. Readers were asked to interpret and locate any presence of cancer, and levels of confidence were scored on a scale of 1-5. Radiology workstations supporting 5MP diagnostic monitors and with full image manipulation tools were used to display all images. JAFROC and ROC methodologies were used and figures of merit and Az values respectively were correlated against key reader characteristics such as experience, qualifications, breast reading practices and physical characteristics using Spearman techniques.
Results: Correlation analysis between reader characteristics and JAFROC analysis demonstrated that four key characteristics were linked to performance: years of qualification as a radiologist (p=0.05, r= 0.18), years reading mammograms (p=0.01, r=0.24), number of mammograms read per year (p=0.001, r=0.24), and hours reading mammogram per week (p=0.04, r= 0.19). The ROC method indicated that determinants of performance were confined to years reading mammograms (p=0.02, r = 0.2), and number of mammograms read per year (p=0.04, r=0.23).
Conclusion: This work demonstrates the practical impact on study conclusions when different methodologies are used. The location sensitivity approach employed and statistical power with JAFROC, would suggest that the findings from this approach should be prioritized.
© (2013) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Mohammad Rawashdeh, Mohammad Rawashdeh, Warwick Lee, Warwick Lee, Mariusz Pietrzyk, Mariusz Pietrzyk, Roger Bourne, Roger Bourne, Elaine Ryan, Elaine Ryan, Warren Reed, Warren Reed, Patrick C. Brennan, Patrick C. Brennan, } "The impact of using a JAFROC or ROC approach on the conclusions of a typical observer performance study", Proc. SPIE 8673, Medical Imaging 2013: Image Perception, Observer Performance, and Technology Assessment, 86730B (28 March 2013); doi: 10.1117/12.2007360; https://doi.org/10.1117/12.2007360
PROCEEDINGS
6 PAGES


SHARE
Back to Top