Influencing clinicians and healthcare managers: can ROC be more persuasive?

S. Taylor-Phillips; M. G. Wallis; A. Duncan; A. G. Gale

doi:10.1117/12.843784

23 February 2010 Influencing clinicians and healthcare managers: can ROC be more persuasive?

S. Taylor-Phillips, M. G. Wallis, A. Duncan, A. G. Gale

Proceedings Volume 7627, Medical Imaging 2010: Image Perception, Observer Performance, and Technology Assessment; 76270X (2010) https://doi.org/10.1117/12.843784
Event: SPIE Medical Imaging, 2010, San Diego, California, United States

Abstract

Receiver Operating Characteristic analysis provides a reliable and cost effective performance measurement tool, without using full clinical trials. However, when ROC analysis shows that performance is statistically superior in one condition than another it is difficult to relate this result to effects in practice, or even to determine whether it is clinically significant. In this paper we present two concurrent analyses: using ROC methods alongside single threshold recall rate data, and suggest that reporting both provides complimentary data. Four mammographers read 160 difficult cases (41% malignant) twice, with and without prior mammograms. Lesion location and probability of malignancy was reported for each case and analyzed using JAFROC. Concurrently each participant chose recall or return to screen for each case. JAFROC analysis showed that the presence of prior mammograms improved performance (p<.05). Single threshold data showed a trend towards a 26% increase in the number of false positive recalls without prior mammograms (p=.056). If this trend were present throughout the NHS Breast Screening Programme then discarding prior mammograms would correspond to an increase in recall rate from 4.6% to 5.3%, and 12,414 extra women recalled annually for assessment. Whilst ROC methods account for all possible thresholds of recall and have higher power, providing a single threshold example of false positive, false negative, and recall rates when reporting results could be more influential for clinicians. This paper discusses whether this is a useful additional method of presenting data, or whether it is misleading and inaccurate.

Citation Download Citation

S. Taylor-Phillips, M. G. Wallis, A. Duncan, and A. G. Gale "Influencing clinicians and healthcare managers: can ROC be more persuasive?", Proc. SPIE 7627, Medical Imaging 2010: Image Perception, Observer Performance, and Technology Assessment, 76270X (23 February 2010); https://doi.org/10.1117/12.843784

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
9 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Mammography

Breast

Medicine

Statistical analysis

Radiology

Analytical research

Digital mammography

Show All Keywords

Keywords/Phrases

Search In:

Publication Years