Variability in reader selection of subtle cases for inclusion in a reduced size ROC study

Jill L. King; Howard E. Rockette; Melinda J. Staiger M.D.; Jane L. Weissman; Howard Eisen; Christopher M. Johns; David Gur

doi:10.1117/12.206855

17 April 1995 Variability in reader selection of subtle cases for inclusion in a reduced size receiver operating characteristic (ROC) study

Jill L. King, Howard E. Rockette, Melinda J. Staiger M.D., Jane L. Weissman, Howard Eisen, Christopher M. Johns, David Gur

Author Affiliations +

Proceedings Volume 2436, Medical Imaging 1995: Image Perception; (1995) https://doi.org/10.1117/12.206855
Event: Medical Imaging 1995, 1995, San Diego, CA, United States

Abstract

Sample size in ROC studies may be significantly reduced by including only difficult cases in the image set, but variability across studies may be a possible obstacle in the development of such a methodology due to case selection. To assess this situation, 300 cases used in a previous large ROC study, which included nine observers, were independently classified as subtle or typical by two experienced readers. Data from the previous study were reanalyzed using data sets consisting only subtle or typical images as designated by each classifier. Results showed a marked decrease in observer performance from the original study's results when only the subtle cases of either classifier were included in the analyses. For 12 of 15 possible comparisons (3 imaging modes and 3 diseases for subtle cases of either classifier were included in the analyses. For 12 of 15 possible comparisons (3 imaging modes and 3 diseases for subtle cases, 3 modes and 2 diseases for typical cases), the Spearman rho correlation coefficient between the performance indices computed for each reader for the subsets classified as subtle and typical was high and significant (P < 0.05). The results obtained in this preliminary study are encouraging and point out areas that warrant further investigation.

Citation Download Citation

Jill L. King, Howard E. Rockette, Melinda J. Staiger M.D., Jane L. Weissman, Howard Eisen, Christopher M. Johns, and David Gur "Variability in reader selection of subtle cases for inclusion in a reduced size receiver operating characteristic (ROC) study", Proc. SPIE 2436, Medical Imaging 1995: Image Perception, (17 April 1995); https://doi.org/10.1117/12.206855

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available