Identification of superior discriminators during non-ROC studies

Jill L. King; F. Leland Thaete M.D.; Jules H. Sumkin M.D.; Cynthia A. Britton M.D.; Arleen M. Peterson; Jeffrey D. Towers M.D.; Thomas Chang; Carl R. Fuhrman; David Gur

doi:10.1117/12.462700

12 April 2002 Identification of superior discriminators during non-ROC studies

Jill L. King, F. Leland Thaete M.D., Jules H. Sumkin M.D., Cynthia A. Britton M.D., Arleen M. Peterson, Jeffrey D. Towers M.D., Thomas Chang, Carl R. Fuhrman, David Gur

Author Affiliations +

Proceedings Volume 4686, Medical Imaging 2002: Image Perception, Observer Performance, and Technology Assessment; (2002) https://doi.org/10.1117/12.462700
Event: Medical Imaging 2002, 2002, San Diego, California, United States

Abstract

Small, efficient observer performance study methods, such as multi-point rank order and forced choice, can be used to either determine the necessity to perform a large scale, receiver operator characteristic (ROC) study or to set the boundary conditions at which it makes sense to perform an ROC study. Because these studies often require observers to discriminate between (among) modes having small visual differences, we decided to address the issue of observers' abilities to make distinctions between modes in these types of non-ROC studies. In this project we reviewed the data of six different non-ROC studies that included a total of 24 observers to determine whether some behave as better mode discriminators. Because of the actual small differences, most observers performed poorly in identifying differences between or among modes. At the same time, at least one and sometimes more observers could identify extremely small differences between modes. These differences were statistically significant. Our results indicate that a good mode discriminator in one study may not perform as well in another such study. Non-ROC studies can be highly sensitive to differences between modes. However, large differences in observer performance combined with observer inconsistency across studies necessitates that these studies include multiple observers.

Citation Download Citation

Jill L. King, F. Leland Thaete M.D., Jules H. Sumkin M.D., Cynthia A. Britton M.D., Arleen M. Peterson, Jeffrey D. Towers M.D., Thomas Chang, Carl R. Fuhrman, and David Gur "Identification of superior discriminators during non-ROC studies", Proc. SPIE 4686, Medical Imaging 2002: Image Perception, Observer Performance, and Technology Assessment, (12 April 2002); https://doi.org/10.1117/12.462700

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available