Quantifying the limitations of the use of consensus expert committees in ROC studies

Miguel P. Eckstein; Thomas D. Wickens; Gal Aharonov; George Ruan; Craig A. Morioka; James Stuart Whiting

doi:10.1117/12.306177

21 April 1998 Quantifying the limitations of the use of consensus expert committees in ROC studies

Miguel P. Eckstein, Thomas D. Wickens, Gal Aharonov, George Ruan, Craig A. Morioka, James Stuart Whiting

Proceedings Volume 3340, Medical Imaging 1998: Image Perception; (1998) https://doi.org/10.1117/12.306177
Event: Medical Imaging '98, 1998, San Diego, CA, United States

Abstract

Many receiver operating characteristic (ROC) studies rely on establishing 'truth' about lesion absence/presence on the agreement of a panel of experts. In addition, in the consensus committee methodology, images where the members of the committee did not reach any agreement about the lesion absence/presence are discarded from the ROC study. But how reliable are 'gold standards' established by these expert committees. And does discarding images where no agreement was reached bias the spectrum of difficulty of the test image set for the ROC study. Computer simulated lesions of different strengths were embedded in real x-ray coronary angiogram background in order to measure the agreement among the decisions of members of the committee as a function of signal strength, to establish the accuracy of the decisions of the consensus expert committee and to compare it to individual more inexperienced readers.

Citation Download Citation

Miguel P. Eckstein, Thomas D. Wickens, Gal Aharonov, George Ruan, Craig A. Morioka, and James Stuart Whiting "Quantifying the limitations of the use of consensus expert committees in ROC studies", Proc. SPIE 3340, Medical Imaging 1998: Image Perception, (21 April 1998); https://doi.org/10.1117/12.306177

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available