1 August 1990 Optimal use of rating scales in ROC analysis
Author Affiliations +
Observers participating in ROC studies are usually required to estimate the confidence with which each observation is made. With a discrete scale, the rating, or score, normally falls into one of 5categories, ranging from 'definitely normal' to 'definitely abnormal'. However, a major problem in data analysis from ROC studies has been found to be caused by observers who have not used the rating scale in a uniform manner, and have made many responses corresponding to the two extreme categories with few responses falling in the middle. The use of a continuous rating scale, with a point selected using a mouse, has assisted in analysis, but only to a limited extent. It has therefore been suggested elsewhere that it is desirable to force observers to select intermediate points. The effect of such an approach on ROC curves was studied by asking a group of observers to re-score a set of difficult clinical images, after training and with continuous feedback on their compliance. Although the resulting fall in the ROC curves was not statistically significant, it is considered unwise to force observers to report in what to them appears to be an unnatural manner.
© (1990) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
R. M. Dawood, R. M. Dawood, Andrew Todd-Pokropek, Andrew Todd-Pokropek, J. O.M.C. Craig, J. O.M.C. Craig, J. H. Highman, J. H. Highman, A. W. Porter, A. W. Porter, } "Optimal use of rating scales in ROC analysis", Proc. SPIE 1234, Medical Imaging IV: PACS Systems Design and Evaluation, (1 August 1990); doi: 10.1117/12.19029; https://doi.org/10.1117/12.19029

Back to Top