24 May 1999 Gains in accuracy from averaging ratings of abnormality
Author Affiliations +
Six radiologists used continuous scales to rate 529 chest-film cases for likelihood of five separate types of abnormalities (interstitial disease, nodules, pneumothorax, alveolar infiltrates and rib fractures) in each of six replicated readings, yielding 36 separate ratings of each case for the five abnormalities. Analyses for each type of abnormality estimated the relative gains in accuracy (area below the ROC curve) obtained by averaging the case-ratings across: (1) six independent replications by each reader (30% gain), (2) six different readers within each replication (39% gain) or (3) all 36 readings (58% gain). Although accuracy differed among both readers and abnormalities, ROC curves for the median ratings showed similar relative gains in accuracy. From a latent-variable model for these gains, we estimate that about 51% of a reader's total decision variance consisted of random (within-reader) errors that were uncorrelated between replications, another 14% came from that reader's consistent (but idiosyncratic) responses to different cases, and only about 35% could be attributed to systematic variations among the sampled cases that were consistent across different readers.
© (1999) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Richard G. Swensson, Richard G. Swensson, Jill L. King, Jill L. King, David Gur, David Gur, Walter F. Good, Walter F. Good, } "Gains in accuracy from averaging ratings of abnormality", Proc. SPIE 3663, Medical Imaging 1999: Image Perception and Performance, (24 May 1999); doi: 10.1117/12.349630; https://doi.org/10.1117/12.349630

Back to Top