The purpose of this study was to compare the performances of two recently-developed image retrieval methods for
mammographic masses, and to investigate the inter- and intra-observer variability in radiologists' assessment of mass
similarity. Method 1 retrieved masses that are similar to a query mass from a reference library based on radiologists'
margin and shape descriptions and the mass size. Method 2 used computer-extracted features. Two MQSA radiologists
participated in an observer study in which they rated the similarity between 100 query masses and the retrieved lesions
based on margins, shape, and size. For each query mass, three masses retrieved using Method 1 and three masses
retrieved using Method 2 were displayed in random order using a graphical user interface. A nine-point similarity rating
scale was used, with a rating of 1 indicating lowest similarity. Each radiologist repeated the readings twice, separated by
more than three months, so that intra-observer variability could be studied. Averaged over the two radiologists, two
readings, and all masses, the mean similarity ratings were 5.59 and 5.57 for Methods 1 and 2, respectively. The
difference between the two methods did not reach significance (p>0.20) for either radiologist. The intra-observer
variability was significantly lower than the inter-observer variability, which may indicate that each radiologist may have
their image similarity criteria, and the criteria may vary from radiologist to radiologist. The understanding of the trends
in radiologists' assessment of mass similarity may guide the development of decision support systems that make use of
mass similarity to aid radiologists in mammographic interpretation.