The composition of atherosclerotic lesions in the carotid arteries is believed to be an important predictor of stroke risk. Several MR contrasts may be necessary to discriminate between different plaque components, and multispectral analysis can used to integrate the information obtained from these multiple contrasts. This study presents the use of registered MR and histological images of carotid endarterectomy specimens as a tool for the quantitative assessment of maximum likelihood classification and other segmentation algorithms. Carotid endarterectomy specimens were imaged in a 1.5T GE Signa scanner. PD, T1, T2, diffusion spin echo weightings were obtained. MR images were registered with digitized images of the corresponding histology. A pathologist identified regions of collagen, calcification, cholesterol, hemorrhage on the histological images. Training and ground truth regions were selected. The accuracy of the maximum likelihood classification was assessed on a pixel by pixel basis using truth regions identified on histological images. The accuracy of multispectral analysis was calcification (73%), fibrin (68%), cholesterol (62%), fibrous plaque (53%). This technique was limited by registration inaccuracies caused by partial volume effects and histological artifacts. Despite these limitations, accuracy results were reasonable. This technique, with continued improvements, provides a framework for evaluating the accuracy of different segmentation algorithms.