We present a novel framework for describing intensity-based multi-modal similarity measures. Our framework is
based around a concept of internal, or self, similarity. Firstly the locations of multiple regions or patches which
are "similar" to each other are identified within a single image. The term "similar" is used here to represent
a generic intra-modal similarity measure. Then if we examine a second image in the same locations, and this
image is registered to the first image, we should find that the patches in these locations are also "similar", though
the actual features in the patches when compared between the images could be very different. We propose that
a measure based on this principle could be used as an inter-modal similarity measure because, as the two
images become increasingly misregistered then the patches within the second image should become increasingly
dissimilar. Therefore, our framework results in an inter-modal similarity measure by using two intra-modal
similarity measures applied separately within each image.
In this paper we describe how popular multi-modal similarity measures such as mutual information can be
described within this framework. In addition the framework has the potential to allow the formation of novel
similarity measures which can register using regional information, rather than individual pixel/voxel intensities.
An example similarity measure is produced and its ability to guide a registration algorithm is investigated. Registration
experiments are carried out using three datasets. The pairs of images to be registered were specifically chosen as they were expected to challenge (i.e. result in misregistrations) standard intensity-based measures, such as mutual information. The images include synthetic data, cadaver data and clinical data and cover a range of modalities. Our experiments show that our proposed measure is able to achieve accurate registrations where standard intensity-based measures, such as mutual information, fail.