In affluent countries, x-ray mammography screening of asymptomatic women has become routine for the early detection of breast cancers.1 An important drawback is that x-ray mammography requires ionizing radiation. Furthermore. it often fails to detect lesions in patients with radiographically dense breasts, resulting in sensitivities less than 40% for these patients.23.–4 Thus, for genetically predisposed populations, other imaging modalities are typically employed to aid in detection and classification. These include ultrasound and magnetic resonance imaging (MRI). In addition to large operator variation in images, localization can be challenging in ultrasound whereas MRI has low specificity and may prove too costly for routine screening. An alternative method with the potential for high sensitivity measurements is optical imaging of the breast.
An optical breast image is a mapping of estimates of the optical properties of a breast that is reconstructed from a set of noninvasive measurements using near infrared (NIR) light. Typical image resolution is on the order of several millimeters to one centimeter. The observed optical properties provide insight into the underlying physiology of the imaged medium. Optical breast imaging is primarily concerned with screening for and diagnosis of breast cancers along with neoadjuvant therapy response monitoring. The validity of optical breast imaging for characterizing and monitoring breast cancers is increasingly becoming established.188.8.131.52.10.11.12.13.14.15.–16 The primary research focus in this area has been the detection of changes in biochemical and physiological properties of a medium and relating them to pathology.56.–7,1011.12.–13,16 In contrast to x-ray mammography, optical breast imaging is less hampered by dense breast tissue. Moreover, there has been work on the introduction of fluorescent dyes as a contrast agent.14,15,17184.108.40.206.22.–23 These studies have shown increased fluorescence in tumors due to increased vascularization and an enhanced retention of the dye due to increased vessel permeability. Thus, fluorescence optical breast imaging could offer the potential to distinguish between malignant and benign tumors.
For in-depth reviews of optical breast imaging see Ntziachristos and Chance,24 Hawrysz and Sevick-Muraca,25 van de Ven et al.,26 Choe and Yodh,27 and Choe.28 A specific discussion of the future of the modality in breast cancer diagnostics and treatment planning when compared with competing and complimentary technologies can be found in Tromber et al.29 A broad overview and discussions on the modality’s various applications can be found in Boas et al.,30 Gibsen et al.,31 and Da Silva et al.32
Unfortunately, because of low resolution and a lack of anatomical landmarks, optical breast images can be difficult to interpret. Furthermore, characteristic imaging artifacts are often present and can obfuscate important image features. Since interpretation of the images is difficult, it is desirable to develop methods to increase the quality of optical breast images or to provide a context in which the information in the images becomes more meaningful. It is therefore desirable to develop automated image analysis tools. One of the standard image analysis problems is the registration (either inter-modality or intra-modality) of a set of images. In the context of this report, the term registration will be used to indicate the alignment of images. The term fusion will be used to indicate when registered images are merged to provide additional information.
Registering multiple optical breast images can be of interest for longitudinal analysis. Specifically, monomodal registration is of interest for neoadjuvant therapy monitoring. Registration of optical breast imaging with other modalities is also of interest because other modalities may provide complimentary information useful for diagnosis and treatment planning. Modalities complimentary to optical breast imaging can be broken down into those concerned primarily with imaging anatomy and those which image function/physiology or molecular processes. X-ray mammography and tomosynthesis, ultrasound, and magnetic resonance imaging (MRI) are used primarily for anatomical imaging, while all three are used to a lesser degree to obtain physiological measurements. Positron emission tomography (PET) and optical imaging provide molecular information and physiological information, but little anatomical information.33 Registration of optical imaging with other modalities can be used to aid in tumor localization or to validate optical breast imaging measurements. Furthermore, registration with anatomical imaging can provide prior knowledge of shape and breast composition/tissue distributions to aid in reconstruction.
This paper reviews the existing methods used in the registration of optical breast images and discusses those methods in other application areas that would be most suitable to use with optical breast data. Extensive reviews of existing multimodal registration algorithms can be found in Elsen et al.,34 Maurer and Fitzpatrick,35 Maintz and Viergever,36 Hill et al.,37 and Zitová and Flusser.38 Recent reviews of image registration techniques specific to the breast can be found in Sivaramakrishna39 and Guo et al.40 in which both intra-modality and inter-modality registration of breast images from x-ray mammography, ultrasound, and MRI are discussed.
This report has two goals. First, it provides a background on breast anatomy and optical breast imaging. Second, it reviews the relevant literature for registration of mono- and multimodal optical breast images. Where relevant, methods pertaining to other imaging modalities or imaged anatomies are presented.
Breast Composition and Optical Properties
We provide a brief overview of the structures of interest, the biological factors that affect imaging, and common spatial variations in optical properties found in breast tissue. We will confine the discussion to what is most relevant for registration. For reference, in Fig. 1 two slices from the same volume of noncontrast T1 weighted MRI of a breast of a 67-year-old volunteer paired with a schematic overview of several anatomical features are shown. Most breast images are acquired parallel to the pectoral muscle so that the largest percentage of the breast is positioned over detectors.41 Breast surface features consist of the skin boundary and the areola. Internal features include the pectoral muscle, fibroglandular tissue, fatty tissue, blood vessels, and lymph vessels.40 Figure 1 also contains an MRI breast image of a 25-year-old volunteer for comparison. Note that the younger subject’s breast has a higher density of glandular tissue than the older subject’s breast.
There has been some investigation into the biological factors that affect the acquired signal in optical breast imaging.42,43 Variation in breast characteristics among individuals can significantly affect imaging. Like many other modalities, optical breast imaging systems have difficulty resolving images at depth, owing to scattering and absorption, so a large breast may result in a lower resolution image. Furthermore, for many optical breast imaging systems, the same number of sensors is used to image breasts of all sizes, which reduces the resolution of images of large breasts in comparison to those of smaller breasts. Moreover, the composition of the breast is important as it has been shown that there is significantly more scattering in fatty breast tissue as opposed to more dense breast tissue, such as glandular tissue where there is more absorption due to increased blood content.44 Furthermore, Shah et al.45 showed the areola is unique compared to the rest of the breast. In optical breast images, the areolar region produced 60% more scattering than other regions of the breast and thus had a brighter signal in resultant images. The authors also showed that the distribution of glandular tissue plays a significant role in determining the received signal. The importance of the areola and a discussion of its appearance in optical images can also be found in Leproux et al.46
Finally, whether or not there is disease present in the imaged breast has a significant effect on the images obtained since breast cancers are highly vascularized tissues. Cancers result in higher total hemoglobin content (THC) and reduced oxygen saturation compared with healthy tissue.47 Thus, a brighter signal is often observed in cancers. Conversely, cysts can be differentiated from malignant tumors because the cyst will have lower signal intensity due to decreased absorption resulting from lower blood content. Furthermore, it is well understood that injected contrast agents will preferentially leak into cancerous tissue. Specific contrast agents are described in Sec. 2.2.4.
In this section, we will provide insight into some of the basic characteristics of optical breast images. For an extensive review of the theory and instrumentation, see Choe and Yodh.27 Optical breast imaging systems vary based on the types of measurements they make, the geometry of the acquisition, and the reconstruction algorithms used. The use of exogenous contrast agents is also discussed.
Optical imaging measurement approaches are classified as continuous wave (CW), frequency-domain (FD), and time-domain (TD). Continuous wave imaging is based on a light source that does not vary with time and measurements of the transmitted waves. Frequency-domain imaging uses an amplitude-modulated light source and measures the change in amplitude along with the phase shift. Time-domain imaging uses a short impulse of light and then measures the “impulse response” of the tissue.27 TD imaging systems are able to assess the temporal distribution of photons and thus allow discrimination between signals resulting from scattering and absorption. Thus, out of the three methods, TD imaging collects the most information about the optical properties of the medium and therefore has the best contrast and resolution. TD acquisitions take longer than the others and measurement equipment typically costs more.26 Similarly, the phase differences in FD imaging are used to distinguish between scattering and absorption. Because the acquisition is band-limited to the range of 50 to 500 MHz, less information is obtained with an FD acquisition than with a TD acquisition.48,49 The simplicity of the CW approach makes it the lowest cost data to acquire, but the inability to distinguish between scattering and absorption makes reconstruction of high quality images complicated. The measurement type used in the acquisition, therefore, has a strong impact on the ability to resolve structures in the images.
There are various measurement geometries that have been proposed for optical imaging of the breast which directly impact the field of view and the sensitivity of the used system. Parallel-plate imaging systems consist of sources and detectors on opposite sides of the breast, which is compressed to produce two-dimensional (2-D) projection views similar to x-ray mammography.27 Compression may significantly alter blood flow, and thus obfuscate measurements. Therefore, minimal compression tomography systems have been developed consisting of sources and detectors positioned all over an imaging cup, in which the breast is suspended. Finally, hand-held devices that operate similarly to ultrasound probes have been developed and are reviewed in Erickson and Godavarty.50 Figure 2 shows schematic representations of each imaging geometry.
The acquisition geometry used in each instrument naturally determines the geometry of the breast in the images. Since the breast is deformable, the position/orientation of the breast in the scanner (e.g., whether the patient is upright, prone, or supine) and any compression applied will have a significant effect on the shape of the breast and the locations of important features in the images (e.g., the nipple, tumors) and is thus, an important consideration for registration.
The goal of an optical breast imaging system is to acquire peripheral NIR measurements which are then used to estimate a distribution of optical properties. There are several algorithms that have been applied to determine optical properties from diffuse measurements of photon propagation.5220.127.116.11.56.–57 The algorithms used to accomplish this vary concerning the choice of photon propagation model, acquisition geometry, and optimization paradigm.27 Because the reconstruction used will largely determine the image quality, the choice of reconstruction method is an important consideration.
There is diverse literature on reconstruction methods for diffuse optical tomography. Because a thorough discussion of all aspects of reconstruction methods is outside the scope of this report, we refer to Arridge and Schotland58 for a thorough review of this topic.
A common paradigm currently used in optical tomography reconstruction is the introduction of shape as prior knowledge. The idea is to provide a segmentation of an object in the optical tomography volume either through a model of the expected shape or through prior knowledge introduced from another modality. Within different regions, optical parameters will vary differently, but often not much, while there may be significant jumps across interfaces, thus motivating direct shape-based reconstruction. These prior shapes are introduced either explicitly through parametrized models or implicitly through level set methods.58 Another approach is to use information from other imaging modalities to anisotropically smooth regions while leaving boundaries intact.5960.–61 Information on theoretical measures have been considered, as well as a means to relate reconstructed images to prior knowledge from aligned anatomical images.62
Contrast agents have been developed to improve the sensitivity and specificity of optical breast imaging, like for many other imaging modalities. These contrast agents are fluorescent dyes that leak into tissues in the breast, specifically highly vascularized tissues, such as cancers. Once excited, the fluorophore emits light. The obtained signal is then filtered to detect only the fluorescent signal. It should be noted that when fluorescence optical breast imaging is available it can be easily co-acquired with absorption images. Therefore, the images are in the same orientation and the registration of either the absorption or fluorescence image to another data set provides a valid transformation for both.
To our knowledge, there has been no previous work on monomodal registration of optical breast images. Nonetheless, there have been experiments that motivate the development of automated strategies. Specifically, longitudinal studies with optical breast imaging are discussed in the literature in Jiang et al.,63 Cerussi et al.,64 and Enfield et al.65 These studies investigated the response of the cancer to primary treatment, specifically neoadjuvant chemotherapy. Optical breast imaging can be useful for quantizing functional changes in the tumor micro-environment, such as properties related to angiogenesis. Registration of a time-series of optical breast images will allow this quantification. The predictive value regarding treatment response added by monomodal registration can be used to inform whether/which types of chemotherapy are effective for a specific patient. Registration of several time-series might more precisely characterize disease progression in general.
The breast is deformable and, as previously discussed, there is moderate compression present in most acquisitions. Therefore, rigid-body algorithms may not be sophisticated enough for registering optical breast images and are not discussed in this section. Nonetheless, image resolution will determine what transforms are possible. In some cases it may prove that image resolution may be too low to provide features suitable to justify nonrigid or even affine transformations.
Registration of Optical Tomography Images
There has been some work on registering optical tomographic data for applications other than breast cancer. Specifically, there is work on registering small animal optical tomography images. Marias et al.66 align temporally acquired data using surface anatomical features. Their method registers data sets by first segmenting the animal from the background and extracting landmarks from this surface that are used to solve a thin-plate spline bending energy minimization. In Andersson et al.,67 the previous method is extended by first automatically finding surface landmarks and then introducing a dynamic time warping algorithm to define regions of similar curvature in the images, which are used for registration. These methods can be applied to optical breast data if surface landmarks are first localized in the images. As discussed in Sec. 2.1, there are several possible features that could be of interest for the registration (e.g., the nipple, the areola, and the skin boundary). Since these features alone are insufficient for nonrigid registration, the use of fiducial markers could be considered.
Other Imaging Modalities
The work just outlined provides a limited context for the registration of optical images of the breast. Additionally, algorithms used to register images from other functional and molecular modalities may be applicable to optical breast images. Of particular interest are approaches used for positron emission tomography (PET) and single photon emission computed tomography (SPECT). PET and SPECT are both low-resolution functional modalities used to characterize complimentary information to optical tomography, such as factors related to angiogenesis.68
Affine methods may be suitable for registration of longitudinal studies with the same machine because imaging geometries often produce only a light compression. The affine model allows shearing, which is specifically important with a parallel-plate geometry. Once again, it is important that the breast is nearly identically placed in the scanner for each acquisition. While the affine approach requires the assumption that the breast is incompressible, it is likely flexible enough for most registration challenges with optical breast imaging. Examples of reliable affine registration algorithms used with PET data include Unser et al.69 who optimize the sum of absolute differences (SAD) for registering brain images and Klein et al.,70 who optimize the sum of squared differences (SSD) subject to a penalty derived from adjacent frames in an imaging sequence and apply the method to respiratory-gated cardiac imaging. While not used with breast images, these algorithms provide a starting point for further investigation.
In situations where the imaging environment can not be easily duplicated in successive scans, such as therapy monitoring where a tumor may change size, or the incompressibility assumption will not hold, fully nonrigid approaches may be more suitable. Thorndyke et al.71 introduce a method to improve PET image quality by using B-spline-based registration to combine pancreas and liver images. Similarly, Rubio-Guivernau et al.72 use a B-spline-based algorithm to estimate deformations and generate super-resolution images of respiratory-gated abdominal PET images.
Due to limited image resolution, it may be desirable to rely on more sophisticated registration algorithms that incorporate prior knowledge regarding the object being registered. For example, Debreuve et al.73 developed a nonrigid registration algorithm for myocardial-gated SPECT by registering level sets of the presegmented organ boundary. This could be advantageous for optical breast imaging registration as well, because the registration is based on the shapes of the objects of interest rather than image features or intensity profiles that may be hard to discern as a result of poor image resolution. Other groups have also sought to incorporate prior knowledge. Murillo et al.74 use a mutual information (MI) based registration algorithm for cardiac SPECT images but extend the method to work better with low-resolution data by introducing a sophisticated interpolation designed to take into account the pose of the object of interest when calculating partial volumes. Jager et al.75 introduce a fully nonrigid registration using a spatially dependent regularizer created by first segmenting the data to constrain the deformations of regions with higher tracer concentration. The authors show this approach outperforms regularization based solely on curvature. Both of the previous methods are of interest because they provide context to the registration that may be absent in optical breast imaging data.
Also potentially of interest is Ouksili et al.76 who first register PET lung images to CT and then register the CT images to obtain a deformation map that is then applied to the PET data. This approach is particularly desirable if optical breast images are co-acquired with another modality because correspondences between the optical breast imaging and the other modality would be known and then the higher quality image could be used for the registration. It may also prove simpler to register the optical breast images with the higher resolution images than with each other.
Registration with Other Modalities
It is often desirable to register optical breast images with either anatomical data or with other functional imaging techniques because both types of data may provide complimentary information regarding pathology or pathophysiology. Anatomical information can be used to better understand the geometries of imaged structures, and may be used to improve optical breast imaging reconstruction algorithms. Methods, not specific to the imaging modality used to determine prior knowledge, for including this information in reconstruction algorithms have been introduced by Intes et al.77 and Guven et al.78,79 In all of these methods, the reconstruction is posed as a Bayesian estimation problem where anatomical information is treated as a prior. Panagiotou et al.62 introduce an information theoretic regularization approach for incorporating anatomical data into the reconstruction. Finally, Jagannath and Yalavarthy80 introduce an algorithm that relies on a 3-D patient breast model that is used to incorporate spatial variation in the refractive index into the reconstruction. Anatomical modalities that have been used with optical tomography currently include x-ray mammography and tomosynthesis, MRI, and ultrasound. While they also provide complimentary information, the primary reason for registration with other functional modalities is for validation purposes. Functional modalities that have been used with optical breast imaging currently include PET and DCE-MRI.
Furthermore, there has been extensive work on the development of hybrid systems for simultaneous acquisition.24 Optical breast image acquisition in these hybrid environments is often limited by the requirements of the other imaging modality.81 For example, other imaging modalities may impose constraints on the acquisition geometry, the placement and configuration of sensors, as well as the materials of which the imaging apparatus is made. It is worth noting that these hybrid systems are often cost prohibitive and typically not available. Therefore there is interest in the registration of nonconcurrent images from optical breast imaging and other modalities.
With X-Ray Mammography
While optical imaging has been pursued often as an alternative to mammography for the diagnosis of breast cancers, it has been suggested that fusion of the two modalities may increase sensitivity.82 X-ray mammography is limited in that it only provides structural information, while optical breast imaging suffers from low spatial resolution. The combination of both modalities has the potential of addressing both these issues. Specific advantages of optical breast imaging in conjunction with x-ray mammography are addressed in Collettini et al.82 The authors show that a radiologist’s ability to detect breast cancer is improved with the addition of optical breast imaging compared with detection using only x-ray mammography. The study also showed a negligible effect on specificity.
Devices have been developed for the co-acquisition of x-ray mammograms and optical breast images.8384.–85 Li et al.83 employ a hard compression setup to obtain optical and x-ray mammography simultaneously. The x-ray mammography data is used only in the optical reconstruction. Zhang et al.,84 in an extension of the previous work, explicitly use the aligned x-ray mammograms as a means to validate optical breast images. Furthermore, the x-ray data was used to determine if optical contrast was within a lesion. Fang et al.85,86 use information about the structure of the breast and its compression in the imaging system drawn from x-ray mammography in the reconstruction of the optical images. A drawback of combining these methods is that there are concerns that hard compression may reduce a tumor’s hemodynamic signatures, which are beneficial for diagnosis.27
To the authors’ knowledge there have been no attempts to automatically register nonconcurrent x-ray mammograms and optical breast images. Nonetheless, while x-ray mammography uses ionizing radiation, it is still the predominant imaging modality for breast cancer and it can be expected that x-ray mammography will be used extensively well into the future. Because of its availability and the complimentary nature of the data, the automated registration of nonconcurrent optical and x-ray mammograms is a desirable scientific goal.
With Magnetic Resonance Imaging
It is additionally of interest to register breast optical breast imaging with MRI. Figure 3 shows optical breast imaging data aligned with MRI of the same breast.
A primary application of optical breast imaging aligned with MRI, whether concurrently acquired or not, is in the reconstruction of optical breast images.60,79,81,8718.104.22.168.92.–93 For example, Brooksby et al.60,87,88 use MRI data to accurately define the imaging volume, regularize the reconstructed contrast to be optimal over an area of interest in the MR image deemed suspicious, and reduce the number of unknown parameters in the reconstruction by segmenting the tissue types visible in the MRI. Azar et al.81,9091.92.–93 register nonconcurrent optical breast imaging and MRI to be used in reconstruction. Boverman et al.89 similarly used segmented MRI to guide reconstruction. Guven et al.79 developed a hierarchical Bayesian approach to incorporate noise models of spatial intensity discrepancies between MRI and optical breast imaging.
Moreover, it is desirable to register optical breast imaging with dynamic contrast enhanced (DCE)-MRI because they both provide information about tumor blood content and vasculature.94 DCE-MRI could also be a good choice as a validation data set for optical breast imaging. Furthermore, while DCE-MRI has been shown to be useful for monitoring tumor size and vascularity during neoadjuvant chemotherapy, its cost is prohibitive for regular imaging.94 Thus, optical breast imaging may serve well in its stead for more frequent imaging.
Co-acquisition of optical breast imaging data with MRI has been pursued to avoid the need for automated registration. One of the early optical breast imaging-MRI systems, introduced by Ntziachristos et al.,95,96 used soft compression in a parallel-plate geometry and identified and diagnosed tumors with both modalities. While MRI has been shown to be useful to define the extent of disease, its high cost has made it impractical for regular monitoring,94 so it may be desirable to obtain these data sets separately and obtain optical breast images with higher frequency. Another significant limitation of co-acquiring optical breast imaging and MRI is the inability to use ferro-magnetic instruments in MRI, thus requiring extensive customization of the optical breast imaging instrumentation.81
Because the automated registration task is challenging and it is time-consuming and cumbersome to manually align data sets, the literature on quantitative comparison of nonconcurrent optical breast imaging and MRI is somewhat limited. Choe et al.94 directly compare tumor metastasis as a result of neoadjuvant chemotherapy by measuring volume changes in segmented tumors. They had an expert segment the MRI and used simple thresholding for the optical breast images. The authors further showed that the optical breast imaging results track well with the MRI measurements. Shah et al.97 performed, in essence, a manual registration by making spatially diffuse optical spectroscopic measurements for various optical parameters at the locations of fiducials placed on the breast during MR imaging. This allowed quantitatively relating physiological maps derived from optical imaging with structural features of tumors from MRI. Current MRI-optical registration methods can be categorized as intensity-based and landmark-based.
Azar et al. have developed a general software package for automated registration between several modalities.81,9192.–93 They apply this method to optical breast imaging and MRI to produce fusion images showing structural and functional data and propose incorporating their registered MRI data into optical breast imaging reconstruction in future work.81 They register the images by maximizing the mutual information of the images. The authors’ approach is a hybrid of the methods reported in Cain et al.98 and Chan and Chung99 and consists of the computation and registration of 2-D projection images from each volume. Registration is limited to affine transforms. They observe that the dominant transform between MR and optical breast imaging is along the axis on which the breast is compressed. In their setup there is lateral compression in the MR and axial compression in the optical breast imaging systems. They posit that an affine transformation is enough to account for this motion. Furthermore, they state that their optical breast images do not possess enough structural information to compute a free form deformation between the modalities.
Rather than registering based on intensity, it may be of interest to use landmarks for registration. Hsiang et al.100 register optical data with DCE-MRI by using fiducials on the breast surface and performing registration using a deformation field derived from the motion of the fiducials. This is similar to the method presented in Krol et al.101,102 for PET/MRI registration. Fiducial skin markers are placed on the breast that can be observed in both MRI and PET (Krol)/optical (Hsiang) to determine geometric displacements and use a finite-element-method to model the breast tissue for the registration. They first segment the MR image and then generate a 3-D finite element mesh using coronal contours from the segmentation. Displacement vectors are calculated from the motion of the fiducials between acquisitions. A dense displacement field is then calculated by solving a steady state heat transfer problem to yield a smooth displacement field.
The primary strength of optical breast imaging is the functional and molecular information it provides for diagnosis, but it is limited primarily in that multiple scattering dominates in the resultant signal, thus making localization and quantification of optical properties challenging.103 Combined optical imaging with ultrasound (US) can provide tumor localization.
A proto-type system fusing optical breast imaging and US data for making hand-held measurements is introduced in Zhu et al.104 The authors install optical detector fibers on an acoustic probe and demonstrate their method with a breast phantom. Holboke et al.105 and Zhu et al.106 apply optical-US co-acquisition to breast cancer imaging. The authors of both works are concerned with the introduction of optical parameters and the increased sensitivity that their data can provide for breast cancer detection. Another benefit is that optical-US co-acquisition is relatively inexpensive and, since there is no ionizing radiation, noninvasive.
Chen et al.107 and Zhu et al.103,108109.110.–111 have developed optical-US co-acquisition systems that fuse the modalities by using the co-acquired US to localize breast lesions while the optical breast imaging system produces total hemoglobin concentration maps which are used for diagnosis.103 Using the previous approach, several types of tumors have been identified and diagnosed.108,110
While noninvasive and inexpensive, optical-US imaging systems suffer from poor image quality leading to poorly defined breast boundaries in US that make reconstruction using this information more difficult than with the previously discussed anatomical imaging modalities.27 Furthermore, obtaining optical breast images simultaneously with US results in constraints imposed on optode locations and numbers when integrating the optical breast imaging device with the US transducer.81 This can result in lower quality optical breast imaging data.
To our knowledge, there has been no attempt to automatically register nonconcurrent optical breast imaging and US images of the breast. This is a challenging problem because scattering dominates the US signal. Furthermore, characteristic imaging artifacts are ubiquitous in both modalities. Finally, the US acquisition results in a limited field of view making registration more difficult.
With Positron Emission Tomography
The comparison of optical breast images with PET images of the same subject is particularly interesting because clinical optical breast imaging is in its infancy and often seeks to quantify functional characteristics that are directly related to parameters that are measured by the more established PET imaging. For this reason, PET could be the ideal modality for validating optical breast imaging results. Furthermore, the registration of FDG-PET and optical breast imaging provides the ability to quantify total hemoglobin, blood oxygenation, and glucose metabolism for the same voxels, thus leading to higher sensitivity and specificity in tumor diagnosis.112
To our knowledge, no systems have been developed that simultaneously obtain breast PET and optical data. Optical breast imaging is noninvasive and relatively low cost while PET is expensive and requires considerable amounts of ionizing radiation. Thus, a hybrid PET-optical system may be unwarranted for routine use.
There is only one published work on PET/optical breast registration. Konecky et al.112 compare optical breast images with breast-only and whole-body PET. Their registration is complicated by the fact that the breast is freely hanging in the breast PET scanner, while a mild compression is applied in their optical breast imaging scanner. The authors use the method of Azar et al.81 described in Sec. 4.2 to register the images. The authors warp the optical breast image volume into the PET space. Because of the resolution limitations of both data sets, the authors limit their transforms to rigid-body motion and scaling.
The authors were unsuccessful at registering the whole-body PET with optical breast imaging. The whole-body PET is obtained with the patient supine, rather than prone, so there is significant compression against the chest that the authors believe led to their method failing to register the data sets. Nonetheless, they show a correspondence between FDG up-take and optical parameters from optical breast imaging. Furthermore, breast PET and optical breast imaging led to similar spatial localization of lesions. The whole-body PET could only be used to show similar contrast because of the inability to register it to the other data.
Discussion and Conclusions
The registration of optical breast images either with each other or with images from other modalities is an important step in improving the clinical value of optical breast imaging. Monomodal registration can allow for quantitative longitudinal analysis of breast lesions, while multimodal registration can be used to aid in the reconstruction of more accurate optical properties, provide a means for localizing tumors, or validate optical breast imaging measurements. Multimodal registration also allows for the fusion of complementary functional and anatomical information. In this review, we have presented a survey of the state-of-the-art in optical breast imaging registration and have highlighted concerns specific to either optical breast imaging acquisition or breast imaging relevant for framing the registration problem. This review considered the literature on monomodal registration and multimodal registration of optical breast images. While there is relatively little past work on fully- or semi-automated approaches to register optical breast imaging data, we have shown a growing body of research in this area and a need for the development of registration algorithms where absent or under-explored.
A clear challenge for registering optical breast imaging data are the design choices involved in stating the registration problem. Variability across patients in breast size and radiographic density (i.e., optical density), determined by differences in the ratio of fat to glandular tissue, results in an uncertainty regarding morphological or intensity characteristics that can be expected in optical images. This is primarily a problem for multimodal and inter-patient registration. Furthermore, optical breast imaging data has a spatial resolution of several millimeters that is highly dependent on the imaging paradigm used. Resolution is further dependent on breast size and acquisition geometry. Thus, image features desirable as registration landmarks that are present in one image may be blurred out in another. Imaging geometry also has a significant effect on the types of deformations seen between images, thus further complicating the registration problem.
While not yet investigated for optical breast image registration, biomechanical models may allow the introduction of more sophisticated deformation models into a registration framework. If the acquisition geometries are well understood, then a prior deformation model could be determined. Previous work is largely based on the finite-element-method (FEM), which has been used for multimodal registration by Ruiter et al.113 Furthermore, recognizing that the mechanical properties of the breast are largely unknown, Tanner et al.114 evaluated several FEM breast models showing they accurately captured breast deformation. It would be interesting to see if these models are beneficial for registering optical breast images.
Automatic algorithms have not yet been developed for monomodal registration of optical breast imaging. Nonetheless, their development is motivated by longitudinal clinical research studies and by the possibility for quantitatively studying inter-patient variability in optical breast imaging. There are several relevant algorithms that have been used for PET or SPECT, which have similar spatial resolution as optical breast imaging and measure comparable physiologic and molecular processes. These methods should translate well to optical breast imaging applications and provide a wealth of new information.
To date, co-acquisition seems to be the dominant approach for registering optical breast images with other modalities. This may be desirable for many clinical applications and provides a simple and direct means of comparing optical breast imaging data with other modalities, mainly for validation purposes. However, acquiring optical breast imaging data simultaneously typically involves imposing constraints on the system that may reduce its efficacy. Furthermore, some hybrid systems may be cost prohibitive or unavailable. In these situations, nonconcurrent imaging is required.
Optical breast image registration is a relatively new and challenging problem. The nonrigid and poorly understood mechanics of the breast along with differences in image geometries, imaging strategies, and inter-patient variations will continue to challenge registration algorithms. There are no clear general methods for registering optical breast imaging with itself or with other modalities as of yet and it can be expected that, as this new modality becomes more clinically relevant, there will be a growing demand for solutions.
This work was supported by the Center for Translational Molecular Medicine–Mammary Carcinoma Molecular Imaging for Diagnosis and Therapeutics (CTMM-MAMMOTH) project and by the Dutch Cancer Society KWF by a research fellowship to S.G.E.