1.
Introduction
Breast cancer is the most common cancer among women and is the second leading cause of cancer deaths worldwide.1,2 Treatment of breast cancer is difficult because breast cancer encompasses many genotypes and phenotypes that affect risk, diagnosis, prognosis, and treatment response,34.–5 and these different cancer types can be hard to quantify with current medical imaging methods.45.–6 Quantifying breast cancer can ultimately assist researchers in answering difficult questions on why such variation exists in patients in the prevalence, aggressiveness, treatment, and outcomes of primary and metastatic disease.7,8 This problem is relevant throughout all stages of the disease, as a patient’s tumor can evolve into new variants through environmental pressure and epigenetics, creating a tumor with multiple regions that respond differently to therapy.5,9,10 Addressing this problem requires new imaging methods to measure the cellular, structural, and morphological differences expressed by different types of cancer. Researchers then need to incorporate these measurements into new quantitative cancer models, for use in classifying different cancer types. As such, improved imaging methods for quantifying breast tissue are desirable from a research and clinical perspective, offering improved understanding of breast cancer and patient care.5,1112.–13
However, there is a major barrier to quantitative characterization of breast cancer; biomedical imaging has an inverse relation between the volume any imaging modality can cover (field of view and penetration depth) and the size of details it can visualize (spatial resolution).1415.–16 Due to this, clinical imaging modalities are often restricted to a single range for the combined resolution, field of view, and penetration depth (the spatial scale). In practical terms, this means that a given imaging modality usually acquires information on either the cellular scale, tissue composition scale, or organ/animal level. In several fields, including neuroscience and oncology, these corresponding spatial scales have been referred to as the microscale, mesoscale, and macroscale, respectively (Table 1).1718.19.20.–21 These scales and definitions are still evolving, and by no means absolute, but can be useful groupings for categorizing imaging technology.
In the context of a problem such as breast cancer imaging, each imaging scale can yield different and useful insights into the disease process.
Microscale imaging reveals the cellular composition of the tumor, its extracellular matrix, and benign tissue surrounding it; all of which affect disease risk, development, progression, and metastasis.10,11,2223.24.–25 However, microscale imaging requires invasive procedures and does not characterize the entire tumor.24,26 Mesoscale imaging can provide real-time information on cancer extent during surgery, but mesoscale imaging in general is in development at the preclinical stage and is not widespread.19,27 Macroscale imaging is predominant clinically and can obtain metrics over an entire tumor or organ. These macroscale metrics are used by prospective computer aided detection (CADe) and computer aided diagnosis (CADx) systems that would assist physicians in detecting and characterizing cancer.7 Similarly, metrics based on breast density and patterns of the fibroglandular tissue are important risk factors for developing breast cancer,2829.–30 play a role in diagnosis of cancer,31,32 and can serve as prognostic biomarkers for certain treatments.3334.35.36.–37 Despite this, macroscale imaging cannot resolve small variations and so fails to detect differences in many cancer phenotypes. Imaging at multiple scales can overcome some of these limitations and can offer a more complete understanding of breast cancer biology and development.
Table 1
Three imaging scales for multiscale breast imaging: the cellular microscale, the tissue mesoscale, and the organ macroscale. These definitions are based on usage in prior literature.17–21 The thresholds for defining these scales are unique to this study and meant to be illustrative and not restrictive.
Scale | Resolution (μm) | Penetration depth (mm) | Field of view (cm2) |
---|---|---|---|
Micro | |||
Meso | 2 to 100 | 1 to 100 | |
Macro |
Quantitative multiscale imaging of the breast (QMIB) is an emerging field of research which encompasses systems and techniques capable of imaging the breast at multiple spatial scales. A large number of modalities exist to image the breast, but until recently there have been few modalities or techniques to make quantitative comparisons between scales.17,19,3839.40.41.42.–43 The field’s relative youth means that the quantitative links among the three scales are not well defined, leaving large gaps in our knowledge.44,45 For example, breast density is measured by clinical macroscale imaging, but the biological reasons it contributes to breast cancer risk and progression are still unclear. Fully characterizing breast density will require quantitative links to the micro and mesoscale, where the biological processes take place.13,45 However, many useful tools to do so are currently limited to research modalities.22,4647.–48 QMIB seeks to address these problems, but QMIB is in its relative infancy. These factors create a fertile ground for the development and application of multiscale imaging research in the near future, making it essential to forward the dialogue on the technology, methods, and overall direction of QMIB.
The authors hope that this review can serve as a useful introduction to this area of research and will spark further discussion on multiscale imaging of the breast. This review shows that QMIB is a growing research topic and that the technologies involved make it relevant to many areas of breast cancer research. In addition, while the primary focus of the review is on breast cancer, many of the topics are relevant to multiscale imaging of other diseases and organs. The review starts with a discussion on quantitative imaging and the challenges of quantification in multiscale imaging. We then move into preclinical imaging modalities used in QMIB research. We mention, but do not extensively cover, clinical imaging, as there are many excellent reviews on clinical imaging of the breast.12,46,4950.–51 Finally, the review describes issues in multiscale data analysis and image processing. We then discuss the current state of the multiscale breast imaging research and possible future directions.
2.
Quantification and Quantitative Multiscale Imaging of the Breast
Quantitative imaging is “the extraction of quantifiable features from medical images for the assessment of normal or the severity, degree of change, status of a disease, injury, or chronic condition relative to normal.”52,53 Based on this definition, quantitative imaging can be divided into two categories. The first is the quantitative analysis of the data in an image. For example, quantitative analysis of a standard mammogram can give a value known as the percent mammographic density (PMD). PMD is a measure related to the extent of fibroglandular tissue and is correlated with breast cancer risk.54 However, the standard mammography PMD can only approximate the actual volume and proportion of fibroglandular tissue in the breast.55 The second method of quantitative imaging is making quantitative measurements of biology. Volumetric breast density, obtained using a quantitative three-dimensional (3-D) modality or through supplemental mammographic techniques, is a direct measure of the fibroglandular tissue and so can be quantitative by both definitions.56
There are many challenges associated with quantitative imaging, and addressing these challenges is a major issue in medical imaging. There are detailed reviews of this subject published by the quantitative imaging biomarkers alliance.8,52,53 In brief, quantitative data acquisition is difficult because a measurement must be based on a physical value, can be affected by many sources, and must be considered statistically. Thus, a measurand should be traceable to a reference value, be repeatable, reproducible, have known components of estimate variance, and should have known estimate bias. The reference, typically obtained with a digital or physical object with known properties (a phantom) connects the measurement to a physical value. A repeatable measurement is one that yields the same result under the same conditions. A reproducible measurement is one that can be acquired by a different observer using different equipment, yet still achieve a similar result. Yet, no measurement is perfectly repeatable and reproducible. The difficulty of these challenges varies with each imaging modality and should be recognized for every quantitative study.53
QMIB faces all the normal challenges associated with quantitative imaging but also introduces other difficulties due to comparisons across spatial scales. QMIB is frequently multimodal, utilizing multiple imaging modalities. Multimodal QMIB faces all the inherent difficulties specific for each modality in addition to their integration into a combined imaging framework across spatial scales.
2.1.
Image Acquisition
The multimodal nature of QMIB further complicates image acquisition due to the technical and procedural requirements for all modes. Among other variations, the images can be taken at separate time points, under tissue deformations that must be corrected for, may use different contrast agents, or are in vivo in one mode and ex vivo in another. The conditions may not be held constant from one session to the next and human error or processing artifacts can introduce unknown changes to the setup. The imaging time also becomes a large concern for in vivo applications. The longest scan time of all modalities limits the time resolution of studies. Encouragingly, QMIB has made much recent progress due to developing quantitative imaging technology and methods that can address these challenges.
2.2.
Data Analysis
The fundamental disparity of spatial scale in QMIB complicates data analysis. QMIB can require orders of magnitude in higher processing time than single-scale imaging due to large datasets and a need for multivariate analysis. This imposes constraints on real-time imaging and currently makes many QMIB methods impractical for widespread use. For multimodal QMIB, a single voxel in a macroscale image can represent several whole microscale images. This causes partial volume artifacts and makes it difficult to delineate the boundary on the microscale image that corresponds to the macroscale voxel, contributing uncertainty further down the data analysis pipeline. Additionally, in multimodal QMIB the modalities may not have the same biophysical contrast mechanism, e.g., tissue acoustic scattering for acoustic imaging verses molecular composition for optical imaging. This makes multimodal QMIB well suited to quantitative studies where it can measure different components of tissue models and how they interact, but characterizing the ground truth of interactions between those sources of contrast is a research area in and of itself.53
3.
Quantitative Multiscale Imaging of the Breast Modalities
This review focuses on preclinical imaging modalities (Table 2), as preclinical modalities drive QMIB research. Multiscale imaging usually combines multiple imaging modalities, with each modality operating over a single spatial scale (Fig. 1). Each scale contains preclinical breast imaging modalities; however, the major clinical modalities are at the macroscale and need to be combined with a preclinical modality for multiscale imaging. Thus, a discussion of preclinical modalities covers the instances where clinical modalities are used for QMIB (Table 3). In addition, many clinical modalities are mentioned in sections for related preclinical modalities. Readers interested in more detail on these clinical modalities may reference several other reviews dedicated to clinical breast imaging.12,46,4950.–51
Table 2
Preclinical multiscale breast imaging modalities (order listed as presented in this review). A practical assessment of imaging modalities for multiscale imaging of the breast. The characteristic measured column describes the information acquired from the tissue. The form factor describes the imaging equipment; the breast or sample is placed inside a cylindrical bore, examined using an external probe, or in the case of tissue placed on a microscope stage. The prospective clinical use describes proposed uses for the technology in patient care, based on current literature.
Imaging modality | Characteristic measured | Form factor | Advantages | Disadvantages | Prospective clinical use |
---|---|---|---|---|---|
Standard microcomputed tomography () | Density | Cylindrical bore | Mature technology, inexpensive, developing dedicated breast systems | Ionizing radiation, macroscale resolution clinically, geometric artifacts, electronic noise artifacts | Rapid ex vivo tumor margin detection,57 tumor staging,58 biopsy analysis59 |
Spectral and photon counting (SPC)- | Density | Cylindrical bore | Whole breast FOV, no geometric artifacts, no electronic noise artifacts, developing dedicated breast systems | Ionizing radiation, slow imaging speed, quantum noise | Diagnostic screening60 |
PhC-, synchrotron source | Refractive index | Cylindrical bore | Whole breast FOV, developing dedicated breast systems | Ionizing radiation, high expense, and limited availability | Diagnostic screening61 |
PC-, x-ray tube source | Refractive index | Cylindrical bore | Whole breast FOV, inexpensive x-ray source, developing dedicated breast systems | Ionizing radiation | Diagnostic screening,62 breast density quantification63 |
HF-US | Mechanical properties | External probe | Noninvasive, inexpensive, commercial preclinical systems | Can be subject to operator artifacts and sensitive to instrumentation differences | Computer aided detection64 and classification65 Image guided biopsy,66 treatment response imaging67 |
MRM | Molecular environment of hydrogen and other resonant elements | Cylindrical bore | Noninvasive, multicontrast | Long imaging time, high expense, preclinical only | Ex vivo IMA68 |
3D-QHP | Various, based on the stain used | Microscope stage | Multicontrast, qualitative HP is the gold standard | Ex vivo only, slide artifacts, destructive to tissue, long processing time | Computer aided detection or prognosis46 |
LSM | Various; modality dependent | Microscope stage or external probe | Noninvasive, multicontrast | Preclinical only, slow imaging time, submillimeter penetration depth | N/A |
WFM | Various; modality dependent | Microscope stage or external probe | Rapid imaging speed | Millimeter penetration depth | IMA69 |
OCT | Refractive index, optical scattering properties, mechanical properties | External probe | Mature technology, inexpensive, noninvasive, rapid imaging, endoscopy and biopsy needle compatible probes | Millimeter penetration depth | IMA,70 image guided biopsy70 |
PAT | Fluorophore concentration, optical scattering parameters | External probe | Noninvasive, multicontrast, intrinsically multiscale, commercial preclinical systems | Requires separate probes to image at multiple scales, significant noise | Treatment response imaging71 |
DOT | Fluorophore concentration, optical scattering parameters | External probe or cylinder bore | Noninvasive, multicontrast | Very low resolution, no commercial systems, variety of implementations | Supplemental screening,72 treatment response imaging,73 breast density assessment74 |
FMT | Fluorophore concentration, optical scattering parameters | Cylindrical bore | Noninvasive, multicontrast, commercial preclinical systems | Quantification artifacts, preclinical only | N/A |
DLIT | Cellular luciferase production | Cylindrical bore | Noninvasive, high specificity, commercial preclinical systems | Quantification artifacts, requires transgenic mice or pathologies, preclinical only | N/A |
Fig. 1
Multiscale imaging uses multiple imaging modalities to operate across two or more spatial scales. This typically requires preclinical modalities, as most clinical modalities are at the macroscale. The breast imaging modalities are colored by their predominant use in the literature. Green modalities are clinical, and blue are preclinical. The limits for each modality were determined through breast cancer imaging literature and do not reflect performance in other applications.66,71,7576.77.78.79.80.81.–82

Table 3
Resolution and multimodal combinations in QMIB (order listed as presented in this review). The resolutions listed in this table are for demonstrated breast imaging applications and do not necessarily reflect the capability of the modality in other applications. All modalities have been combined and validated with qualitative histopathology, and many with 2-D quantitative histopathology, so these combinations were not included in the table.
Imaging modality | Resolution (μm) (small animal/whole breast) | Microscale FOV | Mesoscale FOV | Macroscale FOV |
---|---|---|---|---|
Standard microcomputed tomography () | 4083/N/A | N/A | MRM,84 FMT,85 OCT, 70 | PET,86 SPECT,87 MRI,17 US, radiography, PAT88 |
SPC- | 10089,c/100b,60 | N/A | N/A | N/A |
PhC-, synchrotron source | 3.25c,90/30b,78 | N/A | N/A | N/A |
PhC-, x-ray tube source | 10091/80b,62 | N/A | FMT91 | N/A |
HF-US | 566/N/A | Transmission electron microscopy 92 | PAT71 | ,93 US66 |
MRM | 7582/N/A | FM82 | ,84 PAT | MRI94 |
3D-QHP | a/N/A | N/A | WFM40,95 | MRI40 |
LSM | a/N/A | N/A | WFM96 | Radiography97 |
WFM | Various mesoscale/N/A | 3D-QHP40,95 | MRM40 | MRI40 |
OCT | 1243/N/A | Optical coherence microscopy43 | WFM98 | US99 |
PAT | 4571/190077 | FM71 | HF-US71 | MRI,100 PET,100 DOT73 |
DOT | N/A/200079 | N/A | N/A | PAT,73 MRI,101 x-ray tomosynthesis,102 US72 |
FMT | 2080,103/N/A | N/A | ,85 DLIT80 | MRI,104 SPECT105 |
DLIT | 2080,103/N/A | N/A | ,106 FMT80 | MRI,10680 |
a
Standard implementations of microscale optical modalities are limited by the optical diffraction limit, which is dependent on the wavelength of light used and the numerical aperture of the objective.
b
Advanced μCT has been demonstrated on mastectomy samples, but not non-invasively with patients.
c
No breast specific applications were found and so this resolution is from imaging other organs.
Most current QMIB research features mesoscale imaging modalities (Table 3).18,27 In the near term, studies use QMIB to validate mesoscale imaging for clinical use. For example, mesoscale imaging can perform intraoperative margin assessment (IMA; the imaging of tumor boundaries during surgery). IMA can prevent the need for a second surgery, which occurs in of patients operated for a breast malignancy, and will reduce healthcare costs.107108.109.–110 In the long term, mesoscale imaging makes multiscale coregistration, the spatial mapping of one image to another, more practical. Currently, it is difficult to correlate data based on location between the microscale and the macroscale. For example, positioning a biopsy or imaging probe within a lesion often requires multiple sampling attempts.111 During the sampling process, the breast tissue can be distorted by compression or rolling of tissues to obtain access to a lesion. In addition, the orientation of the biopsied sample to the remaining macroscale tissue is not preserved. These issues can be addressed using mesoscale imaging, which is easier to register to and which can act as an intermediary between the microscale and macroscale. This can allow studies to characterize how biological characteristics express at different scales by building multiresolution maps of tissues. Breast cancer expresses many phenotypes at multiple scales that affect patient treatment, and so such characterization could lead to valuable tools and insights.3,4,112 However, accomplishing these multiresolution maps will require new data analysis methods. For example, there needs to be new methods to accurately register a sequence of images with potential deformations. These multiresolution maps will also depend on the imaging modalities involved, their technical hurdles, and potential applications.
The following sections of this review cover the current status and future perspectives for QMIB imaging modalities. It gives an overview of their biological basis and describes what quantification means to each modality. It covers how the modalities are currently represented in the peer-reviewed literature (Table 2) and how they may be used in the future. In addition, it highlights the many combinations of modalities, including several promising combinations that could bring QMIB into the clinic (Table 3).27,60,72,99,113
3.1.
High-Resolution Variants of Clinical Modalities
Several breast imaging modalities have high-resolution variants that are used in QMIB. These variants are currently preclinical but follow the same principles as their clinical counterparts. This section covers variants of computed tomography (CT), ultrasound (US), and magnetic resonance imaging (MRI).
3.1.1.
Microcomputed tomography
CT imaging utilizes x-rays passing through tissue, obtaining 3-D anatomical information by imparting radiation dose. CT is still developing for clinical breast imaging, but its two-dimensional (2-D) counterpart, mammography, is the most common breast cancer screening modality.114 Standard CT systems produce a spectrum of x-ray energies, and then measure x-ray attenuation through tissue. This obtains semiquantitative 3-D maps of the tissue attenuation coefficient (radiodensity). It is semiquantitative because using a spectrum of x-ray energies results in measurement that varies by depth. The depth variance effectively adds noise to the measurements, making quantification difficult in low-contrast situations. However, quantification is still possible in high contrast situations, e.g., extracting tumor morphology.57 This depth limitation can also be overcome by systems that calculate attenuation based on x-ray energy, also known as spectral CT.115
Most QMIB applications of CT occur at the mesoscale. Systems capable of performing mesoscale CT are labeled micro-CT (). is a well-established preclinical imaging modality with several commercially available systems.115 By comparison, has not yet reached breast imaging clinically due to technological and radiation dose limitations.115 Several groups are addressing these issues with new systems that can perform whole-breast ;50,60,62 however, they impart radiation dose to that produced from clinical mammography or digital breast tomosynthesis systems.60,116 Thus, preclinical and clinical may both prove valuable tools for future QMIB studies.
Preclinical has already been paired with many other imaging modalities for QMIB over a wide range of biomedical applications (Table 3). Some examples include characterizing the biodynamics of molecular imaging agents,83,86,87,117,118 the biological effects of therapeutic interventions,84,119120.121.–122 rapid ex vivo IMA on resected tumors or tumor morphology analysis,5758.–59,123,124 and the study of vasculature and angiogenesis.17,84,120,122,125 There is still much room to expand the preclinical applications of this technology. An excellent review of in general showcases many possibilities for future QMIB research.126 Two notable opportunities include tissue studies with multiscale nano-CT systems, which have submicron resolutions on par with those obtained through microscopy,127 and the use of contrast agents for staining antigens, providing substitutions for some immunohistochemistry (IHC) stains in vivo.127 Developments in this area of will add valuable tools to a researcher’s ability to study breast cancer, especially as they can be combined with the technology to be discussed in the following sections.
is developing clinical relevance for in vivo imaging. Systems using traditional CT technology cannot feasibly reach mesoscale resolution in the clinic, but dedicated breast CT systems based on spectral and photon counting CT (SPC-) or phase contrast (PhC-) may make clinical QMIB with a possibility in the near future.50,114 Both are prospects for clinical QMIB on a single system, as they can obtain mesoscale resolution over the whole breast. This is unique among all mesoscale modalities in this review, combining broad utility with improved ease of use over most multimodal setups while also being familiar in concept to physicians.
SPC- removes the depth dependence of standard CT, making radiodensity a quantitative measurement. In addition, SPC- minimizes geometric and electronic noise, improving contrast and resolution.60,115 Commercial preclinical systems using this technology have been released in the last few years, but clinical systems are somewhat behind due to several issues from upscaling the geometry.115 However, this difficulty is being overcome. For example, Kalender et al.60 recently published a functioning whole-breast prototype that achieved a resolution of at clinically compatible radiation doses. This system obtained 3-D voxels with higher contrast and resolution than the 2-D clinical standards of digital mammography and breast tomosynthesis. Kalender tested this system on lumpectomy specimens to find small calcium deposits (microcalcifications), the morphology and distribution of which may signify cancerous or precancerous cells. This multiscale system detected more calcifications than digital mammography and breast tomosynthesis and was better able to visualize the size and patterns due to high-resolution 3-D images (Fig. 2). Although this study focused on calcifications, the improved imaging capability may lead to earlier detection of other morphological changes that signify breast cancer. In summary, SPC- can make quantitative and multiscale measurements over the whole breast, and it has prospects for clinical use.
Fig. 2
Multiscale imaging with SPC- depicts tissue in 3-D with higher resolution and soft-tissue contrast than 2-D single-scale clinical imaging. Panels (a)–(c) show slices from an SPC- volume. Panel (d) shows a digital mammogram while panel (e) shows a breast tomosynthesis image. There are microcalcifications on each of these images that are pointed to by the white arrows, and specified in a region of interest. The volume in (a–c) has high contrast and locality due to its mesoscale resolution across a whole-breast 3-D volume. © European Society of Radiology 2016.60

PhC- derives contrast from the phase shift of the x-rays passing through the tissue.113 There are several different methods for PhC- that are used in preclinical imaging. However, clinical methods are more limited due to technological constraints.128 For example, past implementations of PhC- have imparted too high radiation doses for clinical trials, but groups have recently demonstrated acceptable doses in phantom models.62,129 Another important caveat to PhC- systems is that prior to 2013 all systems used a synchrotron as an x-ray source.113,130 A synchrotron is an expensive facility rarely attached to hospitals, so implementation of such systems would be highly limited. Encouragingly, there have been studies reporting PhC- using standard x-ray tubes and with acceptable radiation doses, which gives the prospect for a more widespread implementation.63,129130.–131 With the improved resolution of such systems, most recent breast PhC- studies have had mesoscale resolution.78,6162.–63,129,131,132 PhC- mirrors SPC- in being an upcoming monomodal multiscale system that might be implemented clinically. Although it is more difficult and costly to implement than SPC-, it also has several advantages and offers complementary information that may give both technologies a strong future in QMIB.
3.1.2.
High-frequency ultrasound
US, imaging through sound waves, is clinically friendly and is developing strong quantitative imaging capabilities. It is noninvasive, nonionizing, relatively inexpensive, portable, and can be quickly performed. In addition, it is the easiest way to image important biomechanical properties such as stiffness.31,133,134 US can perform quantitative imaging with quantitative US (QUS) and US elastography (USE). QUS can make system-independent estimate of acoustic parameters,135136.–137 such as attenuation, backscatter, and mean scatterer spacing. This removes a large source of variance, which is important for clinical application. USE measures tissue elastic properties by applying a force to the tissue and tracking the deformation.133,138,139
There are several factors that can affect the results of QUS and USE. The parameter estimates can be model-based135,136,140 or be model-free.137,141 Thus, it is important to consider, and validate, the acoustic model and the assumptions involved. In addition, some parameter estimates (e.g., strain elastography) can be heavily dependent on the user,139 whereas others can be user—and even system-independent.140141.142.143.–144 Encouragingly, there are several imaging systems that can reduce user dependence in preclinical research. For example, there are whole-breast US imaging systems in clinical trials that can perform both QUS and USE.145146.147.–148 Overall, US is a promising modality for QMIB, but researchers need to validate assumptions and experimental implementations.
US is typically separated into clinical US (2 to 20 MHz) and high-frequency US (HF-US) (). Clinical US images at the macroscale and is common in the clinic for several existing and upcoming applications.51 HF-US images at the mesoscale and has commercial preclinical instruments but has not yet reached the clinic.66,149 Both types of US can be incorporated into QMIB studies, but so far there have been few QMIB studies performed with clinical US.142
HF-US is common in multiscale imaging studies for several applications (Table 3). One notable application is improving US-guided biopsies. Clinical US systems cannot optimally visualize small microcalcifications, thus preventing accurate US-guided biopsy sampling of lesions containing microcalcifications. By comparison, HF-US does have high enough resolution to visualize microcalcifications. However, HF-US has much lower penetration depth. The lower penetration depth can be overcome by combining HF-US with needle-based probes.150151.152.–153 Cummins et al. developed such a HF probe and performed multiscale imaging with by combing with simultaneous external clinical US.66 Other applications include better characterizing phantom tissue,154 tracking cell death from macro- to submicroscales,67,92,155 detecting metastatic regions in lymph nodes,156 and characterizing contrast agent biodistribution.93
The aforementioned studies are the first few to explore this modality with QMIB, with many more potential opportunities. The parameters that HF-US measures reflect the tissue microstructure,47,137,141,157,158 which is important in breast cancer development and progression.36,37,48 Such parameters could be mapped to other modalities, thus quantifying their sensitivity to microscale structure (Fig. 3). This may allow US to detect different regions in a tumor, which may respond differently to therapy.5,9,10 In addition, the biomechanical information that USE can provide is directly important in cancer imaging, such as tumor heterogeneity,48 but can also support other imaging modalities by improving image registration models.160 Finally, QMIB can also be used to improve the models used by US at all resolutions, by comparing them to modalities that image biology on smaller scales.159 These factors make it likely that HF-US will be one of the main modalities for QMIB in the future.
Fig. 3
Multiscale imaging with HF-US and second harmonic generation microscopy (SHG) can link quantitative US measurements to different regions of collagen structure.159 Patterns of collagen alignment are prognostic in breast cancer.22 As such, the combination of HF-US and SHG may lead to clinically relevant HF-US metrics for breast cancer. This figure is an unpublished example of HF-US combined with SHG microscopy on a breast cancer biopsy. It compares an HF-US image in panel (a) to a corresponding SHG image in panel (b). The images from US and SHG are registered to create the multiscale image of panel (c). Three tumor regions with different collagen structure are enlarged in panels (d–f). The data in this figure come from the Laboratory for Optical and Computational Instrumentation and the Hall lab at the University of Wisconsin–Madison.

3.1.3.
Magnetic resonance microscopy
MRI is a noninvasive imaging modality that is highly sensitive to the relaxation rate of many atomic protons and/or neutrons, but particularly hydrogen protons, that are returning to equilibrium after they were perturbed by pulses of radiofrequency energy. The sequence of MRI excitation and signal readout segments can be assembled in varied ways to make the measurement sensitive to different tissue properties. This allows MRI to perform anatomical, functional, and molecular imaging. MRI can be implemented on the macroscale and the mesoscale. The macroscale implementation is becoming a key tool in breast cancer treatment and diagnosis.49,161 As such, there is great interest in making MRI measurements quantitative. Many researchers are tackling this problem, but there are calls for robust multicenter studies to evaluate reproducibility and accuracy.162,163 However, macroscale MRI is not used in many multiscale imaging studies. The mesoscale implementation of MRI, also known as magnetic resonance microscopy (MRM),164 requires high magnetic field strengths, fast switching magnetic field gradients, and/or long imaging times to obtain mesoscale resolution. This makes it unsuitable for clinical imaging, but this requirement can be fulfilled by commercial preclinical systems.
Preclinical MRM has been used in several multiscale imaging applications, with varying degrees of quantification. In one study, researchers combined quantitative MRM with intravital-window microscopy, studying tumor growth in 3-D and mapping it to the cellular and molecular changes that cause the growth (Fig. 4).82 This information can aid tumor therapy research, as it links the response of the individual tumor cells to the response of the whole tumor. Other small-animal research groups demonstrated multicontrast characterization of cancer,82 monitoring of therapy response,94 and visualization of vasculature and angiogenesis.84,120 A few groups studied MRM of human tissue samples. They found MRM useful for diagnosis and for ex vivo tumor margin assessment.68,165 These studies show that MRM has high potential for QMIB.
Fig. 4
Example of combined MRM and multiphoton microscopy of a mouse implanted with a breast cancer cell line and with an optical window. Multiscale imaging that combines MRM and multiphoton microscopy can link quantitative measurements of tumor morphology and growth (MRM) to corresponding cellular and molecular changes (multiphoton microscopy). This imaging can aid tumor therapy research by tracking how cellular interactions lead to whole-tumor response. Panel (a) is taken with multiphoton microscopy (through the window) and panel (B) T1-weighted MRM. The images were coregistered and two perpendicular slices from 3-D volumes are featured in the illustration. A growing tumor vessel is highlighted by the asterisk in (a), which corresponds spatially to the arrow in (b). © 2009 BioTechniques. Used with permission.82

MRM has many prospects for future QMIB studies. There are many quantitative parameters that are not currently used in breast MRM. For example, one group demonstrated quantitative maps of anatomical parameters in the brain.166 Other studies have focused on improving quantitative data collection and analysis with MRM, to replicate the clinical efforts for robust metrics, e.g., cellularity, vascular properties, and metabolites.162,163 Researchers could also look to bring MRM into the clinic. Clinical MRI units can now reach into the low macroscale in the hundreds of microns but are not quite capable of whole breast MRM in vivo.68 Seven-Tesla field-strength clinical scanners, a technology being validated in clinical trials, have greatly improved resolution in MRI but the imaging times are too long to obtain mesoscale resolution in patient imaging.49 In vivo MRM will either require significant improvements in imaging time or a higher field-strength generation of MRI scanners. If it is accomplished, MRM will become a potent tool for in vivo clinical QMIB. Until then, its preclinical implementation shows promise for many different QMIB applications.
3.2.
Optical Microsopy
Optical microscopy encompasses imaging modalities that are the primary methods for clinical screening and for research of microscale biology.167,168 Quantitative optical microscopy is still a developing field, but it is increasingly used in QMIB studies. There are a large number of optical microscopy imaging modalities, but they can be classified into a few categories based on their usage and application. This section covers QMIB using quantitative histopathology (QHP), laser scanning microscopy (LSM), and wide-field microscopy (WFM).
3.2.1.
Quantitative histopathology
Histopathology is one of the oldest techniques used in cancer imaging yet remains the gold standard for diagnosing breast cancer and for evaluating the capabilities of other imaging modalities.167 It provides significant knowledge about the tumor through tissue stains examined under light microscopy. Major stains include the hematoxylin and eosin (H&E) stains for epithelial and stromal topology, or IHC stains for molecular markers. Until recently this has been a wholly qualitative practice, with the radiologist analyzing the macroscale image and the pathologist the microscale.46 The advent of digital pathology imaging and whole-slide scanning technology over the last decade has lead to QHP.169 The development of commercial and open-source tools for QHP is also making it more accessible to researchers and pathologists.170 Several applications of QHP for breast cancer have been approved by the FDA and many others are in various stages of the approval process.46 A number of excellent recent reviews cover the subject of QHP for cancer in general and specifically for breast cancer.46,169,171,172
The primary use of QHP in QMIB is to validate imaging modalities at other spatial scales by diagnosing pathology, but there are many secondary uses. One use of multiscale QHP is quantifying the biology of MD in terms of the PMD of the whole breast41,173174.175.176.177.178.179.–180 or the local density (LD) of tissue.181182.183.184.–185 The relative proportion of highly x-ray attenuating, radio dense, fibroglandular tissue to adipose tissue (MD) is an important risk factor in breast cancer, with up to a 4- to 6-fold difference in risk between women with high and low MD women.186 It is not known whether MD is a cause or a consequence of this increased risk. If causal, elevated MD could be responsible for up to 26% of breast cancers in younger women and 16% overall.187 Unlike many other risk factors MD can be modified and is potentially a clinical target for intervention. However, it is not known what biological property of dense breast tissue affects the risk. The increased amount of epithelial cells, where most breast cancers originate, is not a definitive explanation.25,44 The percentage dense area on a mammogram is a stronger risk factor than the absolute dense area, suggesting that other mechanisms are at least partially responsible.188 Complementary theories posit that dense breasts have differences in the tissue microenvironment that result in causal or correlated changes in cellular signaling pathways, stromal organization, and other biological mechanisms that are known to affect cancer development and metastasis.26,44,189 As such, the distinction between PMD and LD is important to note for testing a hypothesis of systemic differences between high- and low-density breasts. There are no strong divergent trends according to current literature.174 However, future differences may be discovered as multiscale QHP spreads and facilitates a greater number of studies.
Other applications of multiscale QHP are still uncommon, but those that exist show the untapped potential of the modality. A recent study demonstrated the potential of examining macroscale phenotypes. Bae et al.38 quantified tumor phenotypes obtained using MRI and found that tumor roundness is an indicator of molecular traits, with negative correlations to ER and PgR statuses and a positive correlation to the cellular proliferation. Studying tumor and parenchymal pattern phenotypes with QHP is a fertile area for new multiscale research and compliments radiogenomic discoveries that link phenotype to genetic risk factors.190 There are many more types of macroscale measures, including functional and molecular characteristics that have potential to be examined using multiscale QHP.
There are several near-term challenges for improving QHP and making it a better tool for QMIB. Currently QHP is performed on only a small segment of the tissue in any macroscale voxel and so cannot be easily localized.191 It was not feasible to address this issue in the past, but the advent of high-throughput whole-slide scanning systems has opened new doors. Such scanners rapidly process dozens of adjacent histology slides, allowing reconstruction of 3-D volumes.192 At least two small animal studies use these systems for QMIB. They developed workflows to register to MRI and widefield fluorescence.40,95 There is much room to build on these and other similar methods to make 3-D QHP an accessible and valuable tool for research.18 One final limitation that should be considered is that QHP is inherently destructive to tissue and can only be done ex vivo. The tissues need to be physically sectioned into slices and placed on slides, and they need to be chemically processed to label what is being imaged. This can introduce artifacts throughout the process that effect the finished slides.193 These artifacts are minor problems for qualitative applications but heavily impact quantitative analysis, especially in prospective 3-D applications.
3.2.2.
Laser scanning microscopy
LSM is a category of optical imaging modalities that can perform noninvasive 3-D imaging of microscale tissue composition. Notable LSM modalities include confocal fluorescence microscopy, fluorescence lifetime imaging microscopy (FLIM), multiphoton microscopy (MPM), and second harmonic generation (SHG).22,81,194,195 LSM imaging derives contrast from intrinsic biology or extrinsic molecular probes.195,196 These contrast methods are nondestructive and, unlike histopathology, can be used for live imaging. In addition, the contrast options allow LSM to measure anatomical and molecular tissue composition. These measurements can be quantitative, though with varying difficulties and limitations based on the LSM implementation.22,197,198 LSM can image at depths from 100 to , depending on the tissue composition and the LSM modality. Given long imaging times, LSM technologies can also reach mesoscale field of view by stitching multiple images together into a mosaic. Altogether, these qualities make LSM one of the best research tools to characterize benign tissue and tumor microenvironments.189,199200.–201
LSM’s capability to characterize tissue microenvironments makes it a valuable tool for breast cancer research with great potential for QMIB. This has been shown in tissue sample imaging and in live small animal research. SHG can obtain prognostic information from human breast cancer biopsies.202203.–204 This prognostic information depends on microscale structure, which some mesoscale imaging modalities are sensitive to (Fig. 3).159 Thus, future QMIB studies may allow the quicker mesoscale imaging modalities to detect this prognostic information. Other human biopsy studies used SHG to quantify microscale collagen characteristics against the macroscale measure breast density, supplementing the QHP methods mentioned in Sec. 3.2.1.97 LSM can also be used to image tumor development in small animals. These studies implant optical windows over the mammary gland, known as intravital windows.82,205206.–207 These windows allow LSM to bypass the skin and image the tumor directly. For example, one group combined MPM and MRI for multiscale imaging of tumor vessel development (Fig. 4).82 These studies demonstrate how LSM is valuable for QMIB and hint at its future potential.
There are many ways LSM can be advanced for QMIB research. LSM is widely used in research but is not currently present in the clinic due to equipment complexity, difficulty of application, and long imaging times. However, this may soon change as there are groups working on producing simplified equipment, needle or endoscope compatible imaging probes, and improved imaging speeds.197,208,209 These devices can also be incorporated in multimodal imaging devices that inherently coregister the image, greatly easing multiscale research. Researchers can also tackle the depth limitation in tissue samples using physical sectioning, as is done in 3-D QHP. Physical sectioning with LSM would introduce tissue processing artifacts, but these may be lessened compared with 3-D QHP due to greatly increased section thickness.193 Finally, there is a general need for quantitative imaging standards and improvements to adapt LSM for robust imaging research and clinical use.22,197,198
3.2.3.
Wide-field microscopy
Many optical microscopy imaging modalities can be implemented in wide field, sacrificing resolution to achieve mesoscale field of view but not improved penetration depth. Examples include fluorescence, polarimetry, FLIM, and near-infrared light (NIR) spectroscopy.95,210211.–212 The penetration depth limit mostly restricts them to 2-D imaging, but 3-D imaging is possible ex vivo with serial sectioning of biopsies to obtain adjacent tissue slices and reconstructing those images.40,95 They are being investigated for clinical use with IMA, where QMIB studies use histology for validation.69,213 In addition, recent studies have shown improved resolution in some applications. For example, a lens-free and electronic chip-based technology can achieve mesoscale field of view and microscale resolution in a short timeframe and can be substituted for histology using false-color algorithms.214,215 Wide-field imaging’s future QMIB applications, outside of validation against histology, are somewhat limited by their restriction to surface imaging, but they are likely to be seen in more ex vivo tissue studies as 3-D sectioning technologies advance.
3.3.
Biophotonics
Biophotonics, the study of optical and NIR light interactions with biological systems, is a field that has seen explosive growth over the last few decades. This growth is due to the desirable qualities of light at these wavelengths and technical advances in detection and illumination technology. These wavelengths of light are nonionizing. They allow a range of contrast options due to their absorption, scattering, and transmission properties. In addition, they do not require biochemical labels to generate contrast (though several can use contrast agents). Finally, it is relatively easy to generate monoenergetic optical and NIR light. These advantages have resulted in a collection of imaging modalities that operate at different spatial scales.168,216 These modalities are frequently seen in different multimodal and multiscale combinations. This section covers five biophotonic modalities: optical coherence tomography (OCT), photoacoustic tomography (PAT), diffuse optical tomography (DOT), and fluorescence and luminescence tomographies.
3.3.1.
Optical coherence tomography
OCT is a mesoscale in vivo imaging modality that is noninvasive, free of biochemical labels, images rapidly, and can be fit onto compact probes.76 It is analogous to US for optical waves, giving anatomical contrast through optical scattering caused by differences in tissue refractive index. It is used clinically for several surface and endoscopic imaging applications, but is at the preclinical stage for cancer imaging.217 The near-term clinical applications of OCT are assessing clinical margins for intraoperative surgery (IMA) and biopsy guidance, which are both benefitted by QMIB. Numerous studies, both qualitative and quantitative, have paired it with histology to validate its capability to differentiate pathologic breast tissue in vivo.76 Several studies have developed quantitative diagnosis algorithms for IMA and validated them against OCT, both alone and in combination with other modalities.70,98 OCT can be added onto biopsy needle probes and can be used to ensure biopsies are correctly sampling the tumor.99,218
OCT has several extensions used in breast imaging research, which could be applied to future QMIB studies. One extension measures attenuation, which can be calculated using automatic algorithms. These attenuation maps can help improve contrast of pathological tissue.219 Polarization-sensitive OCT can measure how much light polarization changes as it goes through tissue, a property known as birefringence. Birefringence is primarily influenced by microscale collagen, making it an indirect measure of the microscale structure of tissue.220 Mechanical OCT, or optical coherence elastography (OCE), can measure tissue strain. Strain is a relative quantity, so it is subject to high variance. However, there is high interest in developing OCE that can measure the elastic modulus, which is a quantitative reliable measure.221 Finally, there are other OCT extensions that have not been used in breast imaging research, such as blood flow imaging, but may be valuable for future QMIB research.
OCT is an exciting QMIB modality because it is well developed but still has room for growth. Its existing applications have broad applicability to breast cancer diagnosis and treatment in vivo (Fig. 5), but still need work to transition to clinical use.76 There are other exciting possibilities, for example, minimally invasive needle probes that could reduce the necessity of biopsies, to build on in the future.218 The various extensions give it other contrast options that are less well investigated, which makes it likely that new applications will arise from them. There are also many macroscale modalities it can be paired with for other investigations, of which some have been demonstrated outside the breast.223
Fig. 5
OCT has high potential for QMIB because it has prospects for patient imaging, where it can detect anatomical features useful in breast cancer diagnosis and treatment. This figure compares OCT (a and c) to corresponding histopathology (b and d) from a normal (a and b) and metastatic (c and d) human lymph node. The anatomical features of the lymph nodes can be seen by both modalities. The OCT images are generated as part of a 3-D volume (right). This work is licensed under a https://creativecommons.org/licenses/by/4.0/ Creative Commons Attribution 4.0 International License and is attributed to Nolan et al.222

3.3.2.
Photoacoustic tomography
PAT is one of the more promising frontiers of multiscale imaging. It combines rich optical contrast options with acoustic signal.27,146 In PAT, a laser is used to illuminate areas of tissue. The tissue is heated by the absorption of photons, and this causes it to expand rapidly, producing an acoustic signal that is detected by US transducers. This is known as the photoacoustic effect. The number of photons absorbed by the tissue, and thus the signal generated, varies based on the tissue composition and the wavelength of light from the laser. This effect is quantifiable and can target several molecules in tissue, such as hemoglobin or collagen. The signal can also be enhanced by contrast agents. The varied contrast options allow PAT to perform anatomical, functional, and molecular imaging for many biomedical applications, often at the same time. In addition, PAT has several other advantages that make it well suited to multiscale imaging. It is easily combined with US, as US transducers both detect and emit acoustic waves. Finally, PAT can be implemented for any of the three scales, with configurations capable of imaging microscale organelles ranging up to imaging the macroscale breast.19,27,146,224 There are commercial systems for clinical macroscale PAT and preclinical mesoscale PAT, making it more accessible to researchers.224
Several groups have used the macroscale and mesoscale implementations of PAT for QMIB. Some examples of QMIB with macroscale PAT include detecting micrometastases,88 finding and distinguishing between benign and malignant microcalcifications,225 mapping metastatic sentinel lymph nodes,39,88,226 and for tracking tumor angiogenesis.77 The mesoscale implementation of PAT has also been used in a few QMIB studies. Two studies examined an in vivo multimodal contrast agent for MRI, PET, and PAT using a reporter gene.100,227 A third study validated an in vivo cell-death contrast agent against the gold standard ex vivo fluorescence imaging.71 All of these studies demonstrate the potential utility of QMIB with PAT.
There are many future directions for PAT research. One of the largest standing problems in PAT is measuring the native optical fluence, which can introduce significant unquantified noise to some measurements.146 There are also many opportunities to perform new QMIB research with PAT. We have mentioned several mesoscale and microscale PAT studies, but they are still largely unexplored for QMIB. There are very QMIB few studies using its microscale implementation, photoacoustic microscopy (PAM).224 Significant advancements could also be made to expand the scale range of individual instruments, which might enable a singular PAT system capable of imaging over multiple scales.146
3.3.3.
Diffuse optical tomography
DOT is a form of whole-breast imaging based on NIR scattering and absorption. It operates at a lower resolution than most macroscale modalities seen in the clinic but has several advantages that make it well suited to translational and multiscale breast imaging research.74,216 The main molecules that interact with the NIR, known as fluorophores, are oxy- and deoxyhemoglobin, water, lipid, and collagen. The fluorophores have different absorption and scattering profiles over the NIR. DOT can map the spatial distribution of these fluorophores by imaging at multiple wavelengths, separating out the absorption and scattering contributions of each fluorophore. Hemoglobin imaging allows vasculature and oxygenation imaging, important subjects for studying angiogenesis or for diagnosis and treatment.216 Water, lipids, and collagen are used in many breast density quantification schemes.12 Collagen composition is also important for diagnostic and prognostic reasons.189 In addition, the optical scattering parameters are useful on their own, as they change in pathologic tissue.228 However, the accuracy and resolution of these measurements is limited by light propagation models. Light propagation can vary dramatically by tissue type, so models need large volumes to make accurate calculations. This issue can be overcome using multiscale imaging. Multiscale imaging gives prior knowledge of the tissue composition, allowing models with finer resolution and better accuracy. In summary, DOT can obtain several tissue composition parameters, and these measurements can be improved using multiscale imaging.
DOT is relevant to QMIB because it is a good example of coregistered imaging between significantly different resolutions, though thus far only within the macroscale. Some major modalities it has been paired with include US, x-ray tomosynthesis, MRI, and photoacoustic imaging.72,101,102,212,229230.–231 There are several clinically relevant findings from these studies. Two groups found that using it alongside US could improve diagnostic performance and decrease the amount of biopsies of benign tissue.72,231 Combining it with x-ray tomosynthesis allowed better differentiation between malignant tumors, benign lesions, cysts, and normal fibroglandular tissue.102 Similar results were also obtained with MRI.212 It was also combined with photoacoustic imaging methods, which are reviewed above, to track the biodistribution of a multimodal contrast agent attached to a cancer treatment drug.73,232 DOT is unlikely to be paired directly with mesoscale modalities in the near future, but several other biophotonic modalities share some of its principles for QMIB.
3.3.4.
Fluorescence and luminescence tomography
Fluorescence molecular tomography (FMT) and diffuse luminescent imaging tomography (DLIT) are small animal imaging modalities that operate in the low-mesoscale and high-macroscale range.180,233 FMT is also known as diffuse laminar optical tomography. They are mathematically and conceptually similar to DOT, although with several important distinctions. FMT can quantify the same fluorophores as DOT, but can also be used to image fluorescence from molecular probes.180 DLIT is based on luciferase, which emits light during enzyme reactions in live cells, and DLIT requires a priori knowledge of luciferase production in different cell types.233 However, DLIT is also significantly less noisy than fluorescence-based imaging and is sensitive to as few as a thousand tumor cells.234
These modalities have only recently become capable of sub resolution.180,234,235 Nonetheless, they have been used in several QMIB applications, which include imaging tumor apoptosis,105 tracking metastasis with nanoparticles,85,104 quantifying tumor growth and metastasis parameters,106 and validating a multimodality genetic contrast agent.80 FMT and DLIT still face issues in precise quantification, though new methods are being developed to handle these issues.180 Both have great potential use in QMIB as they mature and are used in new combinations. It should also be noted that there are nonbreast examples of their use for multiscale or multimodal imaging, such as a combination with OCT for phantom imaging.236
3.4.
Quantitative Multiscale Imaging Outside the Breast
The previous sections covered the existing QMIB modalities, but future researchers might take inspiration from biomedical QMI that has been demonstrated outside the breast. Reusch et al.159 combined SHG and US to do preclinical imaging of the nonpregnant uterine cervix. Future QMIB studies could benefit from similar methods, as collagen alignment is prognostic in breast cancer.202 Liang et al.223 built an MRI compatible OCT probe for intraoperative surgery. They showed that the probe can gather complementary information from tissue samples, and that such information could improve the efficiency and accuracy of surgeries. This surgical probe has obvious applications in breast cancer research, as intraoperative surgery is a common topic. Hipwell et al.160,203 developed an optomechanical device that synchronized SHG imaging with tissue deformation, mapping mechanical properties to microscale structure. This is relevant to breast research, as mechanical properties are a risk factor for breast cancer.31 Many research groups have conducted multiscale brain imaging with cranial windows.237238.–239 These methods are similar to the intravital windows used in QMIB research and could be applied to future studies. In summary, these selected examples have relevance to our current knowledge of breast cancer and could be potent tools for QMIB. Readers interested in more information of biomedical QMI in general might reference recent reviews of multiscale imaging.240241.242.243.–244
4.
Data Analysis Challenges
Data analysis and processing is the key component of quantitative imaging. It acts as a gatekeeper to the clinic, as a quantitative imaging modality needs effective and efficient analysis to facilitate clinical adoption.245 This section covers some of the unique challenges that QMIB faces. It is not meant to be exhaustive but rather to highlight important aspects of the developing field. Readers interested in more general information on data analysis methods might reference some excellent reviews for data analysis in breast cancer,7,46 general oncology,167,246 biomedical imaging informatics,14 translational imaging,247 and big healthcare data management.248
4.1.
Hardware and Software Limitations
Multiscale imaging produces large multidimensional datasets that in turn require exponentially more processing power to analyze than typical radiological images.248 Advances in computational power over the last decade have been staggering, but there remains a great deal of ground to be covered before some QMIB modalities become practical for a clinical setting.247 There are medically relevant engineering and programming solutions that can be investigated to speed this transition. Dedicated analysis hardware, such as chipsets that replace parts of algorithms, can be several times more efficient than general computing algorithms. Guerra et al. demonstrated the value of this process, as it allowed them to make the first handheld time-domain OCT probe.249 This approach is particularly relevant where there is little preexisting hardware and can be included in basic design developments. Optimization of software for efficiency is another major area of development. In many cases, software for QMIB is being developed alongside the systems, and there will be room for improvements in speed and architecture at all steps of the process.
4.2.
Registration and Biomechanical Modeling
Many multiscale imaging systems are multimodal and produce independent images, so there is a strong need for image registration to fuse these independent images. Image registration is the process of mapping spatial points on one image to those on another image, often resulting in a fusion image that displays information from both. The breast is a difficult organ to register, as it is made of soft tissue that deforms nonlinearly, thus altering landmarks. As such, breast image registration is an active field of research and only a few methods have been applied clinically.160
QMIB occupies an interesting area of the field, as it both requires unique registration solutions and can contribute to developing better registration algorithms. Multiscale registration needs to take into account the different size scales of images, which makes it difficult to accurately map the high-resolution image to the low-resolution image. In addition, multiscale registration frequently contends with different imaging geometries, which can alter landmarks and deform the tissue. However, other forms of QMIB offer solutions to registration problems. For example, QMIB can help address the breast deformation problem. Accurately modeling breast deformation requires good multiscale biomechanical models.160 QMIB can improve these models by accurately measuring anatomic changes and tissue mechanical properties across scales with one set of modalities. This information can then be incorporated into registration algorithms for another set of modalities. In summation, QMIB depends on registration but can also be used to improve existing registration algorithms.
4.3.
Segmentation
Segmentation is the task of delineating regions within an image based on some biological parameter, and it is essential for quantitative image analysis.247 Multiscale systems can complicate this problem by producing several images of distinct types with different resolutions and which do not necessarily have the same biological contrast. Multiple resolutions can cause misalignment due to partial volume effects, where the size of the macroscale pixel or voxel leads to errors in position of the low-resolution images. If the images have different biological contrast, they may not depict the same separations in biology that are used to segment the image initially. Accurate registration can mitigate this problem and turn it into an advantage, combining information on different traits to better segment the image. QMIB can also help advance segmentation. High-resolution multiscale data can improve the atlases and models that many segmentation methods use, or validate such methods with more accurate depictions of biology.250
4.4.
Imaging Biomarkers
Imaging biomarkers (IB) are defined imaging characteristics, which indicate biological processes or response to interventions.53,251 IB provide objective measures to test hypotheses and can become tools for clinical decision making.245 Strong biomarkers, which can be diagnostic or prognostic, are essential elements providing utility to new systems or techniques and justifying their translation to the clinic. The breast imaging-reporting and data system (BI-RADS) represents a set of qualitative IB based on categorization and physician interpretation.32 For example, physicians commonly classify breasts by the BI-RADS breast density categories using 2-D mammograms (Fig. 6). This visual assessment is clinically useful, but it can vary based on the physician’s training and can be inaccurate compared with 3-D measures.252253.–254 This leaves greater chance for error in patient care.252253.–254 By comparison, a quantitative imaging biomarker (QIB) is an objective characteristic derived from an in vivo image measured on a ratio or interval.8 CADe and CADx algorithms use QIB, and trials have demonstrated that they can improve radiologist performance.7 Biomarker development is a broad field, and interested readers may want to reference several excellent recent reviews covering basic definitions,8 metrology,53 and translation (Fig. 6).245
Fig. 6
MD as assessed by BI-RADS is a qualitative imaging biomarker, as it is visually assessed by physicians. These images show breasts classified by density under BI-RADS as (a) almost entirely fatty, (b) scattered areas of fibroglandular density, (c) heterogeneously dense, and (d) extremely dense.

There are two main ways that QMIB can lead to biomarker development: informed biomarkers and multiscale biomarkers. Informed biomarkers are developed using multiscale imaging, but do not use multiscale imaging when imaging the patient and assessing the biomarker. For example, OCT tumor margin detection is an informed biomarker. The biomarker was developed by correlation to the gold standard of histology, but in practice, only OCT is used.255 By contrast, multiscale biomarkers depend on multiscale information taken from the patient. Multiscale biomarkers are a much rarer method, as they require clinically friendly multiscale systems.52 One preclinical example of such a system involves dual-modality probes for fluorescence and radiographic imaging. It is difficult to characterize the probe’s biological interactions with only fluorescence imaging, as fluorescence imaging has a small field of view. This difficulty can be overcome by adding radiographic images to provide context, as they measure the probe over a much larger volume.256 Overall, both informed and multiscale biomarkers should become more common as the field of QMIB matures and are important to its ultimate clinical relevance.
4.5.
Computational Cancer Modeling
Computational modeling of cancer development and progression is a growing area relevant for both basic understanding and clinical application. For example, in one study the multiscale modeling of tumor growth indicated that some therapies used in breast cancer treatment could negatively impact long-term survival by selecting more dangerous phenotypes with environmental pressure.257 This was corroborated by another multiscale study, which found that environmental pressure encouraged predictable phenotypes, first with models and then experimentally verified with breast cancer models. Readers interested in more comprehensive knowledge of this type of modeling might refer to a recent review by Simmons et al.258 The extant examples of computational modeling only scratch the surface a field that is becoming increasingly accessible, and QMIB will be essential to validating such promising multiscale models in the future.
4.6.
Radiomics, Multiomics, and Precision Medicine
Radiomics is the process of building searchable medical imaging databases that can be mined for high-dimensional quantitative data.259 This collaborative effort yields data that can be analyzed and used in studies beyond the original, often in ways that were not previously possible. Building these databases involves turning images from a vast number of imaging modalities and their various applications into cross-institution and cross-modality quantitative information.259 QMIB generates cross-modality quantitative information and can help build these datasets. In addition, the datasets can help existing QMIB applications. For example, large cross-institution datasets would help address the sample size issues with breast density composition measurements (Sec. 3.2.1).
Radiomics is a subset of the Big Healthcare Data problem, where large amounts of information from various omic sources are being standardized, quantified, placed in computer archives, and processed to improve patient care.248 Integrating QMIB with multiomic research is another major path forward. For example, many studies are looking into radiogenomics, where genetic information is compared with imaging phenotypes.260 While few have been done with QMIB, the same principles could be applied. One of the main goals of these Big Healthcare Data initiatives is precision medicine, where previously unnoticed trends in these large datasets are used to develop methods for selecting and targeting treatments based on patient specific abnormalities.14 QMIB has great potential to contribute to precision medicine in breast cancer, contributing rich quantitative datasets on multiple biophysical characteristics. Doing so will rely on researchers to integrate radiomic concerns into their QMIB research and for all involved to build a collaborative data sharing spirit.
5.
Discussion
Quantitative multiscale imaging of breast cancer is an area well posed for growth in both the research and clinical regimes. The relative ease of imaging the breast makes it a good testing ground for multiscale imaging technology, and this pairing could address many breast cancer research and clinical needs. In the past, multiscale imaging was largely performed using independent imaging modalities and had high skill and time barriers to entry. The most common use was validation, comparing the gold standard of histopathology to images from new diagnostic modalities. In the present, all-in-one preclinical multiscale systems and simplified multiscale workflows are becoming more common.17,91,95,223,224,236 Developments in data acquisition methods are starting to simplify quantitative imaging with historically qualitative modalities, such as MRI or US. Improvements in hardware and software are making quantitative data analysis more accessible. Many of the modalities have been integrated into multimodal systems. Some clinical applications are approaching viability, for example, rapid tumor margin imaging that use macroscale modalities for needle guidance and mesoscale for detecting the margins.
Still, there remains a great deal of work to be done in terms of both basic research or validation and the development of new systems. Multiscale imaging of the breast involves a wide range of modalities in various stages of development. A handful of applications are in or are nearly ready for clinical trials. Research-wise there are unexplored quantitative metrics that could be investigated in a multiscale fashion immediately. Other modalities will require the development of new metrics and the derivation of corresponding biomarkers to make them meaningful. Multiscale methods have been demonstrated that suffer from strict constraints, such as slow imaging speed or high cost, which render them impractical. Attention also needs to be paid to areas in data analysis and handling including new algorithms to transform metrics and biomarkers into an end-user friendly format and hardware for larger storage capacities and quicker processing speeds. Finally, and perhaps most importantly, researchers should develop sets of accepted standards and conventions for facilitating interstudy comparisons and moving through the translational research process, where they do not already exist.
6.
Conclusion
Quantitative multiscale imaging of the breast is a rapidly evolving field that includes both the technology to enable investigations as well as the informational and clinical needs that drive them. It intersects with many breast imaging modalities and analysis approaches, with relevance to both clinicians and researchers. This review focuses largely on the technological challenges and benefits of QMIB. However, there are many additional issues that face any QMIB technology before clinical adoption. As with any new imaging technology, QMIB modalities being considered for clinical use are subject to rigorous FDA testing and evaluation. Other important clinical concerns for QMIB include ratio of cost effectiveness to outcome, availability, and ease of use. Finally, while this review is focused on breast cancer imaging, many of these same technologies, clinical needs, and research problems apply to multiscale imaging in other organ sites and for other pathologies.
Acknowledgments
We acknowledge funding from the Laboratory for Optical and Computational Instrumentation, the Morgridge Institute for Research and NSF CBET Award #1429045 (TJH and KWE). Research reported in this publication was also supported by the National Cancer Institute of the National Institutes of Health under Award #T32 CA009206 (MAP) and the National Institute of General Medical Sciences of the National Institutes of Health under Award #T32 GM008349 (MAP). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the National Science Foundation. We thank Eric Nordberg and Joseph Szulczewski for their assistance in the imaging done for Fig. 3. We also thank John Huston for his graphics design assistance.
References
Biography
Michael A. Pinkert is a graduate student of medical physics at the University of Wisconsin at Madison. He studied physics as an undergraduate at Rensellaer Polytechnic University. He is working on his dissertation at the Laboratory for Optical and Computation Instrumentation.
Lonie R. Salkowski is a professor of radiology at the University of Wisconsin at Madison and has received her doctorate in curriculum and instruction through the UW School of Education. She is a member of the Anatomy Task Force and Year 1 Curriculum Development Committees.
Patricia J. Keely was the Jan and Kathryn Ver Hagen Professor of translational research at the University of Wisconsin at Madison. Her research interests were in understanding how molecular level cellular interactions with the extracellular matrix are altered during carcinogenesis to result in invasive, metastatic carcinoma. She passed away on Saturday, June 24, 2017.
Timothy J. Hall is a professor and vice chair for faculty of medical physics at the University of Wisconsin at Madison. His research interests include acoustic scattering, tissue elasticity, breast imaging, cervical assessment, and ultrasound phantom development.
Walter F. Block is a professor of medical physics at the University of Wisconsin at Madison. His research interests include magnetic resonance (MR) interventional procedures, MR angiography and cardiac imaging, MR contrast mechanisms, signal and image processing, and distributed computing.
Kevin W. Eliceiri is the director of the Laboratory for Optical and Computational Instrumentation at the University of Wisconsin at Madison and a principal investigator in the Morgridge Institute for Research. His current research focuses on the development of novel optical and computational methods for investigating cell signaling and cancer progression, and the development of multiscale imaging methods.