For the validation of optical diagnostic technologies, experimental results need to be benchmarked against the gold standard. Currently, the gold standard for tissue characterization is assessment of hematoxylin and eosin (H&E)-stained sections by a pathologist. When processing tissue into H&E sections, the shape of the tissue deforms with respect to the initial shape when it was optically measured. We demonstrate the importance of accounting for these tissue deformations when correlating optical measurement with routinely acquired histopathology. We propose a method to register the tissue in the H&E sections to the optical measurements, which corrects for these tissue deformations. We compare the registered H&E sections to H&E sections that were registered with an algorithm that does not account for tissue deformations by evaluating both the shape and the composition of the tissue and using microcomputer tomography data as an independent measure. The proposed method, which did account for tissue deformations, was more accurate than the method that did not account for tissue deformations. These results emphasize the need for a registration method that accounts for tissue deformations, such as the method presented in this study, which can aid in validating optical techniques for clinical use.
Optical technologies can potentially be used for tissue characterization in, for example, the detection of positive resection margins during cancer surgery. These technologies are based on the principle that light that has undergone interaction with the tissue contains information regarding the composition, morphology, and microvasculature of the tissue which can be used to discriminate different tissue types. Validation of these optical measurements is done by comparing the experimental results with the gold standard.1 Currently, the gold standard for tissue characterization is an assessment of microscopy sections, which are stained with hematoxylin and eosin (H&E), by a pathologist. However, the tissue in the H&E sections, used for histopathological assessment, is profoundly deformed compared to the appearance of the tissue during the optical measurements.2–4 These deformations are introduced when the tissue is processed into H&E sections as explained and shown in Fig. 1. Part of these deformations is related to tissue shrinkage due to formalin fixation and tissue compression and stretching when the tissue is embedded into paraffin blocks. Consequently, overlaying digitalized H&E sections over images that reflect optical measurements of the fresh, unfixed tissue is prone to registration errors.5–8
In our research group, we use diffuse reflectance spectroscopy (DRS) for tissue characterization by means of a camera [hyperspectral imaging (HSI)] or fiber-optic probe (point measurements). With this optical technology, diffusely reflected light is measured after it has undergone multiple absorption and scattering events in the tissue. Therefore, a DRS measurement will contain information regarding the composition, morphology, and microvasculature of the tissue.
Registration errors between a DRS measurement and histopathology could negatively affect the assessment of the accuracy of this technology. The problem becomes even more relevant when developing classification algorithms, in which adding the correct histopathology label to each measurement is crucial. Accurate correlation between the optical measurements and histopathology is acknowledged as an important challenge encountered during clinical validation of optical technologies.1,9,10 However, there is only limited literature focusing on correlating optical measurements with histopathology,11–13 and when a method for correlation the optical data to histopathology is described, the reported registration methods do not seem to correct for tissue deformation.14–16
In this article, we demonstrate the importance of accounting for tissue deformations when correlating optical measurements with histopathology. For this, we use the histopathologic information that becomes available after the standard histopathologic protocol. No additional tissue was processed into H&E sections outside the standard sampling procedure to prevent impairing clinical diagnosis: embedded material might be needed in a later stage for clinical diagnostic purposes. In this study, we propose a registration method that considers tissue deformations parallel to the tissues’ surface and show that for an accurate correlation, tissue deformations should be taken into account when correlating optical measurements with histopathology.
Materials and Methods
Data were obtained from fresh ex vivo breast tissue slices: the optical measurements (both hyperspectral images and fiber-optic probe measurements) and a white light image were acquired simultaneously. In addition, microcomputed tomography () data were obtained right after the optical measurements. Therefore, we assumed that the tissue did not deform between the acquisition of the optical measurements, the white light image, and the data. As such, both the optical measurements and μCT data were registered to the white light images. A few days after the optical measurements, histopathological information (H&E section) was available. Our proposed method consists of correlating the H&E section to the white light image using a combination of affine and deformable registration. We compared the registration of our proposed registration method with the registration after using only an affine registration method, which does not account for tissue deformations. We quantified this accuracy using the data as an independent modality. Finally, we compared the registered H&E sections using both registration methods with our optical measurements.
Ex Vivo Tissue Specimens
Breast specimens were obtained from patients undergoing breast surgery, either breast-conserving or mastectomy, at the Antoni van Leeuwenhoek Hospital. The study was approved by the Institutional Review Board of the Netherlands Cancer Institute/Antoni van Leeuwenhoek, and according to Dutch law (WMO) no written informed consent was required. The specimens were brought to the pathology department immediately after surgery. There, after coloring of the resection margins, the specimens were frozen and sliced in a bread-loafed manner. One tissue slice was placed in a macrocassette. Black rubber was placed underneath the tissue to ensure that measured spectra only originated from the tissue.
White Light Images
Prior to the DRS measurements, an overview white light image of the specimen in the macrocassette was acquired. This image was used as a reference since the tissue remained in the macrocassette throughout the measurements.
Hyperspectral images were obtained with a push-broom HSI setup (VLNIR CL-350-N17E, Specim, Spectral Imaging Ltd., Finland) that captures light in the near-infrared ( to 1700 nm, 256 wavelength bands, 5 nm increments) with an InGaAs sensor (). The setup [shown in Fig. 2(a)] and measurement calibration are described in a prior publication.17 In short, the tissue is placed under the camera and imaged line-by-line through moving the imaged scene. Thereby a three-dimensional (3-D) hypercube is obtained that contains multiple two-dimensional (2-D) images of the tissue at different wavelengths, i.e., each pixel in the 2-D image contains a full diffuse reflectance spectrum [Fig. 2(b)]. Raw hyperspectral data obtained from the tissue were normalized to a diffuse reflectance percentage relative to Spectralon® (SRT-99-100, Labsphere, Inc., Northern Sutton, New Hampshire).
Fiber-optic point measurements were acquired with a measurement setup [shown in Fig. 2(b)] including a broadband light source, a spectrometer for the visual wavelength region (Andor Technology DU420A-BRDD), and a spectrometer for the near-infrared region (Andor Technology, DU492A-1.7). A fiber-optic probe was attached to the measurement setup to measure diffuse reflectance spectra between 400 and 1600 nm [Fig. 2(d)].18,19 The distance between the illuminating and collecting fiber at the tip of the probe was 1 mm. Before measuring, the setup was calibrated by acquiring a white reference measurement of Spectralon® (SRT-99-100, Labsphere, Inc., Northern Sutton, New Hampshire) in a calibration cap.
To acquire a DRS measurement, the probe was brought in contact with the tissue. During the probe measurements, a custom-made grid with holes [consisting of two pieces and shown in Fig. 2(c)] was placed on top of the tissue to gently fixate the tissue in the cassette and allow correlation between the measurement locations and H&E section afterward. The grid was 3-D printed and transparent. In addition, the grid stabilized the probe while ensuring contact between the probe and the tissue and restraining it from moving during measurements.
Microcomputed Tomography (μCT) Data
After the optical measurements, the grid was removed and the data were acquired. During the optical measurements and imaging, the tissue remained in the macrocassette and the black rubber was not removed. With the SkyScan® 1275B (Bruker, Kontich, Belgium), data were acquired using either 40-kV source voltage and source current, or 50-kV source voltage and source current and a 1-mm aluminum filter. The exposure time varied between 39 and 105 ms, and one scan consisted of 1801 (0.2 deg increments) to 901 (0.4 deg increments) projections. Reconstruction of the data was performed with 30 to resolution using SkyScan’s NRecon software.
After measuring, the tissue slice was brought back to the pathology department and processed in H&E sections according to standard protocol. The side of the specimen used for the optical measurements was marked with methylene blue to ensure that the H&E section was taken from the same side. After digitalizing the H&E sections with the Aperio® ScanScope AT2 (Leica Biosystems, Wetzlar, Germany), tumor tissue was labeled in the H&E images by a pathologist. The remaining tissue in the H&E sections, fat tissue, and connective tissue, was labeled by thresholding the RGB channels of the H&E images since fat tissue is transparent and therefore white in H&E sections and connective tissue colors pink.
Registration of H&E Sections
The registration of H&E sections to the optical measurements required multiple steps, which are shown in Fig. 3. Prior to the registration, we confirmed that the H&E section was obtained from the measured surface of the tissue, in other words, the correct side, using the data. All image registrations were made using MATLAB 2018a® (MathWorks, Natick, Massachusetts). To prevent possible bias, the H&E image without the tissue labels was used throughout the process. Subsequently, after finishing the registration, the same registration steps were applied to the H&E image with the tissue labels. First, the background of both the H&E image and the white light image was manually removed (Fig. 3, step 1). Second, the H&E image was rescaled using the shape of the tissue in both images such that the size of the tissue was matching the size of the tissue in the white light image (Fig. 3, step 2).
To compensate for rotation between the H&E image and the white light image, an affine registration was made between the two images. This was accomplished by using the shape of the tissue on both images and cubic interpolation (Fig. 3, step 3). The registration error, which was minimized using the least squares nonlinear solver in MATLAB, was defined as the sum of the difference between the shape of the tissue on the white light image and the registered H&E image.
Affine + deformable registration
For the affine + deformable registration, an additional step was applied after the affine registration (Fig. 3, step 4). First, control points were manually selected in both the H&E image (moving image) and the overview white light image (fixed image). Thereby, each control point in the H&E image had a corresponding point in the white light image. Two persons manually selected control points independently to make two registrations. In addition, the control points of both observers were combined to form the basis of one mean registration.
Figure 4 shows the control points selected by observer 1 in one representative example. First, control points were placed on distinctive features in both the H&E image and the white light image [Figs. 4(c) and 4(d)]. After assigning points to these features, additional points were placed in large areas of the images without any control points. These additional control points included four points in the corners of the images and a number of points surrounding the tissue. Without the additional control points, the algorithm is unable to accurately perform the local weighted mean transformation. For the deformable registration, a nonrigid local weighted mean transformation with 12 and 24 neighboring control points for the individual and mean registration, respectively, was applied using the built-in MATLAB function “fitgeotrans.m.”
Registration of optical measurements to white light images
The optical measurements were registered to the white light image to bring both histopathological and optical data into the same coordinate system. In the case of the hyperspectral images, a simple affine registration based on the shape of the tissue in both the hyperspectral image and white light image sufficed.
For the fiber-optic probe measurements, a more extensive registration was required. To retrieve the probe measurement locations in the overview white light image, we used (I) a schematic image of the grid, (II) a white light image of the tissue with the grid on top, and (III) the overview white light image (Fig. 5). First, the schematic image of the grid was registered to the white light image including the grid using a projective registration. This was done by aligning the measurement locations in the four corners of the grid in both images. Second, to account for small deformations to the tissue caused by the grid, the white light image including the grid was registered to the overview white light image using a deformable registration (similar to the registration method presented in Sec. 2.7.2). The calculated transformation matrices from both registrations were applied to the schematic image of the grid. With the final result (Fig. 5, IV), each measured probe location can be defined in relation to the overview white light image. All registrations were made using MATLAB 2018a® (MathWorks, Natick, Massachusetts).
To quantify the registration performance after affine registration and affine + deformable registration, two measures were used. First, the shape of the tissue in the H&E image with respect to the shape of the tissue in the white light image was analyzed using the Dice similarity coefficient (DICE).20 DICE is a measure for comparing the similarity of two images by measuring the overlap of the shape of the contour. Second, the tissue composition based on the H&E image was compared with the tissue composition in the data by using the point biserial correlation coefficient ().21 This is a special case of the Pearson’s correlation coefficient (Pearson’s ) suitable for calculating the correlation between a continuously measured variable () and a dichotomous variable (H&E section). An absolute of 1 indicates a perfect linear correlation, whereas an of 0 indicates no correlation. In addition, we compared the composition of the tissue in the H&E image with the composition estimated with the optical measurements using the Pearson’s . To compare the composition of the tissue in all modalities, we focused on discriminating fat from nonfat tissue since these tissue types can be well discriminated in all modalities.
Fat percentage in μCT data
Even though with data no actual fat percentage could be obtained, a lower gray value is related to a higher fat percentage.22 Since the 3-D μCT data will be correlated to 2-D data (the 2-D H&E image and the 2-D estimated fat percentage of the optical measurements), the 3-D data were reduced to a 2-D image. For this purpose, first, the height of the tissue in the 3-D volume was determined to obtain the superficial cross-section. To prevent imaging artifacts at the transition of nontissue to tissue, the superficial cross-section was defined at 0.15 mm underneath the determined tissue height as shown in Fig. 6(a). Subsequently, a second deeper cross-section was defined at 1 mm underneath the first cross-section. Third, over this 1 mm distance, an average was calculated per pixel. Thereby, a 2-D image was obtained that represents the average gray value over that 1-mm tissue thickness. An example of such a 2-D image is shown in Fig. 6(b).
Fat percentage in H&E image
With the labels of the H&E image, the tissue in the H&E image was binary divided into fat and nonfat tissues in MATLAB. The latter group contained tumor, connective tissue, and normal glandular ducts. The entire binary image was compared to the 2-D image of the data. In addition, for comparing the fat percentage in the H&E image with the estimated fat percentage from the optical measurements, the average fat percentage of the measurement locations of the probe was calculated. This percentage was the percentage of fat pixels within a circle () around the centroid of the measurement location.
Fat percentage in optical measurements
Optical measurements were spectrally fitted with an analytical model that was derived from diffusion theory using a Levenberg–Marquardt nonlinear inversion algorithm in MATLAB. For the fiber-optic probe, each measurement location was fitted and for the hyperspectral measurements, each pixel in the image that contained tissue was fitted. For the optical probe, this analytical model was described elsewhere.18,19 For the hyperspectral camera, the analytical model used is given by23
Optical measurements and data were acquired of 13 specimens. One specimen was excluded from further analysis because the slice was not thick enough to obtain a reliable 2-D image. Table 1 lists the characteristics of the tissue slices.
Characteristics of included specimens.
|Specimen||Size (mm) (height ∗ width)||Slice thickness (mm) (depth)|
|1||62 ∗ 38||3.5|
|2||43 ∗ 32||2.5|
|3||60 ∗ 43||3.0|
|4||64 ∗ 46||6.6|
|5||50 ∗ 33||2.7|
|6||35 ∗ 28||2.8|
|7||48 ∗ 25||3.4|
|8||45 ∗ 46||3.8|
|9||45 ∗ 17||3.2|
|10||46 ∗ 35||4.5|
|11||47 ∗ 43||5.5|
|12||60 ∗ 35||3.2|
Registration of H&E Section
Shape of the tissue: DICE
Figure 7 shows an example of an H&E image after affine registration and after affine + deformable registration (mean registration of both observers) and the effect of the transformations. The affine registration only scaled and rotated the H&E section and hardly any deformations are observed between the striped H&E section before [Fig. 7(c)] and after affine registration [Fig. 7(d)]. The DICE was 0.93 for this registration. After affine + deformable mean registration [Fig. 7(e)], however, a large transformation can be observed which increased the DICE from 0.93 [Fig. 7(g)] to 0.97 [Fig. 7(h)].
Table 2 shows the DICE after both registrations, given as the average of all samples and the standard deviation (std). For all samples, this DICE was high after both registrations. On average, the highest DICE was obtained using the affine + deformable registration.
Registration accuracy of the histopathology section using two registration methods; measured using Dice coefficient (DICE) and point biserial correlation (rpb). #cp stands for the mean±std of paired control points used.
|Shape: DICE||Composition: rpb|
|Affine + deformable registration|
|Observer 1 ()||0.94||0.02||0.46||0.18|
|Observer 2 ()||0.93||0.03||0.47||0.17|
Composition of the tissue: rpb
In addition to the Dice coefficient, we used the coefficient to compare the composition of the tissue in the H&E image with the composition of the tissue in the data. As stated in Sec. 2.8, an absolute of 1 indicates a perfect linear correlation, whereas an of 0 indicates no correlation. Inspecting the structures in the tissue that are easy to distinguish [orange arrows in Figs. 7(i)–7(k)] reveals that the composition of the tissue in the H&E section did not match the composition of the tissue in the data after only affine registration. This was corrected for after registering the H&E section with both the affine + deformable algorithm [Fig. 7(k)]. The coefficient increased from 0.24 to 0.53.
As can be seen in Table 2, averaged over all specimens, the increased substantially from the affine registration to the affine + deformable registration. Furthermore, when comparing the affine registration with the affine + deformable registration, on average over all specimens, 23% of the pixels had a different label (fat versus nonfat). Of these pixels, before the deformable registration step, 49% were labeled as nonfat and 51% were labeled as fat.
Practical consequence of the proposed registration method: required time and operator dependence
The required time for registration depended on (1) the size of the tissue and (2) the number of distinctive features in both images when selecting the control points. On average, the time required for removing the background in both the H&E image and the white light image was . Rescaling the H&E image to the white light images and applying the affine registration was fast and took less than 1 min. The most time-consuming part of the registration was the selection of control points in both images. This took around 15 min per observer. The registration of the optical measurement to the white light images required less than 1 min for the hyperspectral images and for the fiber-optic probe measurements.
In this study, two observers, both experienced in registering H&E sections to white light images, selected control point pairs to perform the deformable registration. To assess the operator dependence of the affine + deformable registration method, the registrations of the two observers separately were compared. As shown in Table 2, the number of control point pairs varied between the registration of both observers and the mean registration (in which the control points of the two observers were combined). However, the differences between the mean registration, the registration of observer 1, and the registration of observer 2 are minimal for both the DICE and .
Correlation with optical measurements
The fat percentage derived from the H&E sections was compared to the fat percentage derived from the optical measurements to assess the effect of accounting for tissue deformations in the correlation between histopathology and optical measurements. A clear distinction was made between “unaltered” and “altered” points. In the altered points, the averaged fat percentage derived from the H&E section varied with more than 20% before and after deformable registration. The unaltered points are the remaining points. In case of a perfect correlation between optical measurements and histopathological information, the fat percentages in both modalities would be equal and therefore the Pearson’s r will equal 1.
In the representative tissue slice shown Fig. 8, 7 out of 17 probe measurements are “altered” points (cyan circles). Of these seven locations, the fitted fat percentage of the optical measurements was plotted against the fat percentage according to the histopathology after affine registration [Fig. 8(d)] and after affine + deformable registration [Fig. 8(e)]. The fit results of both optical modalities have small confidence intervals and are similar. Figures 8(d) and 8(e) show that the linear correlation between the optical and the histopathological fat percentage improved substantially after adding the deformable registration for both the probe and the camera. The Pearson’s improved from 0.08 to 0.93 and from 0.01 to 0.93, for the probe and the camera, respectively.
Of all 234 locations that were measured with the fiber-optic probe, we compared the fat percentage of the optical measurements with the fat percentage according to the histopathology after affine registration and after affine + deformable registration (Table 3). We did this for the fitted fat percentages derived from both the probe and the hyperspectral measurements. Even though these fat percentages were not equal, they were similar and highly correlated with a Pearson’s of 0.87.
Comparison of the fat percentage according to the optical measurements and the histopathology after both registrations using the Pearson correlation coefficient (Pearson’s r). The altered points represent points in which the averaged fat percentage derived from the H&E section varied with more than 20% before and after deformable registration.
|Unaltered points||Affine registration||0.90||0.86|
|Affine + deformable registration||0.91||0.89|
|Altered points||Affine registration||0.36||0.36|
|Affine + deformable registration||0.66||0.61|
For the 186 “unaltered” points, the Pearson’s is not different before and after adding the deformable registration. For these points, using only affine registration, the Pearson’s was 0.90 and 0.86, for the probe and HSI data, respectively. Adding the deformable registration step produced a Pearson’s of 0.91 for the probe measurements and 0.89 for HSI data. Thus, as the deformable registration resulted only in a slightly higher Pearson’s , correcting for tissue deformations was considered not necessary for these measurement locations. For the 48 “altered” points, however, adding the deformable registration strongly enhanced the correlation between the optical measurements and histopathological information. The Pearson’s increased from 0.36 to 0.61 for the HSI measurements and from 0.36 to 0.66 for the probe measurements. These findings confirm the importance of accounting for tissue deformations when correlating optical measurements with histopathology.
Coregistration between optical measurements and histopathology is extremely important to validate an optical diagnostic technology. Due to deformation of the tissue during histopathology processing, an accurate correlation between optical measurements and the gold standard requires registration of H&E images to the optical measurements. Especially in tissue with an inhomogeneous character, such as breast tissue, this correlation is crucial for validating the optical measurements with the current gold standard. In this paper, we highlight the importance of accounting for tissue deformations when correlating optical measurement with histopathology. As such, we present a methodology for making a registration between optical measurements and the gold standard using an affine + deformable registration method. In this process, the H&E images were registered to a white light image that was taken simultaneously to the optical measurements. This registration approach was compared to a simple registration method using only an affine registration that does not account of tissue deformations. The performance of both registration methods was quantified using a different independent imaging modality, . The registrations were assessed by first, comparing the shape of the tissue by calculating the DICE for each registration, and second, by comparing the composition of the tissue by calculating the . Although both registration methods have high Dice coefficients (), the affine + deformable registration provides the highest registration accuracy when evaluating both the shape (DICE) and the composition of the tissue ().
H&E sections after affine registration and after affine + deformable registration were compared to data to see which registration method results in the smallest registration errors between histopathology and optical measurements. We showed that accounting for tissue deformations is important for reducing the registration error between optical measurements and histopathology. It should be noted, however, that in all cases the H&E section remains a 2-D image of one cell layer thickness whereas the optical measurements represent the optical properties of a volume of several . Therefore, differences in depth will not be expressed in the H&E section, although such differences will affect the optical measurements. This phenomenon can be seen in Fig. 6 and has also been reported by Ma et al., who showed that the tissue architecture may change between adjacent tissue sections.24 In addition, the thickness of the tissue within a tissue slice varies, as can be seen in Fig. 6. In practice, the pathology technician will keep slicing sections from the paraffin-embedded tissue until a section is obtained that encloses the complete shape of the tissue slice. Therefore, the H&E section does not necessarily represent exactly the surface of the tissue as it was measured with the optical techniques. To account for this, we compared the registered H&E sections with data over a tissue thickness of 1 mm underneath the tissue’s surface. The authors realized that the fact that the H&E sections are only a cell layer thick and the height of the tissue differs over the surface are limitations in obtaining a fully accurate correlation between the H&E sections and the optical measurements. These circumstances also explain why a registration accuracy of 100% can never be obtained. However, as we use human material that is processed for diagnostic purposes, we were only able to obtain the histopathologic information that becomes available after the standard histopathologic protocol. Therefore, methods such as 3-D histology as described by Kartasalo et al. and Pichat et al. could not be implemented.25,26
A limitation of the affine + deformable registration method as currently described is the manual input that was required, which makes it time-consuming. In addition, the method can be operator-dependent as the observers have to manually place points in both the H&E image and the overview white light image based on distinctive features that they recognize in both images.
In this study, registrations were made by two observers, with the same experience in registering H&E images to white light images. We showed that, even though the number of selected control point pairs varied between both observers, the registration accuracy was similar (Table 2). Therefore, registering the white light image with the H&E section using only the input of one of the two observers would also have resulted in an accurate registration. However, care should be taken when observers are less experienced. Also, we did not investigate how the number of neighboring control points that were used in the local weighted mean transformation impacted the registration. Optimization might be possible by using more or less neighboring points.
To reduce processing time, Naranjo et al. presented several registration methods to register infrared images with H&E-stained sections, with the ultimate goal to automatically segment regions in the infrared data.11 However, in their study, H&E-stained sections needed to be registered to sections of the tissue samples that are only a few cell layers thick, whereas in our study the tissue samples are thick. Furthermore, the total area of tissue that was registered to an H&E section in Naranjo’s research was much smaller than the surface that was registered in this study. Due to these differences, their automatic key point selection method, which was based on finding the centroids of cells, was not applicable to our data. In their study, the projective transformation is reported to provide the best registration. This generalization of an affine transformation is a much less complex registration method compared to the affine + deformable method as proposed in this research. The projective transformation was sufficient in their study probably because the tissue slices were much smaller and therefore less deformation was present.
As shown in Table 2 and Fig. 7, the increased substantially from affine registration to affine + deformable registration. Also, when comparing the labels of the H&E after affine registration and the H&E section after affine + deformable registration, on average 23% of the pixels obtained a different label. This result is confirmed by comparing the Pearson’s of only affine registration or affine + deformable registration of the altered points (Fig. 8 and Table 3). However, for the altered points, the Pearson’s remained lower compared to the unaltered points. This might be explained by the fact that the altered points are more likely to be located at a tissue transition. Therefore, there is an increased chance that the estimated tissue composition is different from the composition optically measured. Nevertheless, using a different registration method leads to different labeling of a substantial part of the optical measurements. This potentially has a strong positive effect on the maximum classification performance that can be obtained with the optical techniques.
Although, the fit results of the two optical modalities used are not exactly the same, the linear correlation between the fat percentage derived from the optical measurements and the histopathological fat percentage improved for both optical modalities after adding the deformable registration method to the affine registration. Differences between the fit results of the optical techniques can be explained by a difference in the measurement configuration: the distance between the illuminating and collecting fiber determines the measured volume with the fiber-optic probe, whereas for the hyperspectral camera the volume is mainly depended upon the tissue properties. Therefore, both configurations might measure a different volume, resulting in a difference between the obtained fat percentages. Nevertheless, the linear correlation between the fat percentages derived from the probe and the camera measurements is high with a Pearson’s of 0.87.
In this study, we used as an independent 3-D modality to quantify both registration methods. Abe et al.27 have reported on using MRI for imaging of pathology slices and showed good image quality. In our study, we did attempt to quantify the registration methods with MRI using the mDIXON-Quant sequence. This sequence was chosen as it can make excellent discrimination between fat and nonfat tissues.28–30 However, the resolution, although acquired with a relatively high-resolution voxel size of , proved to be too low for quantifying the registration methods. In the study of Abe et al., an animal MRI with a higher magnetic field (9.4T) was available than the clinical human wide-bore 3T MRI available in this study. For this reason, in this study, MRI data could not be used for quantifying the coregistrations and was thus not included in the analysis.
A disadvantage of using only data was that we could only discriminate fat from nonfat tissue. In which the latter included both connective tissue and tumor tissue. One of the reasons was the low contrast between tumor and healthy fibroglandular breast tissue in the scans. This has been reported by previous research as well.31 Other studies indicate that this differentiation should be possible.22 However, in that specific study, biopsy specimens were imaged with a much higher resolution than the resolution that could be obtained in this study. Nevertheless, the data in this study functioned as an independent measure to discern areas of fat and nonfat. The use of was not intended to replace the current gold standard but allowed us to quantify the registration accuracy of H&E images.
It should be noted that this research was conducted mainly to improve the validation of optical measurements based on the current gold standard. As previously explained, the H&E section remains a 2-D image of one cell layer thickness and will, therefore, no matter which registration method is used, never completely represent the volume measured with the optical modalities. In addition, histopathology can also suffer from other potential flaws arising from factors such as intraobserver variation,32–34 interobserver variation,35,36 under sampling,37,38 and specimen handling and reporting.39–42 Nevertheless, there is no way to validate the technology without considering the current gold standard. Therefore, once established as a robust diagnostic technology, the true potential of optical measurements in clinical practice should be assessed with evaluation of clinical end-points like, for breast cancer surgery, a decrease in positive resection margins or decrease in excised specimen volumes.
In conclusion, for an accurate correlation between histopathological information and optical measurements, tissue deformations should be taken into account in the registration. The proposed registration method in this study, which does account for these tissue deformations, shows a better correlation with the optical measurements. Although it is labor intensive, adding the deformable registration step can aid in validating the optical measurements with the current gold standard.
The authors gratefully acknowledge the financial support of this research by the Dutch Cancer Society (Grant No. KWF 10747). We also thank Bruker MicroCT for providing the microCT scanner. The authors thank N.N.Y. Janssen for help with acquiring data, O. Ivashchenko for help in developing the registration algorithm and the NKI-AVL core Facility Molecular Pathology & Biobanking (CFMPB) for supplying NKI-AVL biobank material. We also thank all students that participated in this research for their time and effort, and all surgeons and nurses from the Department of Surgery and all pathologists and pathologist assistants from the Department of Pathology for their assistance in collecting specimens.
Lisanne L. de Boer achieved her MSc degree in technical medicine in 2013 from the University of Twente. Recently, she finished her PhD at the Netherlands Cancer Institute—Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands. This project was focused on the use of fiberoptic DRS for detecting positive resection margins during surgery to improve the surgical treatment of breast cancer.
Esther Kho is a PhD candidate at the Netherlands Cancer Institute, The Netherlands. She received her MS degree in biomedical engineering from the University of Groningen in 2014. Her research involves the development, testing, and clinical evaluation of a hyperspectral imaging system, specifically for the assessment of tumor-positive resection margins during breast-conserving surgery.
Jasper Nijkamp is a senior postdoc image-guided oncology at the Netherlands Cancer Institute. He obtained his PhD in 2012 studying the uncertainties in radiotherapy for rectal cancer. Subsequently, he moved to image-guided surgery, where he designed and developed a surgical navigation system for mobile tumors. The navigation system is currently evaluated in pelvic lymph node dissections, colorectal, hepatic, and head and neck surgery. Jasper also works on surgical margin evaluation in breast cancer using microCT.
Koen K. Van de Vijver is a surgical pathologist with interest in diagnosis and treatment of cancer of the female genital tract and the breast. He received his PhD in 2007 at Antwerp University and Leiden UMC. In 2008, he became an assistant professor at Maastricht University. After working as a consultant pathologist in the NKI-AvL, he became a professor of gynaecological pathology at Ghent University Hospital in 2018.
Henricus J. C. M. Sterenborg has been active in biomedical optics research since 1987 as a staff member of the Medical Laser Centre of the Amsterdam UMC. In 1998, he (co) founded the Centre for Optical Diagnostics and Therapy at the Erasmus Medical Center and in 2008 he became a professor of photodynamic therapy. Since 2013, he has held a joint position at the Department of Biomedical Engineering and Physics at the Amsterdam UMC and the Department of Surgery at the NKI-AvL.
Leon C. ter Beek is an MR physicist at the Netherlands Cancer Institute since 2014. Before he worked as a research scientist at GE Healthcare in Japan and Philips Healthcare in the Benelux, collaborating with Academic Hospitals in the field of MR imaging and spectroscopy. He holds a PhD in chemistry on NMR spectroscopy and liquid crystals from the University of British Columbia in Vancouver, Canada.
Theo J. M. Ruers is an oncological surgeon and head of the Department of Surgery at the Netherlands Cancer Institute. In 2010, he became a professor at the Faculty of Science and Technology at the University of Twente. He has extensive experience in oncologic surgery and the implementation of techniques in cancer care. Special focus is on using optical (imaging) techniques for diagnosis and treatment of cancer.