2.1.

Data Acquisition

2.2.

Co-Registration of FMT and XCT

Co-registration was achieved by means of an imaging cartridge (FMT 2500, VisEn Medical, Woburn, Massachusetts) shown on Fig. 1. The transparent plates are made of glass-reinforced extruded nylon and covered with an anti-reflection coating for the NIR. The anaesthetized animal was inserted between the two transparent plates which apply mild compression to the mouse body. In this manner, the animal stayed immobile in the same position, while sequentially imaged by FMT and XCT. Figure 1(a) shows the transversal schematic view of such a compressed object and the general imaging setup geometry.

Fig. 1

FMT system and mouse bed geometry and orientation: (a) schematic of a limited-projection-angle FMT system (not in right scale): a laser is scanning along the $x$- and $y$-axis and a camera is acquiring the trans-illumination images for each scan. (b) Imaging cartridge containing the phantom (left) and a mouse (right). Small holes in the nontransparent margin of the cartridge serve as fiduciary markers (arrows). (c) The coordinate systems of XCT and FMT, respectively, are mapped to each other through rotation and translation.

The cartridge was always inserted in the same horizontal position in the FMT system, by means of rigid rails, while it also contained fiduciary markers [Fig. 1(b), arrows] consisting of small cavities in the frame which were visible in the x-ray CT images. The fiduciary markers are manufactured to mark a plane that is parallel to the detection plane of the CCD camera and were employed to track the position of the cartridge when inserted into the XCT bore so that FMT and XCT data could be accurately co-registered.

For registration purposes, information from the fiduciary markers was combined with boundary information of the animal, as seen on XCT images and the animal photographs obtained by the FMT system, and guided the translation and rotation of the reconstructed 3-D XCT volume to fit in a common coordinate system with the FMT data. A schematic of this concept can be seen in Fig. 1(c). The white light image, taken in the FMT device, defined the coordinate system to which the XCT volume had to be co-registered. Since, however, the XCT information is 3-D and the photograph is two-dimensional (2-D), it was not possible to directly extract information on possible rotation of the cartridge in the XCT system, but the plane defined by the fiduciary markers was considered as the horizontal plane corresponding to the photograph. The fiduciary markers were then employed to measure, and correct, for the rotation angle between the $x$-, $y$-, and $z$-axis in XCT and FMT. After rotation, $x\text{-}y$ translation was applied to align the XCT images and FMT data to a common coordinate system. More specifically, the orientation of the cassette (transversal, sagittal plane) was extracted from the reconstructed XCT images based on the cassette plates and fiduciary markers. According to Fig. 1, the angles denoted by $\alpha$, $\beta$, $\gamma$ between the XCT planes ($x_{\mathrm{xct}}$, $y_{\mathrm{xct}}$, $z_{\mathrm{xct}}$) and the ($x_{\mathrm{fmt}}$, $y_{\mathrm{fmt}}$, $z_{\mathrm{fmt}}$) plane of the photograph from the FMT device were computed by projecting the FMT photograph to the $x_{\mathrm{xct}}\text{-}{y}_{\mathrm{xct}}$ ($z_{\mathrm{xct}}=0$) plane and computing the angles between this plane and the ones defined by the fiduciary markers along the transversal and sagittal slices. Then, the 3-D XCT image was rotated so that it could be aligned with the ($x_{\mathrm{fmt}}$, $y_{\mathrm{fmt}}$, $z_{\mathrm{fmt}}$) plane. Having consequently produced two parallel data sets, a translation step was then performed to map the boundary and fiduciary information from the XCT to the mouse photograph. To achieve this, the animal boundary from the XCT was produced by projecting the XCT slices along the $z$-axis on one single plane that corresponded to the FMT photographic plane (i.e., a maximum intensity projection and subsequent boundary extraction by thresholding). This step is visualized in Fig. 2 (see also the results section). Therefore, in contrast to previous FMT-XCT co-registration approaches, mapping 3-D data sets to each other using fiduciary markers placed on a cartridge, the approach herein performed 2-D to 3-D mapping and employed fiduciary and boundary markers for improving the accuracy of the co-registration.⁴⁷

Fig. 2

Co-registration approach: (a) The final accuracy of this approach was confirmed through imaging of a wire phantom. The XCT image (1st panel from left) and optical image (2nd panel from left) were co-registered and over-layed (3rd and 4th panel). (b) Fiduciary markers and boundary mapping were used for the fusion of the FMT and XCT data sets (left). The source and detector information from the FMT device were projected on the boundary computed from the XCT (middle and right).

The width dimension ($z$) of the animal was obtained from the XCT and confirmed by the internal measurement system in the FMT device which measures the cassette width using two ultrasound transducers. The accuracy of the co-registration method was examined using a 230 μm copper wire placed in random directions in the cartridge and imaged by XCT and FMT (only reflectance image). All calculations were performed with a custom made algorithm implemented in Matlab (Mathworks Inc., Massachusetts).

2.3.

Image Segmentation

Segmentation of the XCT data was performed to single out different anatomical structures. For imaging the upper mouse thorax, 4 organs were typically segmented such as the lung, bone, heart, and remaining tissue. The segmentation was performed automatically using thresholding, seed growing, and signal detection algorithms as previously reported by Freyer et al.⁴⁸

Summarized, because of the high contrast, bone segmentation could be performed using an automatically derived threshold. From this first segmentation, the ribcage was obtained and used for localization of the lung which was finally segmented using a seed growing algorithm. The seed points were automatically determined from an intensity histogram of voxels lying inside the region of interest determined by the ribcage. For the segmentation of the heart, a shape model was used that was iteratively adjusted based on the ribcage and lung segmentations. In contrast to the method described by Freyer et al.,⁴⁸ the initial position of the heart was not determined automatically but through user input, since, in our case, the typical thorax shape was deformed by squeezing in the imaging cartridge.

For the phantom study, only the tubes and the background medium were segmented from the complete volume since no other distinguishable structures were present. For segmentation purposes, the tube walls were employed as boundary indicators since they are visible with high contrast on the XCT images due to their different x-ray absorption characteristics over the phantom background medium. We manually selected the tube boundaries in two transversal slices of the XCT reconstruction, each at one end of the phantom, and used this input for automatic extraction of the rest of the tubes throughout the volume. All voxels lying inside the extracted tube walls were consequently assigned to the tube segments.

2.4.

Phantom: Ex Vivo Imaging

Cylindrical, tissue mimicking phantoms with inclusions containing a NIR fluorescence dye with peak excitation at 679 nm and peak emission at 702 nm (Alexa Fluor 680, Life Technologies Ltd., Paisley, United Kingdom) were imaged in both modalities. The phantoms were composed of a mixture of an adequate ratio of intralipid, ink, agar, and water in order to attain tissue properties of ${\mu }_s^{\prime }=12{\mathrm{cm}}^{-1}$ and ${\mu }_a=0.2{\mathrm{cm}}^{-1}$ and were shaped to semi-cylinders resembling the shape of a mouse torso as illustrated in Fig. 1(a). The phantom measured approximately 36 mm in length and had a diameter of 30 mm at the widest point, the two inclusions had a diameter of approximately 3 mm. The tubes were positioned in the upper curved part of the semi-cylinder, as depicted in Fig. 1(a) by the red dotted circles and illustrated in more detail in Fig. 3 in the results section, and filled with the NIR fluorescent dye diluted in liquid with the same optical properties as the phantom, resulting in a fluorochrome concentration of 100 nMol.

Fig. 3

Phantom experiment showing reconstructions from the stand-alone FMT device (a) and hybrid approach (b and c). (a) Left: Three-dimensional (3-D) representation of the phantom with cartridge and adapter, showing the reconstruction provided by stand-alone FMT. In the lower image the fiduciary markers can be seen as red dots. Right: Slices through the reconstructed volume. (b) Left: 3-D view of the reconstructed fluorescence distribution using Tikhonov regularization. Here, only the outer boundary of the XCT was used in the reconstructions and no prior information was yet applied. The top image shows the position of the five slices that are shown in a, b and c. Right: Slices through the reconstructed volume. (c) Left: 3-D view of the reconstructed fluorescence distribution using differently weighted segments for regularization. Right: Slices through the reconstructed volume. The 3-D images were created using AMIRA software.

2.5.

In Vivo Imaging

Kras mice spontaneously developing tumors in the lung were used.⁴⁹ Approximately 24 h prior to imaging, a targeted fluorescence agent comprising an integrin ${\alpha }_v{\beta }_3$ antagonist and a NIR fluorochrome with peak excitation at 675 nm and peak emission at 693 nm (IntegriSense 680, PerkinElmer, Waltham, Massachusetts) was administered via tail vain injection. At the beginning of the imaging session, the mice were anesthetized (Isoflurane 2%, O2 $0.9\mathrm{l}/\min$) and placed in the previously described multimodal imaging cartridge that held the mouse motionless during the entire acquisition period in both FMT and XCT in a fixed position. First, x-ray CT was performed during approximately 10 to 15 min. Immediately following the XCT imaging session, FMT imaging was performed with a typical duration of 7 min. All procedures were performed in accordance with Helmholtz Center and Government of Bavaria law and regulations.

2.6.

Ex Vivo Validation

Following in vivo imaging, the mice were euthanized by i.v. injection via catheter, of a lethal dose of a NaCL-Ketamine-Xylacine mixture, and frozen while still in the imaging cassette in order to keep their slightly squeezed shape for validation.

The mice were sliced in a cryotome (CM 1950, Leica Microsystems GmbH, Wetzlar, Germany) and each slice was imaged using an epi-fluorescence system at the according excitation/emission wavelength of $680/700\mathrm{nm}$.⁵⁰ Images of ex vivo slices showing the true fluorescence distribution were compared to the corresponding reconstructed slices from the in vivo measurements as to their accordance in signal localization and relative signal intensity. This was done by defining, in the cryosections, a region around each tumor where a reconstructed signal would be admissible. Since the FMT reconstruction signal is more diffuse than the real signal, this region was chosen up to 2 mm around the actual tumor. The ratios of maximum fluorescence intensity in the tumor over maximum intensity in the muscle and in the lung were calculated for the cryosections. The same ratios were calculated for the in vivo data in the corresponding regions as determined from the cryosections. Generally, if the FMT reconstruction corresponds well to the ex vivo slices, the tumor/lung or tumor/muscle ratios should, ideally, linearly increase or decrease with increasing or decreasing cryoslice ratios.

The localization accuracy of the reconstructed fluorescence in the phantom was determined by defining concentric circles with increasing radius around the tubes and calculating the percentage of the total reconstructed signal per slice that was located inside the circles depending on the distance from the tube.

For validation of the fluorescence distribution in the mouse lung, we reconstructed a 3-D representation of the lung based on the lung cryoslices, using slices obtained every 250 μm. This volume was interpolated to a voxel size of 50 μm and rendered. By applying an intensity-based threshold at 50 percent of the maximum value observed (after shot-noise image filtering), a measure of fluorescence above background could be obtained for comparison with the FMT reconstruction.

2.7.

Image Reconstruction

Stand-alone FMT reconstruction was based on a normalized Born approach previously published.⁵¹ Incorporating XCT prior information into the FMT inversion code was based on a previously described algorithm which is briefly described here.⁴⁰

Modeling of photon propagation in tissues was based on the diffusion equation, that is:

For the mouse study, different optical properties were assigned in the forward model to each of the segmented organs to take into account their varying level of scattering and absorption. They were experimentally determined for this mouse model and are listed in Ref. 52. For the phantom study, we used the same optical properties for all segmented regions, as described above. The difference in XCT and FMT resolution was taken into account by proportionally assigning each FMT voxel to its higher resolved anatomical segments as previously described in Refs. 33, 40, and 41.

Three ways of integrating anatomical data into the reconstruction of optical signals were considered in this study: the first did not use any information from the XCT but was based on boundary detection through the FMT camera, and further assumed slab geometry, due to the imposed shape of the imaging cartridge. Reconstructions based on this method were automatically provided from the used FMT imaging system. The second used the exact boundary detected by the XCT scan but no further anatomical information. The third fully integrated all available information from the XCT data, such as boundary and segmented organs. Consequently, depending on the different level of available anatomical information, different inversion schemes had to be implemented.

Inversion of Eq. (2), to solve for the unknown fluorescence image $x$, is an ill-posed problem which was solved herein by minimizing the residual of function $F(x)$:

For inversion, we assigned the smallest weight ($w_1=1$) to the lung region (for the in vivo measurements) and to the fluorescence tubes (in the phantom measurements). Reconstructed intensities appear smoother (more regularized) as the weight increases. For the mouse measurements, all other tissue regions segmented such as bones, heart, and remaining tissue assumed the same weight ($w_2=4$).

This manual selection of weights is only appropriate when using phantoms or specific mouse models where the appearance of disease or fluorescence distribution is known before-hand.^33,40 This approach was also used, herein, for demonstration purposes with limited-projection-angle schemes using known phantoms and animal models. However, data-driven weight allocation is instead recommended to the more general case where the appearance of disease or the fluorochrome bio-distribution is not known before-hand.⁵²

To determine the $\lambda$ values, the minimization of Eq. (3) was solved for 200 different values of the regularization factor $\lambda$. Those values were chosen to be logarithmically spaced between ${10}^{-6}*\Vert W\Vert$ and ${10}^1*\Vert W\Vert$. An optimal $\lambda$ value was then selected by $L$-curve analysis by plotting the solution norm versus the residual norm and then choosing the lambda value at the first corner of the resulting $L$-curve.⁵³ Following the $L$-curve analysis in which all optimal $\lambda$ values of different data sets were found to be in a similar range, a common $\lambda$ value was selected for all the reconstructions shown herein.

3.1.

Co-Registration Approach

Figure 2 shows the results obtained from the validation study examining the co-registration of FMT and XCT data.

Figure 2(a) shows an XCT image (1st panel from left) and a photograph (2nd panel from left) of the wire phantom as well as the overlay of both data sets after rotation, scaling and translation. The wire shape, extracted from XCT data and projected in one plane, could be perfectly aligned with the FMT reflectance image without any apparent nonoverlapping regions. The co-registration accuracy was measured as the distance between the wires as seen by the optical and the XCT image on the overlay of Fig. 2(a), 4th panel. This difference was found to be one pixel (worst case), i.e., max. 154 μm, i.e., much smaller than the resolution achieved by FMT.

Figure 2(b) shows related results from animal imaging. The left panel depicts a black and white image of a mouse acquired with the FMT camera and the overlaid boundary extracted from the XCT image, shown in blue. The laser source positions and the location of the fiduciary markers are projected onto the overlay in red. This projection is enabled by the integration of both modalities into one geometrical framework. The acquired fluorescence images could then, accordingly, be projected onto the same geometrical scheme.

3.2.

Phantom Experiment

Figure 3 depicts results from the phantom measurements employed to further examine the accuracy of the registration approach and for comparing stand-alone reconstructions [Fig. 3(a)] to reconstructions using the XCT boundary [Fig. 3(b)] and XCT tube segmentation [Fig. 3(c)]. Although the phantom had homogeneous optical properties and simplified internal boundaries compared to the complexity of tissue, the use of priors demonstrated regardless reconstruction improvement over reconstructions obtained in the absence of priors.

Figure 3(a) shows the reconstruction of the fluorescence signal assuming the phantom being an infinite homogeneous slab and without the use of priors (stand-alone reconstruction). The left column renders in gray 3-D x-ray CT images of the cassette and phantom inside the XCT scanner. Reconstructed fluorescence signals are co-registered on the images in dark red color. In the right column, transverse (axial) slices at different locations along the $y$-axis (as referred to in Fig. 1) are showing the localization of the reconstructed fluorescence signal inside the phantom. The location of the tubes can be seen on the XCT images and are highlighted in the first slice with red circles. Both in the upper image of the 3-D representation and in all transversal slices of Fig. 3(a), it can be observed that fluorescence distribution seen on stand-alone FMT reconstructions is partially located outside the real volume (highlighted by red arrows) and only partially coincides with the fluorescent inclusions.

Figure 3(b) shows the corresponding reconstructions without priors, but using the actual phantom shape as boundary into the inverse code. The upper image in the left column shows the location of the five slices inside the phantom. The use of the boundary overall improves imaging performance, over stand-alone reconstructions in particular, along the horizontal axis [Fig. 2(b), right column]. On the vertical axis, both reconstructions reconstruct an elongated shape rather than a circle. The stand-alone reconstruction shows a displacement of the center of the ellipse toward the phantom boundary. Conversely, the use of the actual boundary better estimated the center of the fluorescence activity.

Figure 3(c), finally, depicts the reconstruction using the actual phantom boundary and anatomical priors. Here, additionally to the improved horizontal alignment, we can observe improved vertical alignment and confinement of the main fluorescence signal to the tubes.

3.3.

In Vivo Imaging

Figure 4 depicts results from the in vivo studies. It compares stand-alone reconstruction using slab geometry and weighted segments reconstruction using priors in a lung tumor mouse model with ex vivo fluorescence images of the corresponding in vivo slices. The extraction of boundary information, and its incorporation into such optical imaging problems, was previously shown using other means like imaging with photogrammetric 3-D cameras or volume carving based on silhouette images.^54,55 The phantom study, additionally, showed best results when using priors. In this in vivo study, we, therefore, concentrated on the methodology and improvements specific to the hybrid FMT-XCT approach, namely through incorporation of priors due to organ segmentation.

Fig. 4

Comparison of FMT reconstructions computed using stand-alone FMT without boundary or prior information (a) and hybrid FMT-XCT and the segmentation information derived from it (b) with corresponding ex vivo slices (c). All slices in one row are the same. The positions of the slices in each column are depicted in (g). 3-D representations of both stand-alone reconstruction (d) and hybrid reconstruction using priors (e) visualize the distribution and the fluorescent hot spots. The 3-D representation acquired from the ex vivo slices (f) shows signals in the same spots as the hybrid reconstruction (e). All units are arbitrary. The 3-D images were created using AMIRA software.

The first column [Fig. 4(a)] shows stand-alone FMT reconstruction as it is output by the FMT 2500. In the second column [Fig. 4(b)], the reconstruction using weighted segments method is presented. Finally, the third column [Fig. 4(c)] contains ex vivo slices indicating the actual fluorescence intensity and distribution. The corresponding position in the mouse body can be seen in Fig. 4(g). All slices in one row but belonging to different columns correspond to each other.

The ex vivo validation images [Fig. 4(c)] show several areas with increased fluorescence intensity compared to surrounding lung tissue. Some examples are highlighted by red arrows. While, in some slices, only single outstanding points can be observed (e.g., slice 2) others show more than one above-average fluorescent signal source (e.g., slice 4). The hybrid reconstruction using priors [Fig. 4(b)] resolves the single fluorescence sources very well (e.g., arrows in slices 1 and 2) but seems not to be able to distinguish between three proximate but distinct spots (like in slice 4). Those are reconstructed as one single fluorescence source centered between the three source points.

The stand-alone method [Fig. 4(a)], in turn, allocates most of the reconstructed signal outside the lung tissue. Most of this distribution is, moreover, not in the expected region but above and below the lung.

As in vivo optical imaging achieves less resolution than the ex vivo validation method, reconstructed distributions can “irradiate” into adjacent tissue slices and appear there with less intensity. This can, for example, be observed in the third and fourth slice of the hybrid reconstruction [Fig. 4(b)]. The fluorescence signal in the middle of both (beneath the spine) only appears in the fourth ex vivo slice and is quasi anticipated in the third slice of the hybrid reconstruction.

Note that due to the process of freezing and small displacements of the animal during this time, small shifts between ex vivo and in vivo images are expected and no utterly identical representation can be achieved.

Figure 4(d)–4(f), finally, show the 3-D view of the fluorescence distribution as to the ex vivo distribution [Fig. 4(f)], hybrid [Fig. 4(e)] and the stand-alone [Fig. 4(d)] reconstructions. Here, it becomes even more obvious where the main fluorescence signals in the stand-alone reconstruction are allocated. We applied a threshold to both 3-D representations in order to receive similar sized fluorescence signals in the lung. In the stand-alone case [Fig. 4(d)], this leads to proportionally higher signal intensities in those regions that were determined by the reconstruction to be the main fluorescence hot spots, such as around the shoulder and below the lung.

For the hybrid reconstruction, basically all areas are located inside the chest. This corresponds well to the ex vivo 3-D image, where the chest region around the heart [transparent structure inside the chest in Fig. 4(f)] is displayed. The most intense fluorescence spots are shown in red and arrows highlight the associated regions in Fig. 4(e) and 4(f).

3.4.

Reconstruction Evaluation

Figure 5 evaluates the difference in reconstruction accuracy for the three reconstruction methods applied to ex vivo imaging [Fig. 5(a)] and the two reconstruction methods applied to in vivo imaging [Fig. 5(b)].

Fig. 5

Comparison of reconstruction accuracy using the different reconstruction approaches in the phantom study (a) and in the in vivo study (b). (a): Plot of the percentage of reconstructed signal located inside a certain radius around the actual tube. (b): Plot of tumor/muscle and tumor/lung ratios in the ex vivo cryoslices versus in vivo stand-alone (diamond) and hybrid (dot) reconstructions.

Figure 5(a) (top) shows one exemplary slice with the concentric circles at distances of 0 to 5 mm from the tube depicted in red. The innermost circle is at 0 mm distance from the tube which means that it represents exactly the tube boundary. Each of the other circles has a distance of 1 mm from the other circles. The graph in Fig. 5(a) (bottom) shows the percentage of the recovered signal as a function of the distance from the tube for the stand-alone FMT (blue, diamond), FMT-XCT using only the boundary of the phantom (yellow, square) and FMT-XCT using priors (red, dot). Stand-alone FMT, thus, only localized 19 percent of the reconstructed signal inside the tube and, even in a distance of 5 mm around the tube, not more than 68 percent of the signal could be found. The reconstruction using the accurate mouse boundary and Tikhonov reconstruction recovered 33 percent of the signal inside the tubes and went up to 91 percent in a distance of 5 mm. The hybrid reconstruction using priors finally started with a localization of 77 percent inside the tubes and already reached 100 percent in a distance of 2 mm around the tubes.

For the evaluation of the reconstruction accuracy in the in vivo mouse study, we plot, in Fig. 5(b), the ratios of tumor/muscle and tumor/lung in the cryoslices versus the ratios of the same regions in the FMT reconstruction. In order to correlate with the ex vivo data, the in vivo data should, therefore, linearly increase with increasing cryoslices data. Therefore, acceptable data pairs would lie in the two quadrants highlighted in light red. It can be seen that almost all data pairs (92 percent) from the reconstruction using priors fulfilled this requirement and the general trend of the data showed a linear increase (red dots and dashed line). The reconstructions from the stand-alone FMT, in contrast, only plotted half of the data pairs inside the red quadrants (50 percent) and the trend line shows that the FMT signal was decreasing with increasing cryoslices signal (blue diamonds and continuous line).