Fluorescence imaging can become an ideal modality for improving surgical procedures, since it relates directly to the surgeon’s vision and offers attractive characteristics, including flexibility in generating contrast, high detection sensitivity, and portability. As a result, significant attention has been given toward identifying fluorescence imaging strategies that reveal surgical markers otherwise invisible to the naked eye, with applications ranging from improving tumor delineation,1, 2, 3, 4, 5 sentinel lymph node identification, and metastasis staging,6, 7 to neuronal activity monitoring8, 9 or vascular mapping. 10, 11 To fully capitalize on the potential of fluorescence for intraoperative applications, robust imaging technologies need to be developed for in-vivo imaging.
Fluorescence imaging of tissues can be compromised by the spatial variation of the tissue optical properties. This is because the fluorescence intensity collected depends nonlinearly on the tissue’s optical properties. For example, highly absorbing lesions, such as highly vascularized tumors, may show darker than surrounding less-absorbing tissue, even if they contain larger amounts of fluorochrome compared to the surrounding tissue.12 Likewise, a semitransparent or nonabsorbing lesion such as a cyst or a lymph node may appear brighter than surrounding muscle tissue, even if both the cyst and the muscle contain the same fluorochrome concentration. The fluorescence intensity on an image has also a strong nonlinear dependence on the depth of the fluorescence activity. For example, a superficial fluorescent lesion will appear brighter than an identical lesion that is located deeper in tissue. Finally, tissue autofluorescence, i.e., the fluorescence emerging from native tissue fluorochromes such as collagen or NADH, may provide useful diagnostic information13, 14, 15 but it can also reduce the contrast in visualizing exogenously administered fluorescence signals by yielding a nonspecific background signal.
Tomographic fluorescence imaging developed for in-vivo applications typically utilizes illumination and data collection along different projections and combines the measurements with mathematical descriptions of photon propagation in tissues to reconstruct images corrected for the effects of tissue optical heterogeneity and depth.12, 16 By using volumetric 3-D reconstructions, narrow bandpass filters, or appropriate subtraction methods, the effects of tissue autofluorescence can also be minimized. When considering real-time fluorescence imaging however, the application of multiprojection schemes and tomographic methods becomes challenging due to the scanning and computational times involved, which are generally significantly longer than video-rate speeds. Correspondingly, planar epi-illumination imaging approaches have been considered that cannot easily resolve depth but can lead to easy-to-implement video-rate imaging while offering correction for autofluorescence or the effects of optical property variation.17, 18, 19, 20 These methods suggest a computationally efficient processing of images acquired at two or more spectral bands, typically utilizing a form of ratio imaging, either between fluorescence measurements18 or a combination of fluorescence and light attenuation images at the excitation or other wavelengths where the fluorochrome of interest does not fluoresce.19, 20 Besides the use of ratios, another strategy to reduce autofluorescence is to utilize near-infrared fluorochromes, since tissue autofluorescence in the near-infrared is much smaller compared to the visible.21 In this case, increased penetration depth can be achieved compared to systems operating in the visible, due to the significantly lower attenuation of near-infrared light by tissues, compared to visible light.
In this work we consider the development of a novel fluorescence imaging system, operating in the near-infrared, which enables real-time operation and ratio-based image correction for heterogeneous light absorption in tissues, developed for intraoperative surgical imaging. The system’s unique design allows the collection of three-channels, i.e., fluorescence, light absorption, and color images simultaneously, through one lens and the same field of view, while operating in real-time mode, i.e., at video-rate capacity of . These data are then used to implement a previously developed fluorescence correction method,16, 20 which corrects the measurements at the emission (fluorescence) wavelength by dividing them with geometrically identical light attenuation measurements in tissue at the same or similar wavelengths, for example at the excitation wavelength. This approach has been shown to be capable of yielding accurate epi-illumination fluorescence images for at least a three-fold change of spatially varying absorption in the tissue imaged.20 We showcase implementation aspects of this system and further demonstrate the performance achieved using phantoms and controlled postmortem mouse measurements. Taken together, the imaging of anatomy in the color channel combined with corrected fluorescence measurements using dual-channel imaging may provide the basis for accurate fluorescence epi-illumination imaging and video in a variety of interventional procedures, offering fluorescence quantification ability in real-time mode.
System Design and Overview
The key design parameters of the system developed were: 1. the capability of simultaneous video-rate imaging in NIR emission (fluorescence signals), NIR excitation (intrinsic signal), and visible wavelength bands; 2. the capability of providing image normalization; 3. operation at long working distances, large field of view, high detection sensitivity, and high resolution; and 4. a highly optimized detection sensitivity in all spectral bands considered.
To reach these functional specifications, a three-channel detection system was developed as shown in Fig. 1 . The selection of three cameras is based on the need to use different camera specifications for color and near-infared (NIR) imaging while providing simultaneous readings of all the necessary measurements. The system specifications, as described in Sec. 3, are shown in Table 1 . Illumination was provided by a cw laser diode (BWF2-750-0, B and W Tek, Newark, Delaware; maximum power ) for fluorochrome excitation, and a halogen lamp (KL-2500 LCD, Edmund Optics, Barrington, New Jersey) for white-light color imaging. The system has the capacity to operate in epi-illumination and transillumination modes. Epi-illumination imaging was performed using a front illuminating expanded beam by guiding laser light through a multimode fiber ( core/0.22 NA) to a collimator and a diffuser (F260SMA-B, ED1-S20, Thorlabs, Newton, New Jersey) for beam expansion and uniform illumination. White light was also coupled with a fiber optic bundle and homogenized by a beam expander (EG8546, Edmund Optics). Transillumination was accomplished by guiding laser light through a multimode fiber that was scanned on the bottom surface of the target, through an antireflection-coated glass window, using the translation stage (MAXY4009Q1-S4, Velmex, Bloomfield, New York). Transillumination measurements were achieved using a diffuser in front of the multimode fiber to achieve more homogeneous back-illumination. and axes were scanned with four steps of and three steps of , respectively.
Imaging system specifications.
|Physical size of imaging module||(mm)|
|Surgical field of view||(mm) to (mm)|
|Primary magnification||0.73 X–0.09 X|
|Sensor resolution (spatial)||(pixel), pixel size|
|Sensor resolution (temporal)||(full frame), (binning 2Xver)|
|Image windows||Visible, fluorescence, intrinsic, normalized|
Light from the surgical field was collected using a zoom lens (NT58-240, Edmund Optics). Linear polarizers were employed for minimizing the effects of glare and specular reflection in the attenuation and color images by enabling cross-polarized measurements for white and excitation (intrinsic) light measurements. The additional advantage of polarization is that it samples deeper into tissue compared to nonpolarized imaging, which is an important parameter in getting a more accurate reading on tissue attenuation in epi-illumination mode. However, when switching to imaging through the cross-polarizers, lower resolution is generally achieved. This feature is used selectively when specular reflection indeed occurs.
The primary image of the object formed by the zoom lens falls at the focal plane of each relay lens group (MAP10100100-A1, Thorlabs) of three imaging channels. The setup employs two dichroic mirrors (700 DCXXR, z 750 RDC, Chroma, Rockingham, Vermont) that splits light by excitation, emission, and visible wavelengths into three imaging channels. The final images are formed on each chip surface of a grayscale charge-coupled device (CCD) camera with 20% quantum efficiency in the NIR area and a color CCD camera (PCO AG, Donaupark, Kelheim, Germany) by each relay lens group. This relay lens group has the same object and image distance, so it does not change the magnification. Its purpose is the delivery of the primary image from the lens to each CCD chip, while allowing the intersection of dichroic mirrors, filters, and potentially other imaging components. All optical pathlengths from the lens to the cameras are equal (not plotted in scale in Fig. 1). For accurate wavelength-band selection and cross talk elimination, bandpass filters (z , HQ , Chroma) were used in front of CCD cameras in excitation (intrinsic) and emission imaging channels. The transmission spectrum of each imaging channel, which is induced by the combination of dichroic mirrors and the bandpass filter, can be seen in Fig. 1c. For implementing a portable self-contained system, all optical components were tightly mounted in custom coupler/holders via C-mount optics after exact optical alignment.
The control software and user interface for surgical guidance was developed under the LabVIEW (National Instruments, Austin, Texas) and Matlab (The MathWorks, Natick, Massachusetts) environments. In continuous acquisition mode, NIR excitation (intrinsic), NIR emission (fluorescence), and anatomical color image can be displayed simultaneously in real time and refreshed continuously at a rate set by the camera exposure time using two computers. In a single snap acquisition mode, when a set of images is captured, then instantly the normalized fluorescence image is calculated and displayed, as shown in Fig. 1b. The user interface provides complete control on the imaging system including camera gain, exposure time, number of binning, image display, and image archiving.
Measurement of Optical Resolution
To measure the spatial resolution of the system, the image of a standard three-bar resolving power test target (USAF-1951, NT53-714, Edmund Optics) was taken using the visible wavelength channel and white-light illumination. This target contains numbered bar groups, each consisting of three vertical and three horizontal bar patterns. The ratio of the spatial frequency of an element to the one of the next higher element is . After extraction of cross-sectional intensity profiles from each horizontal and vertical bar element, the points of the contrast transfer function (CTF) are obtained with the contrast equation, i.e.,and are the maximum and minimum gray values of adjacent line patterns, and is the number of line pairs per millimeter (lines/mm), which shows corresponding spatial frequency for each bar element.
To examine the metrics of fluorescence quantification correction as it relates to spatial variations of excitation light in tissue, we constructed a phantom consisting of five tubes containing the same fluorochrome concentration but different concentrations of absorbing dye, resulting in variation in the underlying optical properties. The five tubes were made of translucent -diam plastic and contained of Alexa Fluor 750 dye (Molecular Probes, Eugene, Oregon) and 1% Intralipid solution. In addition, the tubes contained India ink of varying concentration from with increments to impart differences in attenuation between tubes. The imaging chamber had an open upper surface, whereas its bottom surface was a glass window, which was antireflection coated for the NIR to facilitate both epi-illumination and transillumination measurements. The tubes were placed in an imaging chamber and positioned parallel to each other, allowing a gap between consecutive tubes. The chamber was filled with 1% Intralipid solution (Sigma, Saint, Louis, Missouri) and India ink (yielding optical properties of absorption coefficient and reduced scattering coefficient , characteristic of small animals). The thickness of the fluid phantom was and the tops of all the tubes were positioned below the top surface of the fluid solution. A similar phantom containing a single tube was also constructed and was employed to examine the effects of background absorption variation. A single tube was immersed in the Intralipid solution, and the absorption of the Intralipid solution was altered by titrating India ink in the solution that surrounded the tissue.
Euthanized mice (C57BL/6J, ) were employed to showcase imaging feasibility simulating intraoperative imaging. After euthanasia, the abdominal sides of the mouse were surgically exposed, and the lumbar nodes beside the inferior vena cava were visualized by removing the surrounding peritonea and fat. Imaging of the nodes was selected because it allows for visualizing well defined volumes. of Alexa Fluor 750 with of India ink was injected into one lymph node and the same fluorescent solution, and without ink into another lymph node ( for each node), using a 30-gauge needle.
Measurements and Normalization
The correction scheme considered assumes a raw fluorescence image in epi-illumination or transillumination modes, i.e.,is the raw fluorescence image and is the dark current image, respectively. We also examined the corresponding images after normalization with the light attenuation (intrinsic) images acquired. The corresponding normalized image can be written as is the light attenuation image obtained at the excitation wavelength (intrinsic image), and is the corresponding dark current image obtained with no illumination for the same acquisition parameters used for . We set to zero unless the denominator values are above a threshold , typically set to 5% of the maximum of the denominator values to avoid divisions with very small noise-affected values. In transillumination mode, all fluorescence and attenuation images obtained from the light source scanning are summed up after subtraction with the noise (dark current) images and to show the whole field of view.
The spatial resolution of the implemented system was determined utilizing the USAF-1951 pattern target image and contrast transfer function (CTF) as shown in Fig. 2 . The test pattern image consisting of repeated gratings is obtained at visible wavelength channels with the highest zoom setting [Fig. 2a]. For every different grating size in the field of view, we calculated the CTF parameter using Eq. 1. Figures 2b and 2c illustrate the CTF values (vertical axis) versus the different grating frequencies (horizontal axis).
Figures 2b and 2c show the contrast transfer function in the image plane as a function of the spatial frequency along the horizontal and vertical directions, respectively. To quantify the transverse or lateral resolution of the system, we applied the Rayleigh criterion and the Sparrow criterion to the profiles measured from the phantom by the system developed. The Rayleigh criterion defines resolution as the distance between two point objects, where the Airy disk minimum of the image of one object coincides with the maximum of the Airy disk of the other object. Typically at this limit the contrast between the maximum and minimum intensity observed between the two objects becomes about 26.4%. The Sparrow criterion defines the resolution of an optical system as being equal to the cut-off distance, i.e., the distance below which two objects cannot be distinguished by means of contrast. Since the line pair corresponding to a contrast value of 26.4% was 7.33 and in the horizontal and vertical CTF curves, respectively, by taking the half-width between two line pairs we determined the transverse resolution of the system to be and the lateral resolution to be by the Rayleigh criterion. Correspondingly, the transverse and the lateral resolutions of the system by the Sparrow criterion were found to be 36 and , respectively. Since three imaging channels are constructed with identical optical components and pathways, the resolution characteristics were similar in all channels with minor variations , largely due to wavelength-dependent characteristics of the optical components utilized.
Figure 3 shows the results obtained in epi-illumination from the fluid phantom containing the five tubes of identical fluorochrome concentration but of varying attenuation, simulating tissue optical heterogeneity. Figure 3a depicts the image obtained at the excitation wavelength (intrinsic image). A characteristic pattern of varying light absorption by the tubes is visible, in an increasing fashion from tube 1 to tube 5, due to the increasing concentration of the absorber. The uncorrected fluorescence image in Fig. 3b shows the dependency of the fluorescence signal on the varying absorption in the tubes, and demonstrates correspondingly a decreasing fluorescence signal with increasing tube attenuation. In this case, while all the tubes have the same fluorescence concentration, the image reports varying fluorescence intensity that could be erroneously interpreted as changes in the amount of fluorescence present in the tubes. In contrast, the normalized fluorescence image shown in Fig. 3c correctly depicts a similar fluorescence strength from all tubes by correcting the effects of background attenuation. In addition, it shows a better delineation of the tubes compared to the uncorrected images. Finally, the images demonstrate very low specula reflection, as it is successfully eliminated by means of the cross-polarized linear polarizers employed in the system.
To quantitatively determine the differences between non-normalized and normalized images, the average photon counts in all tubes were calculated over identical regions of interest during image post-processing. The region of interest was chosen as an area spanning the length of the tube and a width of , with the center being the center of the tube. The ratio graph in Fig. 3d indicates that the quantification performance of the normalized fluorescence image is significantly better than in the standard epi-illumination image, the latter showing signal variations as high as 25%.
Figure 4 correspondingly depicts the quantification results between corrected and uncorrected fluorescence imaging when the absorption of the background was changed. As can be seen, changes in the background also affect the signal recorded, as expected, and in this case again the corrected approach yields better insensitivity to the variation of light attenuation.
Figure 5 depicts the transillumination imaging results obtained from the same phantom as in Fig. 3. In transillumination, the light propagates through the entire volume of the tubes. This generally leads to more prominent variations of signal intensity compared to epi-illumination, since a larger optically heterogeneous volume is sampled. Figures 5a and 5b depict the raw data obtained in the excitation and emission wavelengths, respectively. The fluorescence signal intensity varies significantly in the uncorrected image. In contrast, Fig. 5c depicts the normalized image showing a significantly more homogenous response from the fluorescent tubes. Finally, the chart shown in Fig. 5d summarizes the imaging results. It can be observed that the quantification performance of normalized transillumination imaging is significantly better than the uncorrected transillumination image. Especially for absorption values up to of the reference sample, the error of normalized imaging is less than 5% while the corresponding error for uncorrected transillumination images reaches 24%.
While Figs. 3, 4, 5 demonstrate general performance metrics obtained from measurements on phantoms, Fig. 6 depicts epi-illumination images obtained from a mouse after euthanasia and surgical exposure of the abdominal area. In this case, two lymph nodes were injected with solutions of identical fluorochrome concentration but different absorption to simulate varying tissue optical properties, as described in Sec. 2. This model could be relevant to real-time pathology studies, where nodes are sampled for achieving real-time intraoperative pathology, but it is more generally used here to obtain performance metrics using tissues.
Figure 6a shows a color image obtained from the field of view using the color video camera. This view closely relates to the surgeon’s field of view and it is important for reference purposes. The inferior vena cava (double arrow) and two lumbar nodes (single-line arrows) are seen on the color figure. Figures 6c and 6d depict the attenuation (intrinsic), fluorescence, and normalized fluorescence images, respectively. Although the same concentration of fluorochrome was injected in both lymph nodes, the right lymph node (dotted line arrow) was darker (i.e., more attenuating) compared to the left node (solid line arrow), as identified in Fig. 6b. Figure 6c is the uncorrected fluorescence image that shows fluorescence activity from areas surrounding the node. This behavior has been previously documented20 and is likely due to escaping photons from the node toward the lower attenuating surrounding tissue. From an imaging standpoint this is an erroneous result, as it indicates that the fluorescence dye was injected in the tissue surrounding the node. Conversely, the normalized image of Fig. 6d depicts a significantly more accurate picture of fluorescence biodistribution and correctly resolves the nodes as the areas of fluorescence origin.
These images further confirm the observations of Figs. 3, 4, 5, where the measured fluorescence intensity did not depend only on the fluorochrome concentration but also on the heterogenous attenuation of light in the underlying tissue. It also further showcases that the spatial appearance and contrast of the image also improves when the fluorescence image is corrected for the underlying optical properties. In that respect, correction can lead to a more accurate and robust performance.
The propagation of fluorescence imaging into interventional procedures requires a modality that is robust and reliable, independent of tissue optical parameters or illumination variations. While often unappreciated in epi-illumination imaging, the light intensity incident on the surface is spatially modulated by optical properties, which in turn modulate the fluorescence intensity in an unknown fashion unless explicitly measured and used to correct the corresponding fluorescence images.
We developed a system appropriate for real-time intraoperative imaging that employs a three-channel imaging approach, used to deliver color real-time video imaging as well as corrected (normalized) fluorescence images. This approach is offered as an alternative to simple “photographic” fluorescence imaging or video, and leads to improved accuracy independently of optical property variation in tissues, as demonstrated here with phantoms and tissues postmortem. In particular, we found that the system allows quantification errors of less than 8%, even at a five-fold change of absorption variation within the fluorescent lesion, whereas uncorrected images show errors of 25%. In transillumination, image correction yielded 16% quantification error versus 45% for the uncorrected image. Finally, when the background absorption was varied by three-fold, the corresponding errors recorded were 35 and 60% for corrected and uncorrected images, respectively. Overall, the performance of corrected (normalized) images shows significant insensitivity over a wide variation of background absorption versus the uncorrected images. This performance is particularly important, for example, when imaging highly vascular, nonvascular, or more general areas with variations in hemoglobin concentration, i.e., the major light absorber in tissue. This performance is also relevant to volumetric sampling of tissues, i.e., in imaging entire tissue masses or nodes using transillumination with a small catheter illuminator,22 since the correction applied is particularly well suited for improving the quantification and overall image quality in transillumination images as well.
It should be noted that since the system utilizes light of constant intensity, it is insensitive in sensing whether a spatial light attenuation variation is due to absorbing or scattering variations. Illumination and detection of light of constant intensity (in contrast to using intensity modulation or ultrafast pulsing technologies12) is a common technology applied to intraoperative imaging, as it leads to the use of practical and economic systems.3, 4, 6, 9, 19, 23 However, in contrast to the insensitivity of the ratio to spatial changes of the absorption coefficient, the ratio of fluorescence to intrinsic intensity scales with tissue scattering is a factor of .16 Therefore, although the method can accurately improve fluorescence quantification in cases of tissue vascularization changes, more advanced technologies such as frequency domain24 or time-resolved approaches will be important for decomposing absorption from scattering effects and for better application of the ratio correction when strong scattering variation is present in the tissues. Such systems should still correct for the variation of optical property in tissues, before yielding accurate quantification of surface and subsurface fluorescence activity, using the known dependency of the ration on scattering variations as well. It should be also be pointed out that the division of Eq. 3 also corrects for inhomogeneous illumination due to imperfect beam shape or light obstacles that may interfere with the illumination, since they multiply both intrinsic and fluorescence measurements and therefore are exactly cancelled out by a ratio approach.
There is a strong dependence of imaging performance on depth. This dependence has been studied in the past20, 25 in the context of analyzing the performance of the method implemented here by the proposed system. For deeper seated fluorescence activity, the correction advantages proposed are only possible in transillumination mode. This is because the use of the “intrinsic signal” measurement for correction is appropriate only in the first of depth, where relevant light attenuation contrast is present. This is the reason why the system has also a developed transillumination branch. The premise is that for deeper seated activity or large tissue volumes, the surgeon would be able to direct focused illumination from a certain angle (projection) into tissue, using a catheter system, instead of using wide-plane epi-illumination. Light diffusion in this case plays the role of “beam expansion” in the tissue, and a larger or deeper volume of tissue can be sampled in this way. It was also shown in mice that this approach offers significant betterment compared to uncorrected fluorescence patterns.20
The increasing availability of fluorescent probes with enhanced in-vivo sensitivity and specificity to molecular disease markers makes fluorescence imaging a very appealing method for real-time intraoperative visualization of invasive tumor borders and local metastatic foci. The fluorescence concentration values utilized here have been drawn from typical in-vivo animal measurements. The ability to detect with high sensitivity fluorescence distributed in tissues has been amply demonstrated in the literature using highly sensitive CCD cameras appropriate for low light imaging. Overall the detection performance of such system is of the order of a few tenths of femptomoles or better for typical organic fluorescent dyes operating in the NIR.12 Using the previous normalization technique, fluorescence imaging can become a robust imaging technique that is insensitive to tissue optical heterogeneity and illumination variations. While quantitative volumetrical sampling can be better achieved by fluorescence molecular tomography12 or multispectral optoacoustic tomography (MSOT),26, 27 these techniques require time-consuming scanning and computation that is not appropriate for real-time imaging. The system presented can work synergistically with other methods, such as intraoperative radiological or nuclear medicine methods to offer an integrated solution of macroscopic noninvasive imaging combined with high resolution invasive imaging. Using different magnifications, the same system can be used for fluorescence inspection of the field of view and then zoomed in for high resolution imaging of regions of interest. Likewise the basic setup developed could be used as the back end of a surgical microscope or endoscope system. In addition, by using different spectral areas, multiple fluorescence probes can be further resolved to improve the information available to the surgeon. We have recently developed a multispectral approach for true real-time imaging,28 and we are currently considering its implementation with the correction system and strategy presented. Overall we expect that quantitative normalized fluorescence imaging will become an important intraoperative enhancement in improving surgical vision.
The author’s acknowledge useful discussions with Go Van Dam. In addition to institutional funding, this work was further supported by the DFG grant MOBITUM. Jung Sun Yoo was supported by the Korea Research Foundation Grant funded by the Korean government (MOEHRD, KRF-2007-612-C00046).