The ability to image multiple fluorophores simultaneously (multiplexing) in the visible regime has been critical for understanding a variety of biological processes.1, 2, 3, 4 Conventionally, multiplexing is achieved using fluorophores with different emission wavelengths and filtering the combined emission into multiple channels, or by postprocessing a spectral measurement to unmix the combined signals. These techniques have been widely adopted for visible fluorescence imaging at the microscopic and whole-animal levels.
In the NIR spectral regime , imaging has traditionally been performed on one to two fluorophores. Limited bandwidth and the highly overlapping spectra of most commercial NIR fluorophores make it difficult to resolve more than two probes at once without sophisticated unmixing algorithms. Near-infrared fluorescence (NIRF) tomography is further complicated by the differential effects of tissue absorption for a multiwavelength measurement.5 Fluorescence lifetime is an identifying characteristic, separate from the emission spectrum, that can be used to distinguish fluorophores.6 We have developed a methodology for NIRF tomographic multiplexing using characteristic fluorophore lifetimes to separate multiple probes. This approach relies on the fact that fluorophores exhibit a time-resolved decay pattern that can be easily characterized and is very often monoexponential with a decay rate of . A mixture of fluorophores, when excited by an impulse, will exhibit time-dependent fluorescence emission that is a sum of the respective fluorophore decays:, amplitude components can be determined by directly fitting .7, 8 Equation 1 provides a simple way to unmix or separate the signal from multiple fluorophores and has been used for surface imaging in a number of recent studies.8, 9, 10 When fluorophores are imaged deep in tissue, i.e., in tomographic measurements, it can be shown that fluorescence decay can be expressed in a form similar to Eq. 1 with slight modifications to account for the time-dependent effects of tissue absorption and scattering.11
We present here the first in vivo lifetime-multiplexed NIRF tomography based on linear unmixing of the time-domain signal and subsequent tomographic reconstruction of the individual lifetime components. We begin by characterizing the lifetime behavior of a number of commercially available NIR fluorophores in vivo to determine the suitability for lifetime-based multiplexing. We next compare lifetime-based fluorophore multiplexing to traditional monoexponential analysis, and establish the limits of quantitative multiplexing based on imaging system noise parameters and lifetime differences. Finally, we demonstrate tomographic lifetime multiplexing with up to three fluorophores in nude mice.
Time-resolved imaging was performed with a free-space, time-domain, tomographic imaging system described previously.12 Excitation was provided by a fiber-coupled, femtosecond laser at , with average power, repetition rate (Mai Tai, Newport-Spectra Physics, Mountain View, California). Excitation light was either: 1. focused (point source) and scanned across the ventral surface of the animal from below for tomography, or 2. expanded with an engineered diffuser (Thorlabs, Newton, New Jersey), resulting in a illumination region at the imaging platform for planar fluorescence imaging. Excitation and emission images were acquired with a cooled CCD camera (Picostar HR-12 CAM 2, LaVision GmbH Goettingen, Germany; or hardware binning) mounted to a voltage-gated image intensifier (Picostar HR-12, LaVision GmbH, Goettingen, Germany). For all measurements, a background image (with no excitation) was subtracted. Light was focused into the intensifier with a lens (AF Nikkor, f2.8, Nikon), and filtered with filters mounted directly to the front of the lens (TFI Technologies, Greenfield Massachusetts; Chroma Technology Corporation, Rockingham, Vermont). 3-D boundaries were captured with a photogrammetric camera (3-D Facecam 100, Genex Technologies Incorporated, Kensington, Maryland). Gates were collected every for in planar scans and every for in tomographic scan. CCD exposure time ranged from , and the buffer readout time was for binning, and for hardware binning. A typical planar scan with exposure and 80 frames (taken every ) took to acquire. A typical tomographic scan imaged at 52 source positions for in steps of (46 gates) with exposure took , including the excitation and 3-D camera measurements. The total tomography acquisition took roughly .
Asymptotic Lifetime Fitting
Lifetime analysis was performed in the “asymptotic” regime, defined as the portion of the temporal decay where the intrinsic diffuse temporal response has completed its evolution, i.e., for all times greater than the diffuse time constant .11 The beginning of the asymptotic time regime was determined for each source-detector (SD) pair independently by finding the time gate for which the time integral was at 99% of its maximum value. The rationale behind this approach is that this integral is an approximation of the time-dependent amplitude coefficient for the decay of the particular lifetime .12 Lifetime components were fit according to Eq. 1 for time points . The time gate , which corresponds to the time of incidence of the laser pulse at the boundary of the imaging medium, was determined directly from an excitation measurement taken with sources projected on a piece of white paper in the imaging plane;12 the time axis for all subsequent measurements and analysis was translated such that for the time gate. We used weighted, nonlinear least squares with non-negative constraints (fit.m in Matlab) to determine amplitude components. Weights were , determined from a quantitative noise model of our system, .13 was determined experimentally for each of the binning schemes used: hardware and software binning, 6.53; hardware and software binning, 8.53; hardware and software binning, 11.38; and hardware and software, 15.72.
Tomographyis the three point or composite Green’s function, calculated at the tissue optical properties and reduced by a factor proportional to the respective lifetime, and is the source-detector scaling coefficient that compensates for free-space propagation from a curved surface. Note that Eq. 2 is similar to Eq. 1, but with the decay amplitudes replaced by the integral over the tomographic weight functions. We accounted for transmission heterogeneities by directly measuring excitation light propagation and reconstructing optical properties with the Fourier components of the TD data.5 The recovered optical properties were then input into a voxel-based Monte Carlo forward model14 at resolution to generate . was calculated by comparing the forward generated fluence with the excitation measurement , i.e., by solving the linear equations .15 Equation 2 was solved for each lifetime component independently, with generalized, weighted linear least squares.16
In Vivo Evaluation of Near-Infrared Fluorophore Lifetime Characteristics
A variety of NIR fluorophores (indocyanine green, Akorn, Lake Forrest, Illinois; Osteosense 750 nm, Genhance 750, VisEn Medical, Bedford, Massachusetts; IRDye 800CW, LI-COR Biosciences, Lincoln, Nebraska; Kodak X-Sight 761, Carestream Health, Rochester, New York: Atto 740, Sigma-Aldrich, Saint Louis, Missouri; Alexa 750, Invitrogen, Carlsbad, California) were evaluated for in vivo lifetime characteristics. Mice (C57, , Charles River Laboratories, Wilmington, Massachusetts nu/nu, , , the Jackson Laboratories, Bar Harbor, Maine; NCr- , , Taconic, Hudson, New York) were anesthetized with 2% isoflurane or avertin (1.3% 2,2,2-tribromoethanol, 0.8% tertpentylalcohol; ), administered NIRF fluorophores (at IV), and imaged in the prone and supine positions with planar fluorescence excitation at time points from to after injection, depending on the fluorophore pharmacokinetics. For time points more than after fluorophore administration, animals were allowed to recover from anesthesia, and then anesthetized again immediately before the later time point. Lifetime histograms were generated by a monoexponential fit of all pixels that had cw intensity at least 10% of the maximum intensity. Following euthanasia, organs were imaged in situ and en bloc after removal.
Multiwell Calibration Experiment
Osteosense 750 and Kodak X-Sight 761 were prepared at varying concentrations in 5% albumin solution, which has been shown to better approximate in vivo lifetime.17 We filled a grid of wells in a 96-well plate with concentrations from , by steps of , resulting in a two-axis gradient of the two probes. The wells were imaged in planar fluorescence mode with 750 excitation and a bandpass filter. Amplitude uncertainty was calculated as the root of the diagonal of the covariance matrix of the coefficient estimates, or , where is the design matrix, sse is the sum of the squared error, and dfe is the degrees of freedom.
In Vivo Lifetime Multiplexing
Nude mice ( , , the Jackson Laboratories, Bar Harbor, Maine; NCr- , , Taconic, Hudson, New York) were administered Osteosense 750 ( , before imaging) and Kodak X-Sight ( per mouse, before imaging) by tail-vein injection. For three fluorophore experiments, nude mice ( , NCr- , Taconic, Hudson, New York) received Osteosense 750 ( , prior to imaging), Atto 740 ( , prior to imaging), and Qtracker 800 ( , prior to imaging) from the Qtracker 800 cell labeling kit (Invitrogen, Carlsbad, California). Mice were imaged at multiple time points before and after administration of fluorophores in planar fluorescence mode with excitation and in transmission geometry for multiple SD pairs at the excitation wavelength ( excitation, or BP filters) and emission wavelength ( LP filter).
In Vivo Lifetime Characteristics of Commercial Fluorophores
Lifetime multiplexing assumes that each fluorophore present has a characteristic lifetime decay in vivo. The variety of chemical environments encountered in vivo could potentially cause a single fluorophore to exhibit a range of lifetimes across the mouse, as described for some probes.6, 18 This could complicate multiplexing, because of the loss of lifetime-based specificity.
However, for a number of commercially available probes, we observe narrow in vivo lifetime distributions (Fig. 1 ). With intravenous administration, these fluorophores have surface temporal decays that are well fit with a monoexponential model, and lifetime distributions that have little variance across the mouse. Lifetimes for the different dyes range from (Table 1 ). Some fluorophores undergo characteristic lifetime changes during normal biodistribution and pharmacokinetics. Kodak X-Sight 761 exhibits lifetime shortening when accumulating in the liver (from over ). Atto 740 has a long lifetime in the kidneys , but shortens when accumulating in the bladder or when excreted in the bile to the gastrointestinal (GI) tract . Most probes that are cleared via the kidneys undergo similar lifetime shortening in the bladder, as described by other groups.19
Lifetime characteristics of commercial NIR fluorophores. Emission wavelengths are taken from product literature. Mean lifetime ± the standard deviation are derived from monoexponential fit of images taken at time points from 1minto24h following fluorophore injection with the mouse supine or prone, depending on the organs of interest. Only pixels with peak intensity > 10% of the maximum intensity were considered in the fit (to avoid low-intensity noisy pixels). The standard deviation for all pixels was ∼2 to 3× the values listed here. τ±(ps) for Kodak X-Sight 761 shows lifetime shortening in the liver over 14h and for Atto 740 lifetimes in the kidneys and bladder, respectively. Alexa 750 is antibody conjugated.
|Kodak X-Sight 761||789|
In summary, a number of commercial NIR fluorophores have lifetime properties in vivo, conducive to lifetime-based multiplexing. Lifetime characteristics for these probes are relatively uniform across the entire animal and can be easily determined in advance.
Single Versus Multiexponential Analysis
When two or more fluorophores are present simultaneously, the measured fluorescence is a sum of photons from the respective fluorophores. Monoexponential analysis (e.g., fitting with a monoexponential function) of a mixed signal provides an average lifetime for each detector or pixel, but can be misleading, given the contribution of the fluorophores at different concentrations and quantum efficiencies (see Fig. 2 ). In contrast, a multiexponential analysis based on expected lifetime components can produce a quantitative measure of multiple fluorophores. Moreover, a monoexponential analysis does not exploit the full power of lifetime-based tomographic separability as afforded by Eq. 2.
As an example, consider multiplexing Osteosense 750, a bone-targeted probe, with Kodak X-Sight 761, which remains in the blood stream and accumulates slowly in the liver. When just Osteosense 750 is present in the animal, the monoexponential lifetime map exhibits a narrow lifetime distribution with mean [Fig. 3 ]. Administration of Kodak X-Sight results in a lifetime map with a large distribution of lifetimes, ranging from [Fig. 3]. The average lifetime at bony structures is significantly higher than with Osteosense alone, due to the contribution of vascular Kodak X-Sight. Biexponential fitting using average lifetimes for Osteosense and Kodak X-Sight components results in clear anatomical separation of the two probes [Figs. 3, 3, 3, 3, 3, 3]. The Osteosense components before and after Kodak X-Sight administration are qualitatively similar [compare Figs. 3 and 3] and no Kodak X-Sight is detected by the biexponential fit before administration [Fig. 3].
Noise Considerations for Lifetime Multiplexing
It is clear from Fig. 3 that the amplitude images recovered from lifetime unmixing are influenced by noise. The effect of measurement noise is an uncertainty in the recovered decay amplitudes, which can be quantified by the variance, . This uncertainty depends on both the measurement noise and the separation of the lifetimes involved. To estimate , we analyze the propagation of noise from the measurement (time domain image) to the recovered amplitudes using linear regression theory.20 The time-dependent measurement is linearly related to the amplitudes of the individual decay components decay as , where the columns of are the normalized, time-dependent monoexponential decays and are the component amplitudes. The uncertainty in the recovered amplitudes is dependent on the measurement noise at each time point, expressed as the weight matrix , where , and the respective basis functions :3 allows calculation of recovered amplitude uncertainty given known system noise parameters, which determine , and fluorophore lifetimes, which dictate the basis functions . Simulations with Eq. 3 show that, as expected, amplitude uncertainty increases as the lifetime separation between probes decreases [Fig. 4 ]. For a given lifetime separation, the relative amplitude uncertainty for one component increases as the amplitude of the other component increases [Fig. 4]. Practically, this means that if one component is much weaker than the other (Note that the decay amplitude is the product of fluorophore concentration and extinction coefficient.), it will have increased relative uncertainty. Fluorophore separability, defined arbitrarily as the conditions under which relative uncertainty is , can be estimated directly as described before. For example, assuming one fluorophore with lifetime , quantitative unmixing is possible for at relative concentrations as low as 1:5 [Fig. 4].
These observations are corroborated experimentally with Osteosense and Kodak X-Sight in 5% albumin solution, , mixed at varying concentrations in a multiwell plate (Fig. 5 ). Amplitude uncertainty is estimated from the recovered amplitude sample uncertainty (an approximation to ). Recovered amplitudes are linear in concentration with . For the imaging specifications typically used in our system, the relative amplitude uncertainty is at concentrations ranging from (Fig. 5), and for all concentrations except Osteosense with Kodak X-Sight. The amplitude uncertainty is highest for high amplitude signals, regardless of which fluorophore is contributing more photons to the mixed signal. Under conditions where one fluorophore is at a low concentration relative to the other, the low concentration dye has the lowest signal-to-noise ratio and therefore the highest relative amplitude uncertainty.
Measurement noise also affects the determination of the lifetime from a monoexponential fit, e.g., when measuring the in vivo lifetime characteristics of a fluorophore. We estimated the propagation of measurement noise by simulating a fluorophore that has a fixed and realistic amplitude distribution (chosen from the mean lifetime and amplitude distribution of a similar in vivo measurement), and noise according to a conservative empirical model; the simulated measurement is then fit at each pixel for . As shown in Fig. 6 , the uncertainty in due to noise is , which accounts for the majority of the in vivo lifetime heterogeneity observed in this study, and provides a qualitative explanation for the limits on fluorophores unmixing (described before). Finally, measurement noise sets a limit on the number of fluorophores that can be multiplexed. Figure 7 shows the amplitude uncertainty with a simulation of lifetime multiplexing using up to five lifetimes with the shot noise model. This indicates that under the noise statistics used here, the maximum number of fluorphores that can be multiplexed reasonably (less than 30% relative uncertainty) is three.
Tomography of Multiple Fluorophores
We demonstrate tomographic lifetime multiplexing (described in Sec. 2) for two and three anatomically targeted fluorophores injected in living mice. As a first example, we image Kodak X-Sight and Osteosense in nude mice , with a time-resolved, limited angle, free-space tomographic imaging system.12 Excitation and fluorescence measurements are collected from a grid of sources and detectors ( separation) after Kodak X-Sight administration and after Osteosense administration [Fig. 8 ]; at these time points, the two probes are localized to specific organs (liver and bones, respectively) with characteristic lifetimes and little off-site accumulation. Individual measurements are fit for two exponentials ( and ) and display a fluorophore-specific distribution [Fig. 8]. Note that the scale for the Kodak X-Sight amplitude component is -fold greater than the Osteosense amplitude scale.
At this time point ( after Kodak X-Sight administration) on planar fluorescence images, Osteosense is confined to bony structures, including the spinal column, pelvis, skull, and long bones, whereas Kodak X-Sight has localized to the liver, with residual Kodak X-Sight in the vascular compartment [Figs. 8 and 8]. This observation is consistent with previous biodistribution studies of the respective fluorophores alone. Lifetime-based 3-D reconstructions corroborate the planar fluorescence images: Kodak X-Sight is observed in the liver region [Figs. 8, 8, 8, 8] and Osteosense is localized to the pelvis and the back. These findings were confirmed by postmortem imaging of the organs and bones. Thus, the two lifetime components are regionally distinct and correctly localized.
3-D reconstructions exhibit some plume artifacts, especially in the axis. In addition, Osteosense is not detected at the highest point of the spinal column, above the liver and thoracic cavity, due to the low transmission across those structures. These effects may be attributed to the limited-angle measurement scheme employed by our system, and primarily impact the Osteosense reconstruction because of its anatomical distribution.
Lifetime multiplexing can be extended to more than two fluorophores. As an example, we image Qtracker 800, a quantum dot with an extremely long lifetime, in addition to Atto 740 ( at postinjection) and Osteosense 750. Because it is much longer than the pulse repetition frequency of our laser, the Qtracker lifetime can be approximated as the DC component in Eq. 1. The three fluorophores have distinct biodistributions and anatomical targets: Atto 740 is cleared via the liver and secreted with the bile to the gastrointestinal tract; after , Atto is localized to the duodenum. Qtracker (from the cell labeling kit) is cleared immediately by the liver, and remains stable there for .
When imaged together, there is distinct localization of the three fluorophores. Qtracker is resolved primarily in the liver, Osteosense in the bony structures, and Atto in the proximal small bowel, localized to the right upper quadrant (Fig. 9 ). These locations match surface fluorescence images and postmortem imaging. Surface and tomographic images are obtained with Atto and Osteosense components -fold greater than the Qtracker component, and for a relatively small difference in lifetime for the organic compounds .
Lifetime-based multiplexing enables simultaneous imaging of multiple fluorophores. This simple approach overcomes the spectral bandwidth limitations of the NIR regime, and in contrast to frequency domain (FD),21 monoexponential analysis,6 forward Laplace,22, 23 or moment-based approaches,24 determines fluorophore concentrations directly from the time-domain signal without the need for additional data processing. Unmixed amplitudes are linear with fluorophore concentration (Fig. 5) and can be input directly into the inversion equation for 3-D reconstructions.
Lifetime multiplexing should be generalizable to the full NIRF probe repertoire, after proper a-priori characterization of in vivo fluorophore decay characteristics. The commercial fluorophores we tested have narrowly distributed, monoexponential lifetimes and are therefore suitable for in vivo multiplexing (Table 1). It is expected that antibody-conjugated probes would exhibit similar in vivo lifetime characteristics. From simulations and experiments, we expect that any two fluorophores that differ by could be imaged together; for example, any of the short-lifetime probes tested (indocyanine green, Osteosense 750, and Alexa 750) could be multiplexed with the long-lifetime probes (Kodak X-Sight 761 and Atto 740).
The separability of a given dye pair can be determined directly from the lifetimes and imaging system noise. For our system, quantitative unmixing of multiple fluorophores is not possible for at lifetimes around ; above , unmixing is possible with better than 30% amplitude uncertainty for relative fluorophore concentrations of at least 1:5 (Figs. 4 and 5). This limit in can at least qualitatively be explained by noise propagation from the image to : simulations show that the standard deviation in is . Thus, asymptotic lifetime separation, as described in this study, is noise limited and therefore system dependent.
Some have suggested that because asymptotic fitting includes noisy data points from the “tail” of the fluorescence decay, it is inherently disadvantaged compared to cw imaging. However, this is at least partially alleviated by increasing the CCD exposure time to achieve detector saturation at the earliest portion of the asymptotic regime, and by weighting the lifetime fit with the measurement variance. For fluorophores with large lifetime separation, the amplitude variance approaches the measurement noise, and thus is comparable to cw measurement noise, but with the benefit of multiplexing.
Although monoexponential probes are desirable, more complicated fluorophores could be imaged with multiexponential fits (assuming a-priori lifetimes). In the case of a fluorophore with a distinct lifetime when accumulated at the target,6 two lifetimes could be used in a multiexponential fit to produce separate images of target-bound and unbound probes. With more complicated decay functions, such as autofluorescence, it may be possible to approximate fluorophore decays with multi-or stretched exponential basis functions,25 although calculation of the tomography forward problem in these cases may be more difficult. We conclude that most, if not all, NIR fluorophores could be used for lifetime-based tomographic multiplexing.
In this study, we extended lifetime multiplexing to 3-D tomography using a theory developed previously.7 We imaged in vivo in nude mice with two and three fluorophores that localized to distinct anatomical targets, as confirmed by postmortem imaging (Figs. 8 and 9). Lifetime-separated reconstructions corresponded well to unmixed 2-D planar images and known organ locations.
Tomographic reconstruction also has limitations inherent to measurement geometry and light diffusion physics. The reconstructions presented here are prone to poor -axis resolution and full-thickness penetration in the thorax because of the limited-angle configuration of our tomographic imaging system. More sophisticated, full measurement schemes have somewhat alleviated these issues for cw tomography.26 We anticipate that integration of time-resolved measurement into a full rotational tomography system will enable better -axis resolution and penetration in highly absorbing regions.
Lifetime-based NIRF multiplexed tomography is a powerful technique that should allow translation of other lifetime-based methods to 3-D imaging. Fluorescence lifetime is sensitive to FRET between donor and acceptor fluorophores,27 and thereby may enable tomographic imaging of protein interactions throughout the mouse. Furthermore, fluorophores can be designed to exhibit lifetime shifts upon target binding;28 with lifetime multiplexing, signals from the unbound probe (due to imperfect uptake) could be effectively removed, resulting in improved signal-to-noise reconstructions of the target-bound probe. Probes with two distinct lifetime states (ligand bound versus unbound) also offer the possibility of quantitative measurements of ligand concentration. For example, visible fluorescent sensors have been used to measure absolute calcium concentrations in solution and in vivo.29, 30 The success of these future applications will depend primarily on the development of NIR analogs to existing visible fluorophore sensors.
In summary, we have demonstrated a new technique for tomographic imaging of multiple fluorophores in small animals. Our lifetime-based approach uses a-priori information about fluorophore decay characteristics to separate a fluorophore’s distinct signal from that of others. Once unmixed, the signal can be directly input into the tomography inverse problem, resulting in multiplexed 3-D reconstructions. Asymptotic lifetime separation is possible for fluorophores with even small lifetime differences and for many fluorophores at once. We anticipate that lifetime-based NIRF tomography will be a powerful tool for molecular imaging in the future.
This research was supported by NIH EB000768 and AG026240. Raymond was supported by NIH T32 EB001680.