Fluorescence lifetime imaging and fluorescence lifetime imaging microscopy (FLIM) are molecular imaging techniques that are useful for preclinical and clinical studies in living cells, small animals, and human tissues, with fluorophore excited-state lifetime providing image contrast.1
However, low fluorescence signals from biological samples can be a challenge, causing poor lifetime precision, and this will affect, to a great extent, the quantitative applications of FLIM, such as the detection of Förster resonance energy transfer for molecular interactions and the sensing of fluorophore microenvironments.2, 3, 4, 5 When endogenous fluorophores are imaged, low fluorescence signals may result from low intrinsic fluorophore concentrations and/or unfavorable optical properties of fluorophores (e.g., fluorescence excitation and emission wavelengths, quantum yield, photobleaching rate). When exogenous fluorophores are imaged, low signals can result from the low fluorophore concentrations that are required to minimize effects on sample physiology and/or from the low transfer efficiency of fluorophores/fluorophore precursors (genes or different forms of fluorophores) from extracellular media into live cells. To increase measured fluorescence signals from biological samples, high-intensity excitation sources, such as lasers, can be used in FLIM, but this may cause unexpected cell responses and sample damage,6, 7 and may also increase photobleaching rates. Another way to increase fluorescence signals is to use lower excitation intensity coupled with longer image acquisition times, but sample movement will be a major concern of this approach in live-cell imaging.8 Considering the above challenges, a combination of low fluorescence signals, low excitation light intensity, and fast image acquisition can make FLIM data very imprecise in biological applications.
In this study, we focused on lifetime precision improvements in time-gated FLIM, where fluorescence intensities at different delay times along a decay curve were integrated by a detector, as in Fig. 1 . To improve the precision of lifetime determination in FLIM, error analysis with Monte Carlo (MC) simulations may be used to determine optimal gating schemes.8, 9, 10, 11, 12 Optimal gating schemes of rapid lifetime determination (RLD) for single-exponential decays (a two-gate protocol) with respect to relative standard deviation [(RSD), also commonly known as coefficient of variation] of lifetime have been reported,10, 12 and an error analysis of RLD for double-exponential decays has also been addressed.11 Recently, optimization of fluorescence lifetime sensing in frequency domain was studied.13 In our laboratory, we constructed optimal gating schemes for double-exponential decays with several different lifetime determination methods.14 In this report, because of the robustness15 of the single-exponential four-gate protocol (Sec. 3.2), we used this protocol and determined its optimal gating schemes with both MC simulations and analytical solutions (Sec. 4.1) for the first time, and then used the optimal schemes to improve the precision of time-gated FLIM (Sec. 4.2) under low-light and fast imaging (up to ). The approach presented here helps avoid sample damage, photobleaching, and unwanted sample movement detection in fluorescence lifetime imaging applications.
In addition, we combined optimal gating with image “denoising” (Sec. 3.5), which also has the potential to improve FLIM precision. The term “to denoise” means “to remove noise,” especially the noise introduced by imaging systems. The image-processing algorithms commonly used to denoise images can be either local or global. Local denoising algorithms [such as Gaussian smoothing, Tikonov denoising, and total variation (TV) model denoising] are sometimes preferred because they only need neighboring pixels to implement smoothing of a certain region in an image and work well in most cases.16 Global denoising (such as Fourier–Wiener filtering), on the other hand, might be best used for images with repeated patterns, in which fine structures may be preserved because the information of the whole image is adopted to determine the value of a certain pixel in the processed image. A summarized classification of currently used image denoising algorithms, as well as some comparisons among them, has been reported.16
In this study, we demonstrate, for the first time, how optimal gating (a method applied to the temporal dimension of a FLIM image) and TV denoising (a method applied to the spatial dimension of a FLIM image) can be used in combination to improve the precision of lifetime determination in time-gated FLIM. Because the two methods apply to different dimensions, we assume that they can work either independently or in combination. We demonstrate that lifetime precision can be improved in a regime pertinent to live-cell FLIM studies. In addition, both Poisson- and non-Poisson-distributed noise were taken into consideration (Sec. 2) because although Poisson-distributed noise is a common form of noise for photon-counting devices, other forms of noise may appear due to nonunity gain and nonideal behaviors of real imaging systems, as well as image-processing procedures, such as lifetime determination. The models reported here can remove Poisson-distributed noise and other forms of noise with high flexibility and speed.
TV models are constructed with the definition of their “energy,” or , through minimization of which the processed image evolves to a stable state that should be close to the original image without noise corruption. The basic form of the energy [Eq. 1] includes a regularization term, which utilizes total variation (defined as the integral of the absolute value of the gradient of the image, assuming the image is a continuous function) to denoise the input image , and a fidelity term, which implements fitting of the processed image to the input image and decides how large the “distance” can be between these two images. A favorable property of TV models is that they perform selective smoothing and hence are edge-preserving,
The Rudin–Osher–Fatemi model17 is a commonly used TV model, but it assumes that the noise level, or magnitude, is constant. To deal with varying magnitude of noise, which usually occurs in real imaging systems, Le 18 developed a TV model that was suitable for Poisson noise. They demonstrated that Poisson noise in artificial images could be removed with their model, while low-contrast features were preserved in regions of low intensity. Other TV or non-TV denoising methods have also been developed either to handle Poisson noise or to have varying regularization parameters that can potentially be used to remove Poisson noise.19, 20, 21, 22
However, although Poisson-distributed noise is a common form of noise for photon-counting devices, other forms of noise may appear due to nonunity gain and sometimes nonideal behaviors of real imaging systems. In addition, to directly denoise FLIM lifetime maps, the deformed noise distribution after lifetime determination and the dependence of this distribution on intensity and lifetime need to be considered as well. This produces an entirely different form of noise. The novel TV models we used in this study14 not only can deal with Poisson noise, but also can be easily and flexibly adapted to take into consideration any forms of intensity-dependent, lifetime-dependent, or even spatially dependent noise introduced by imaging systems and image-processing procedures.
The novel TV models we developed for this study have the general form, denoted as variance-weighted TV (VWTV),denotes the signal domain, indicates the local variance of (as a function of and ), is the fidelity coefficient, and the variables and represent the spatial location of the pixels. The fidelity term (second term on the right-hand side) is a variance-weighted least-squares fitting term. The weighing here helps us to adjust the importance of the fidelity term, based on the local noise level, relative to the TV regularization term, which is the first term on the right-hand side and is the term that removes noise. With this algorithm, the final that gives minimal should still look like (hence, the features are preserved) due to the fitting term, while noise is being removed due to the TV regularization term. The values of were determined by the “discrepancy rule,”18 which requires the fidelity term evaluated with and the final to be the same as that evaluated with and the estimated uncorrupted image.14 For the specific application of denoising intensity images, we further developed a novel, modified -weighted TV (FWTV) model14 based on an -weighted fidelity term, represents the ratio of the signal variance to the mean intensity counts. can be either a constant (for imaging systems with constant gain values) or a function of local mean intensity, in which case , which can be evaluated for real imaging systems.14
To implement VWTV and FWTV denoising, the gradient descent method was used to obtain the time derivative of . The processed image , with initial guess as , then evolved through iterations (time steps) to minimize energy.
To implement time-gated FLIM, we employed a novel time-domain, wide-field FLIM system for picosecond time-resolved imaging for biological applications [Fig. 1].5, 23 A dye laser (GL-301, Photon Technology International, Lawrenceville, New Jersey) pumped by a nitrogen laser (GL-3300, Photon Technology International, Lawrenceville, New Jersey) for UV–visible–near-infrared (NIR) excitation provided a wide-field, less expensive, and potentially portable alternative to multiphoton excitation for subnanosecond FLIM of biological specimens.23 A sample was illuminated by an excitation pulse, and the fluorescence emission was recorded by an intensified charge-coupled device (ICCD) camera (Picostar HR, LaVision, Germany) at a gate delay controlled by the intensifier, with emission intensities integrated during a gate width. The ICCD had variable intensifier gain and gate width settings varying from and could be used to implement high-speed imaging in other applications as well.24 In addition, this system had a large temporal dynamic range ( to ∞), lifetime discrimination, and spatial resolution of , which made it very suitable for studying a variety of endogenous and exogenous fluorophores in biological samples.2, 4, 25, 26, 27, 28 Fluorescence lifetime maps were determined by first acquiring fluorescence intensity images at four delays and then calculating the lifetime values from the intensity images on a pixel-by-pixel basis (described in Sec. 3.2).
The gating parameters [the gate width, , and the time interval between the starting points of two consecutive gates, , see Fig. 1] can be optimized by using MC simulations10, 11, 12 or applying error propagation (described in Sec. 3.4).
Four-Gate Lifetime Mapping
To create fluorescence lifetime maps rapidly, a four-gate protocol with a linearized least-squares lifetime determination method was used on a pixel-by-pixel basis. This method is more precise than the two-gate protocol while still easy to implement,11, 29, 30is the lifetime of pixel , is the intensity of pixel in image , is the gate delay of image , and is the number of images. All sums are over .
Additional steps in data processing are needed for more accurate lifetime map production. Before lifetime calculation, the step “background subtraction” takes average of the intensities of pixels within a specified background region and subtracts that average value from all pixels. After background subtraction, the step “reject” sets intensities to zero for all pixels with intensities below a certain value (assigned as the parameter “reject”). After lifetime calculation, the step “ range” sets lifetimes to zero for all pixels with lifetimes above a certain value (assigned as the parameter “taurange”) to remove lifetime values in physically meaningless regions. In this study, “reject” was set to 10 and “taurange” was set to 15.
Sample Preparation and Imaging
Fluorescent beads with diameters of (Cat. no. 18140, Polysciences, Warrington, Pennsylvania) were suspended in distilled water to produce a solution with a final concentration of . Before imaging, of the solution was placed on a T dish (Bioptechs, Butler, Pennsylvania), and the imaging process with the time-gated FLIM system was begun after the beads had settled to the bottom of the dish. All beads had excitation/emission maxima of , as specified by the manufacturer. A 40× microscope objective was used. The voltage across the microchannel plate of the intensifier was set at . The beads were excited at using the laser dye coumarin 440 and the fluorescence was collected at .
In this study, single-exponential gating optimization was utilized. The optimal gating of the four-gate protocol [Eq. 4] was first determined by MC simulations, in which the relative standard deviation (RSD) (defined as the standard deviation divided by the mean value) of the determined lifetime values was minimized by changing the gate width and the time interval between the starting points of two consecutive gates , assuming that only Poisson noise was present (Fig. 2 ). In addition, and did not vary with different gates, meaning that once and are chosen for a simulation, the values for all the four gates are the same and the values between the first and second, second and third, and third and fourth gates are the same as well. For a different simulation, new and values will be chosen. Because these distributions were constructed by MC simulations with the repetitive addition of noise, RSD provided a quantitative measurement of the precision of the determination of a certain parameter, such as lifetime. Because this study considered only a single-exponential decay, the intensity profile could be written as , and the RSD of either (the pre-exponential term) or (the lifetime) could be minimized. In this study, we optimized the gating scheme for the best precision of determination.
Alternatively, the RSD of the lifetime determined by the four-gate protocol could be analytically determined by applying error propagation to Eq. 4, with the assumption that the variance of the fluorescence intensity was the same as the intensity magnitude, which is a characteristic of Poisson noise. Again, the optimal gating parameters were determined when the minimal RSD was achieved.
As mentioned above, other forms of noise in addition to Poisson noise may appear in real imaging systems. This will be considered in our future studies and should further improve FLIM precision with optimal gating.
Total Variation Denoising
Two approaches were used with TV denoising to improve the precision of lifetime determination in FLIM. In “lifetime denoising” [Fig. 3 ], a lifetime map was first constructed by four-gate lifetime mapping. Because the variance of lifetime was not proportional to the lifetime values, VWTV [Eq. 2] was used. The variance estimation, as a function of , , , and total photon counts [(TC), the photon counts integrated under the entire decay curve, see Fig. 1], was performed by analytically solving the error propagation of Eq. 4, which led to the following equation:3], each time-gated intensity image was denoised before four-gate lifetime mapping. In this case, TV denoising was performed with FWTV [Eq. 3], using the values of previously characterized as a function of local mean intensity, because other forms of noise, in addition to Poisson noise, were expected.14
Results and Discussion
Determining Optimal Gating
The RSD values as a function of lifetime-scaled and were consistent (Fig. 4 ) whether obtained from the MC simulation (Fig. 2) or the analytical solution [derived from Eq. 4], especially for high total photon counts (or ). The analytical solution suggests that the RSD of lifetime is inversely proportional to , which can be observed in Figs. 4 and 4: The contours had exactly the same shapes with tenfold differences in their values. When TC was large enough , the MC simulation and analytical results were consistent. When TC was small , higher RSD values were predicted by MC simulation in the outer nonoptimal regions.
Indeed, we do not expect the results from both methods to be exactly the same. The accuracy of the analytical solution may suffer from the linearization approximation in error propagation derivation, therefore underestimating the RSD when the errors were highly nonlinear at low photon counts [Fig. 4 versus Fig. 4]. On the other hand, the MC simulation should be more accurate at low photon counts, but the accuracy may suffer from the limited number of simulations and the usually more discretized parameter values used as inputs for the simulations.
In spite of the differences in the results of the two methods, both results from the MC simulations and the analytical solutions suggested the same optimal gating scheme, which was independent of TC: The optimal was of the lifetime value and the optimal should be greater than at least threefold of the lifetime value (only negligible improvement exists once ). In this case, the gates actually overlap. Although the determination of the optimal gating scheme requires the lifetime value of the sample, an approximate lifetime value is usually available from previous knowledge of the sample, or a test experiment with arbitrary gating.
Therefore, for the fluorescent bead sample mentioned above (Sec. 3.3) with lifetime value of (determined with the time-gated FLIM system), the optimal gating should be around and . This prediction was validated experimentally, as shown in Fig. 5 .
What can also be observed in Fig. 4 is that apparently RSD had a greater dependence on than on . Therefore, in our experimental validation of the optimal gating (Fig. 5), we only changed (the open circles in Fig. 4) around its optimal value, , while having fixed at (the dashed lines in Fig. 4).
Figure 5 shows that the RSD curve from analytical solution overlapped with that from MC simulation multiplied by 10. This means that when TC was high, the curves from two approaches were consistent. As mentioned previously, the MC simulation at low TC predicted higher RSD when the gating was not optimal. All these three curves had the minimal RSD at ns, which was confirmed by the RSD curve from the experimental data.
The above approach assumed single-exponential decay. If the number of components in the sample is unknown, then the suggested procedure will be to use the optimal gating of single-exponential decay first, aiming at the averaged lifetime value, to acquire the least noisy overall decay behavior. This curve can then be fitted with single- and double-exponential decay to determine which one fits the curve better. For a decay curve with more than two components, however, more than four gates will be needed (see below).
As for the optimal gating with more than one decay component, first, we can further apply the optimal gating schemes of double-exponential decays, which have been constructed in our laboratory, for lifetime precision improvement, either independently or in combination with image denoising.14 More than two components can also be considered in the future, using the same procedure shown in Fig. 2. However, in this case, at least gates are needed to fit the intensity profile and solve all the parameters (there are one lifetime and one pre-exponential term for each component), and the computational work for the optimal gating determination will become much more complicated.
Improving Precision of Lifetime Determination in Time-Gated FLIM
Reduction in relative standard deviation
Optimal gating, lifetime denoising, and intensity denoising all improve FLIM precision. This is demonstrated in Fig. 6 , where the noise distribution within the FLIM maps of fluorescent beads is illustrated. We note that this is a fairly low-light case with total photon counts only around 100. The holes inside the fluorescent beads [Fig. 6] came from one of the lifetime calculation steps, in which the values above a certain threshold were set to zero. Because this threshold was set to , random fluctuations in low-light imaging caused some pixels to have lifetime values more than four times larger than the expected values if the gating scheme was not optimal. After intensity denoising [Fig. 6], the image became smoother and the RSD value dropped to 46%, but the extremely high values above the threshold still could not be removed. This was similar to the lifetime-denoised map [ , Fig. 6]. Optimal gating [Fig. 6] removed these artifacts and further decreased the RSD value to 20.1%, as well as reducing the diameter of the beads so that it became closer to the actual bead size of . This effect on the spatial pattern was actually due to the fact that more pixels were properly “rejected” in data processing (Sec. 3.2) after denoising. Further improvement was then achieved by denoising the optimally gated image with either the denoising approach [ and 13.7%, Figs. 6 and 6, for lifetime denoising and intensity denoising, respectively]. A comparison of Fig. 6 to Fig. 6 [or to Fig. 6] shows that most of the remaining lifetime random variations within the beads in the optimally gated image could be removed by denoising. Here, the combination of optimal gating and TV denoising resulted in about a fourfold improvement in precision. In addition, the results in Fig. 6 suggested that the improvement from the intensity denoising ( , in this case) could be independent of that from optimal gating ( , in this case). Although 6% may seem small relative to an RSD of 51.5% (nonoptimal gating), it is quite large relative to an RSD of 20.1% (optimal gating), because it is a one-third reduction in RSD. Therefore, image denoising is particularly important when optimal gating is also applied.
The nonoptimally and optimally gated intensity images, their gating schemes, and their corresponding FWTV-denoised images are shown in Fig. 7 . We can clearly see that, because the optimal gating had a larger value [Fig. 7 versus 7], the intensity decay trace could be more easily observed in Fig. 7 compared to Fig. 7. This was also true for Fig. 7 compared to Fig. 7. On the other hand, after denoising, the removal of noise [Fig. 7 versus Fig. 7 and Fig. 7 versus Fig. 7] was obvious while the geometry of the beads was not affected.
In this section, we conclude that optimal gating and image TV denoising can be employed either independently or in combination to improve precision in low-light time-gated FLIM. When these two methods are combined, their overall fourfold (from 51.5 to 13.7%) improvements in precision can be easily observed in our low-light example (Fig. 6).
Lifetime denoising versus intensity denoising
We note that when comparing lifetime denoising and intensity denoising in terms of RSD, intensity denoising had a greater influence on the precision of lifetime determination than lifetime denoising. However, it is obvious that they actually produced somewhat different denoised lifetime maps [Fig. 6 versus 6, and Fig. 6 versus 6]. Their individual strengths and weaknesses arise from their different denoising mechanisms. Lifetime denoising appeared to be worse for removing the irregularities in the geometry of objects arising from noise [such as the noisy edges in Figs. 6 and 6]. This was probably because TV denoising is edge preserving, and therefore, the irregular edges could not be easily removed once they were already introduced into the lifetime map by noise. On the other hand, lifetime denoising appeared to be better for smoothing off-edge, internal pixels for pattern revealing because it worked directly on the lifetime map and therefore could remove the overall uncertainties from the intensity images at once (as long as they were not on the edges). These uncertainties may have a better chance to remain unremoved after individual and independent intensity image denoising.
Our techniques can be further applied to time-correlated single-photon counting (TCSPC) FLIM, which is a commonly used method for live-cell lifetime imaging. Optimal gating could be applied to TCSPC FLIM by virtual gating, in which the values of data points within each virtual gate were summed up to form an intensity image. The four-gate protocol can again be used for virtually gated TCSPC FLIM. Similarly, denoising approaches, including both intensity denoising and lifetime denoising, can also be applied to TCSPC FLIM. It has been demonstrated that a greater than fivefold improvement in lifetime precision can be achieved in TCSPC FLIM images when optimal virtual gating and TV denoising are applied in combination.31
As for the image denoising methods alone, it will be interesting and useful to employ, optimize, and compare other image denoising techniques specifically for the applications of FLIM, either independently or in combination with optimal (virtual) gating.
Finally, because optimal denoising improves other advanced image processing techniques such as image deconvolution,32 segmentation, and object tracking, the combination of denoising and these techniques can also be studied specifically for FLIM use.
We report promising techniques that can remove uncertainties and improve precision in time-domain FLIM maps. With time-gated FLIM, notable fourfold improvements in lifetime precision (RSD from 51.5 to 13.7%) can be easily observed in our low-light (total photon ) example.
Theoretically, optimal signal gating is generally applicable because the relative RSD reduction is independent of feature geometry and total photon counts and, therefore, it should work for all kinds of samples (different lifetime values will have different optimal schemes, though, according to the results in Sec. 4.1). Furthermore, our novel TV denoising models have been tested on artificial images with different geometries and lifetime values,14 and the results indicated that our TV models could always improve local lifetime determination while still preserving lifetime fidelity. The algorithms reported here have been encoded in Matlab for ease of implementation.
In conclusion, the approach presented here helps improve FLIM data while increasing imaging speed and minimizing sample light exposure to avoid biological sample damage, photobleaching, and unwanted sample movement detection in FLIM applications.
This work was supported in part by a research grant from National Institutes of Health (No. NIH CA-114542).