1.2.1.

FLIM of NAD(P)H and FAD for metabolic imaging

Nicotinamide adenine dinucleotide (NAD) and flavin adenine dinucleotide (FAD) are two metabolic coenzymes that play a myriad of roles in cellular oxidation and reduction reactions. The reduced form NADH and oxidized form ${\mathrm{NAD}}^{+}$ are involved in mitochondrial function, energy metabolism, calcium homeostasis, gene expression, oxidative stress, aging, and apoptosis. The reduced NAD phosphate (NADPH) is associated with reductive fatty acid biosynthesis, steroid biosynthesis, oxidative stress, and antioxidation, while the oxidized form (${\mathrm{NADP}}^{+}$) is involved with calcium homeostasis.⁹ Real-time monitoring of cellular metabolism during pathophysiological changes is possible by measuring the redox ratio ($\mathrm{NADH}/{\mathrm{NAD}}^{+}$). NADH is the principal electron acceptor in glycolysis, which results in two NADH molecules per glucose molecule. The Krebs cycle also reduces ${\mathrm{NAD}}^{+}$ to NADH in three of its reactions. During oxidative phosphorylation, NADH is oxidized to ${\mathrm{NAD}}^{+}$ by donating electrons to the electron transport chain, and these electrons are ultimately accepted by oxygen.^8,9 In the case of anaerobic glycolysis, ${\mathrm{NAD}}^{+}$ is converted to NADH and oxidative phosphorylation is diminished, which creates an overall increase in NADH abundance. Thus, the reduction–oxidation pair $\mathrm{NADH}/{\mathrm{NAD}}^{+}$ serves as an indicator of balance between oxidative phosphorylation and glycolysis. Flavins such as FAD are also involved in cellular oxidation–reduction reactions. The reduced form (${\mathrm{FADH}}_2$) is oxidized to FAD in complex II of the electron transport chain, while FAD is reduced to ${\mathrm{FADH}}_2$ in pyruvate decarboxylation and the Krebs cycle.

NADH and FAD are fluorescent while ${\mathrm{NAD}}^{+}$ and ${\mathrm{FADH}}_2$ are not. The fluorescence of NADH and NADPH are difficult to distinguish, and their combined fluorescence is referred to as NAD(P)H. Due to the pivotal role of NADH, NADPH, and FAD in cell biology and metabolism, these endogenous fluorophores have been used to monitor cellular redox reactions, energy metabolism, and mitochondrial anomalies under different pathophysiological conditions. Chance and others in the 1980s established NAD(P)H and FAD fluorescence for metabolic imaging.^44–48 The use of endogenous fluorescence enables noninvasive metabolic imaging of cells and tissue in their native physiological environment without perturbations associated with contrast agents. After the development of FLIM instrumentation, biophysicists began to relate the fluorescence lifetimes of NAD(P)H and FAD to cellular metabolism.^24,25,29 The fluorescence lifetime of NAD(P)H is significantly shorter in the free state ($\sim 400\mathrm{ps}$) compared with the protein-bound state ($\sim 1$ to 5 ns) of the molecule.^24,25,27 This is due to quenching in the free state as the NAD(P)H molecule folds and diminished quenching in the protein-bound state as the NAD(P)H molecule extends. Conversely, FAD has a longer lifetime in its free state (2.3 to 2.9 ns) compared with its protein-bound state ($<0.1\mathrm{ns}$).^29,30,49,50 Bird et al. used FLIM to demonstrate a correlation between the redox ratio ($\mathrm{NADH}/{\mathrm{NAD}}^{+}$) and the relative amounts of free to protein-bound NAD(P)H.⁵¹

2.1.1.

Wide-field FLIM

WFI uses a parallel illumination field at the focus of the objective lens and collects fluorescence from the focal plane onto a camera (Fig. 5). Wide-field FLIM is often used for rapidly imaging large sample areas since light from the entire field of view is collected using a camera-based detection. Wide-field FLIM uses either time-domain techniques, such as TCSPC¹³⁵ or TG, in which a series of fluorescence images are collected by shifting a timing window (order of nanoseconds) through the emission decay,^136–138 or frequency-domain methods of demodulating the fluorescence signal from the excitation frequency.¹³⁹

Wide-field FLIM has the advantages of higher frame rates and less photodamage when compared with LSM. However, camera sensitivity and SNR are not as high as that of LSM detectors, which results in poorer axial resolution. In wide-field collection, every camera pixel simultaneously detects scattered light from all other pixels of the illuminated area, which intermixes the timing-spatial coordinates.¹³³ Assuming a fixed photon emission rate from the sample, image optimization is a trade-off between either spatial or temporal resolution.¹³³ Wide-field techniques such as structured illumination and spinning-disk confocal can achieve higher spatial resolution without compromising imaging speed.^{136,140–143} Other advantages of wide-field systems are their simpler implementation and the low computational cost to assign photon detection times in each pixel. Some of the advancements in wide-field FLIM include its implementation with Nipkow disc microscopy for fast 3-D FLIM imaging,¹⁴² wide-field coupled with single plane illumination microscopy for high-resolution 3-D FLIM,¹⁴⁴ TG single photon avalanche diode (SPAD) cameras for phasor-based high speed wide-field FLIM,¹⁴⁵ multifrequency widefield,^146,147 and image gating by pockel cells.¹⁴⁸ Current wide-field FLIM systems are discussed in detail by Suhling and Hirvonen et al.¹⁴⁹

2.1.2.

Laser Scanning Microscope FLIM

Laser scanning microscopes have out-of-focus rejection methods that enable higher contrast and spatial resolution than wide-field systems. In comparison with wide-field FLIM, LSM-FLIM modalities are often coupled to faster electronics to generate precise photon detection times per pixel. As discussed previously, these methods fall under two modalities of timing calculation: TDC-based or TAC-based.

2.1.3.

Laser sources

FLIM uses modulated laser sources for illumination. This can be achieved by many modern pulsed laser diodes that are modulated with an internal trigger or the high power density, ultrafast laser sources developed in the 1990s. These lasers have a remarkably short pulse duration (hundreds of femtoseconds), durable repetition frequency (in the order of 0.1 GHz), and tunable wavelengths in the near-infrared region. These lasers are currently used extensively for in vivo and small animal imaging due to their use as a multiphoton excitation (MPE) source (explained in the section below). Pulsed light sources are popular because of numerous applications in digital communications and remote sensing. Nonlinear light sources, such as supercontinuum sources, are also popular because they achieve near continuum tunability over a large wavelength range.

2.1.4.

Confocal and multiphoton microscopes

Both CLSM-FLIM and MP-LSM FLIM are broadly used in applied sciences to study biology and materials. Confocal imaging methods use a pinhole (small aperture) to reject out-of-focus light. Most biomedical confocal systems use a low power laser for excitation and focus the light at one point in space using a pair of galvanometric scanners (XY scanner). Precise movement of the objective controls for the $Z$ position. The fluorescence emission from the 3-D focal volume retraces back through the XY scanner (thus descanned) and reaches the detector. The focused spot is scanned across the sample to detect photon density pixel-by-pixel. A computer records the photon density (i.e., fluorescence intensity) along with the location of the XY scanner and $Z$ position to generate a CLSM image. The difference between LSM and CLSM is the use of the pinhole in CLSM that enables axial ($Z$-plane) selection. Comprehensive reviews of CLSM are available elsewhere.^150–152

Multiphoton FLIM uses MPE, generally two-photon (2P) or three-photon (3P) excitation, which relies on high photon density to achieve nonlinear excitation of fluorescence. This high density is achieved by lower energy, higher wavelength photons in the near-infrared region. In 2P excitation, two photons of half the energy spontaneously come together to excite the molecule to a higher electronic energy level, which then follows its regular radiative decay (fluorescence) route to relax back to the ground state. Multiphoton FLIM is widely used for tissue imaging because near-infrared wavelengths achieve deeper penetration depths in tissues compared to the visible wavelengths that are commonly used in single photon excitation. This is due to reduced scattering and absorption in tissues within the near-infrared wavelength window. The nonlinear excitation scheme of MPE limits the fluorescence excitation to a small focal volume comparable to the confocal detection volume, but without a pinhole. This allows MP-LSM detectors to be placed in the transmission mode (or nondescanned mode), instead of descanning through the scanning optics. This nondescanned geometry enables higher detection efficiencies. Multiphoton systems use tunable, mode-locked lasers that provide ultrashort, high intensity pulses. A popular source is titanium–sapphire crystal lasers with tunability between 680 and 1100 nm. Most MP-LSM systems include pulsed sources to achieve high photon density, so additional FLIM capabilities only require the timing electronics to estimate photon arrival times. Therefore, many MP-LSM systems likely include FLIM, unlike CLSM systems that conventionally use a continuous-wave excitation source. MP-LSM systems often collect 3-D image tomograms over deeper depths than CLSM. Reviews of MP-LSM are available from Refs. 152 to 156.

Simultaneous excitation of multiple fluorophores is advantageous over sequential imaging because it minimizes FLIM acquisition times. Simultaneous FLIM of three endogenous fluorophores in addition to second harmonic generation (SHG) signals have been achieved by multiphoton wavelength mixing.⁶ Furthermore, one wavelength has been used to excite the two intrinsic fluorophores, NAD(P)H and FAD.¹⁵⁷ In addition, two single-photon wavelengths have been temporally interleaved to alternately excite NAD(P)H and FAD.¹⁵⁸

2.1.5.

Detectors

Detectors in LSM are often characterized by their sensitivity, reproducibility, quantum efficiency, photon-counting capability, narrow temporal responses, relatively fast transit time, low dark counts, and high SNR. Most LSM detectors are photomultiplier tubes (PMTs) that can be used in a photon-counting mode, which uses discriminators as described above. MCPs, avalanche photodiode, SPAD, hybrid PMTs, and SPAD arrays are also used for FLIM detection, each with merits and challenges. For example, SPAD arrays are capable of $256\times 256\text{pixels}$ including a TDC in each pixel, so an entire FLIM image can be acquired with $<100\text{picosecond}$ resolution.^159,160 However, SPAD arrays suffer from lower quantum efficiency at 460 nm ($<35\%$) and a low fill factor (additional microlens arrays can help to effectively guide light), which results in lower photon collection efficiency.¹⁶¹ A detailed discussion on current detectors is given by Bruschini et al.¹⁶⁰

Cameras with integrated FLIM capabilities have recently gained popularity. Chen et al. demonstrated wide-field FLIM using a frequency-domain CMOS FLIM camera,¹⁶² while Mitchell et al. implemented a frequency-domain CMOS FLIM camera in a lightsheet microscopy system.¹⁶³ Raspe et al. developed single-image fluorescence lifetime imaging microscopy (siFLIM) with a modulated electron-multiplied-charged couple device FLIM camera capable of simultaneously recording phase-shifted images.¹⁶⁴

3.1.1.

Curve fitting

Fluorescence lifetime measurements often capture multiple fluorophores within each pixel, resulting in multiexponential decays. In TCSPC, each photon is assigned to a time bin within a lifetime histogram.⁷⁴ Histograms are fit to multiexponential decay functions described in Eq. (1).⁷⁶ An IRF is measured from a sample with an instantaneous lifetime (e.g., SHG signal from a urea crystal for two-photon microscopy), which accounts for the temporal response of the optical system.⁷⁴ Curve fitting analysis requires some prior assumptions, including the number of lifetime components, temporal offset of detected signals, and sources of background fluorescence.⁷⁴ This method is also highly dependent on the number of photons detected per pixel as higher photon counts will improve the accuracy of the fit. The multicomponent exponential decay estimate is then convolved with the IRF and compared with the experimentally measured lifetime decay curve [Fig. 6(a)].⁷⁴ The chi-squared (${\chi }^2$) goodness-of-fit test is used to evaluate agreement between the fit and the measured data. Parameters of the model ($a_i$, ${\tau }_i$) are iterated to achieve a chi-squared value closest to 1, indicating the best model fit to the experimental data.⁷⁴

Fig. 6

Examples of fluorescence lifetime estimation methods. (a) Curve fitting analysis determines lifetime decay variables $({\alpha }_i,{\tau }_i)$ by fitting an estimated decay function and estimated or measured IRF to experimental data. This process is iterated with the measured data to optimize goodness-of-fit parameters (${\chi }^2$). (b) Fit-free methods for estimating lifetime parameters of time-domain or frequency-domain lifetime data also exist. The phasor approach is one such technique frequently used for intuitive representation of fit-free lifetime estimation. Here, measured lifetime data can be transformed into phasor space to visualize pixels with similar lifetime values [Eqs. (12)–(15)]. Universal circle shown as blue semicircle. Example phasors for single exponential species $({\tau }_1,{\tau }_2)$ and a two-component mixture of ${\tau }_1$ and ${\tau }_2$ illustrate the rule of linear addition. (c) Neural networks can be trained with simulated or experimental FLIM data for fast generation of fluorescence lifetime maps. Adapted under CC BY-4.0 with permission from Ref. 181.

These parameter estimates and fit quality measurements can be determined from analytical approaches, such as least squares fitting, maximum likelihood estimation, and Bayesian analysis.^165,168,172 These methods describe the likelihood of detecting specific photon counts within each time bin from the experimental decay, based on statistical assumptions unique to each method. For example, least squares fitting minimizes the squared difference between measured fluorescence and estimated signal and assumes Gaussian noise, whereas maximal likelihood methods assume Poisson-distributed noise.^165–167 Both approaches provide comparably accurate estimates of fluorescence decay parameters for lifetime histograms with high photon counts, though maximum likelihood analysis performs better for low photon counts.¹⁶⁵ Maximum likelihood analysis also allows for varied bin widths.^172,182

Global analysis is another approach to estimating fluorescence lifetimes from low SNR images.^168–170 One implementation of global analysis assumes that all fluorescent species are present within each pixel.¹⁷¹ This assumption improves estimation accuracy for data with low photon counts or high background signal. Lifetime parameters of the frequency-domain or transformed time-domain data are estimated by fitting for phase shift and modulation values at multiple frequencies, as shown in Fig. 4(d).^169,171 Alternatively, simultaneous analysis of per-pixel lifetimes assumes fixed lifetime values across all pixels and iterates lifetime parameter $({\tau }_i,{\alpha }_i)$ estimates to improve a whole-image goodness-of-fit measure.^183–185 This approach provides better parameter estimates by conserving the spatial information typically lost from averaging photon counts across pixels.¹⁶⁹ Lifetime estimates can also improve with segmentation prior to global fitting, which is important when shorter acquisition times are needed to capture fast dynamics.^170,186

Bayesian analysis has also been used to improve lifetime estimations. This method empirically determines both the prior distribution of the fluorescence decay (not limited to Gaussian distributions) and the likelihood function¹⁷³ to establish the posterior distribution of parameters.^172,187 Parameter estimates are iterated to maximize the posterior distribution and provide reliable lifetime estimates.¹⁸⁷ In general, this method yields optimal lifetime fits even with high noise and low total photon counts, but careful selection of the prior distribution is critical to ensuring accurate estimates.^173,187 Recent developments in Bayesian fluorescence lifetime estimation can bypass fits to the measured data and therefore bypass assumptions of the prior distribution of lifetime parameters, which can bias the lifetime estimates.^172,174,188

3.1.2.

Phasor analysis

Phasor analysis is a fit-free technique in which the fluorescence decay from each pixel is transformed into a point in two-dimensional (2-D) phasor space. Phasor representation provides a visual distribution of the molecular species in an image by clustering pixels with similar lifetimes [Fig. 6(b)]. Phasor analysis is instantaneous because it does not require an iterative fit procedure, and visualization in phasor space is especially advantageous for large FLIM datasets.^175,189 In phasor analysis, phasor distributions corresponding to similar lifetimes (decays) can be selected to locate the corresponding pixels in the image with similar lifetimes, even if they are spatially separated.¹⁷⁵

Phasor analysis can be applied to both time-domain and frequency-domain FLIM measurements. If $P(i,j)$ represents a pixel in the FLIM image with coordinates $(i,j)$ and $I_{i,j}(t)$ is the fluorescence intensity decay at that pixel, the corresponding coordinates in the phasor plot $(g,s)$ for time-domain measurements are given as¹⁹⁰

In the case of frequency-domain measurements, the coordinates are given as

Phasors follow the rule of linear addition. For example, the phasor location of a mixture of two species falls on a straight line joining the phasor location of the two individual species on the universal circle [Fig. 6(b)].^189,191 The position on this line is determined by the relative fractional contributions of each species. Similarly, the phasor distribution of a three-exponential species will fall in the triangle formed by the three individual phasor locations and similarly for higher order exponentials.^192,193 Hence, the phasor distribution of a heterogeneous sample will have a position inside the universal circle. This representation also provides a straightforward interpretation of the biological significance of lifetime values compared with other lifetime estimation analyses.¹⁰²

3.1.3.

Deconvolution analysis

Deconvolution methods recover the lifetime decay from the measured fluorescence signal by deconvolving the estimated optical system response. Deconvolution includes variations on the least squares approach discussed above, stretched exponential/lifetime moment analysis, and methods of transformation (e.g., Fourier and Laplace transforms).^{177,179,194–198} Stretched exponential and lifetime moment methods are fundamentally similar because the lifetimes of individual species are estimated from the total measured decay distribution.^177,179 Stretched exponential and lifetime moment methods have specific implementations for time-domain or frequency-domain data. Furthermore, Fourier and Laplace techniques transform the measured decay curve to be proportional to the product of the transformed source excitation pulse and system response.^196,197 The lifetime and contribution of each species are then recovered from the transformed system response.^{196,197,199,200} Deconvolution methods avoid assumptions about the instrument response but still share limitations associated with standard model fitting methods.

Laguerre deconvolution is an alternative to model fitting for fluorescence lifetime estimation. Here, the measured fluorescence decay is transformed and represented in the form of Laguerre polynomials, resulting in a series expansion of the decay and convolved IRF.^{176,180,195,201} This Laguerre transformation produces linearly independent functions that enable expansion of decays with low susceptibility to noise and proportionality between fluorescence intensity and lifetime decay.^176,180 The pixelwise linear combination of Laguerre coefficients provides the per-pixel fluorescence decay.^180,201 The Laguerre method is less accurate but more precise in lifetime estimates compared with the similarly fit-free phasor method, especially for values at either extreme.¹⁷⁶

3.1.4.

Machine learning analysis

Machine learning techniques are another alternative to time-intensive curve fitting procedures. A few key algorithms commonly used for this purpose are highlighted here. Simple neural networks can estimate fluorescence lifetimes directly from TCSPC data by learning weights from examples of curve fitted pixels.¹⁸¹ Variations of convolutional neural networks (CNNs) can also rapidly calculate fluorescence lifetimes. Briefly, CNNs downsample recorded images through kernel convolution and window pooling steps, resulting in a low-resolution image on which predictions of pixel class membership are made (i.e., contraction).²⁰² Pixel positions from the initial pooling steps are recalled to assign class predictions to pixels in upsampled images (i.e., expansion).²⁰² This computational structure has been used to analyze hyperspectral fluorescence lifetime images and to dynamically monitor fluorescence lifetimes in vitro and in vivo [Fig. 6(c)].^203,204 Neural networks continue to improve, including simultaneous prediction of fluorescence lifetimes and object segmentation masks, which will be discussed below.

3.2.1.

Pixel-level analysis

Pixel-level analysis of fluorescence lifetimes can inform on subcellular and cell-level heterogeneity within a sample. Lifetime histograms provide a useful quality check of curve fitting from TCSPC pixels, confirm the presence of distinct fluorescence lifetimes, and/or confirm expected changes in lifetime values from an experimental condition or FRET interaction.²⁰⁵ Distributions of pixels within phasor space provide complementary information on the identity of fluorophores in the sample and lifetime changes throughout an experiment.^175,205 Pixel-level FLIM analysis has been previously used to quantify lipid membrane integrity and heterogeneity, immune cell heterogeneity, cell development, protein conformation and organization, and other phenomena.^206–211

3.2.2.

Object-level analysis

Object-level analysis provides a biological context for interpreting FLIM images by averaging lifetime values across all pixels within a single object of interest (e.g., cells, organelles, and bacteria).³² This approach quantifies diversity across cells, organelles, and other features in a similar manner to established techniques such as flow cytometry or colony counting. Segmentation is required for object-level analysis. Automatic segmentation can be achieved with computational approaches such as multiresolution community detection and morphological filtering and thresholding.^212–214 In addition, unsupervised clustering techniques (e.g., K-means clustering) can segment single cells and intracellular compartments for phasor-based lifetime data.²¹⁵ Open source packages such as FLIMfit and FLIM-FRET analyzer have been developed for multiple functionalities, including automatic segmentation, lifetime decay fitting, and data visualization.^216,217

Machine learning is also popular for image segmentation due to its high accuracy and generalization across imaging formats. Neural networks can learn features of pixels within objects to generate segmentation masks.²⁰² Several architectures have been designed to improve segmentation performance, primarily for intensity-based segmentation of cellular compartments. Variations of CNNs have been developed for object segmentation. These variations include UNets, feature pyramid networks, and Mask-RCNN.²⁰² UNet employs the standard CNN framework, described earlier, but maintains symmetrical contraction and expansion branches, and concatenates contracted layers with expansion layers to better preserve the initial structure of the image.²¹⁸ Feature pyramid networks follow this scheme, but they sum convolutional layers from contraction and expansion to inform image upsampling.²¹⁹ Similarly, mask-RCNN uses feature pyramid networks to generate feature maps for regions of interest (ROIs) before convolutional steps that classify pixels within an ROI for object masks.²²⁰ Modified versions of these algorithms have been used to segment numerous cell types and intracellular features across imaging platforms,²⁰² and a combination of these algorithms has enabled nuclei segmentation across image types [e.g., hematoxylin and eosin (H&E) and immunofluorescence].²²¹ This is not an exhaustive list of machine learning approaches for fluorescence image segmentation, but it provides an overview of some object-level segmentation and classification tools.

Segmentation of autofluorescence images is challenging due to low SNR and poor spatial specificity of the fluorescence signal. CellProfiler has been used to isolate cells, nuclei, and cytoplasms from two-photon NAD(P)H autofluorescence images.²²² This approach identifies nuclei within a specified size range by thresholding pixels above the background fluorescence but below cytoplasmic fluorescence values. Whole cell masks are defined by propagating outward from nuclear masks. Nuclear masks are subtracted from cell masks to isolate cytoplasmic areas. Similarly, a collection of ImageJ plugins have been developed for integrated lifetime decay fitting (SLIM curve) and object segmentation (Trainable Weka Segmentation).²²³ This approach first requires input of a small subset of fluorescence images in which the user annotates the objects of interest. A number of ImageJ-based filters and transformations are applied to the images to extract features specific to annotated objects.²²⁴ A suite of machine learning algorithms (Weka) are then applied to annotated inputs and extracted features to classify pixels in unlabeled images.²²⁵ CellProfiler has developed a similar annotation-based method, Ilastik, optimized for fluorescent proteins and dyes but applicable to autofluorescence images.²²⁶

Histograms of lifetime values can be plotted across numerous objects for population distribution analysis. This approach visualizes heterogeneity within an object class (e.g., cells or mitochondria) under basal conditions or in response to perturbations. Histograms can be fit with population density models to summarize the distribution of objects and to identify distinct subpopulations of objects. Gaussian mixture modeling is a common population density modeling approach in which multiple Gaussian probability density functions are iteratively fit to each frequency histogram [Fig. 7(a)].^227,230 Goodness of fit is assessed by the minimum Akaike information criterion.^230,231 However, this approach is limited by assumptions about the number of populations within the data and the Gaussian distribution of the data.^232,233 Approaches have been developed to circumvent these assumptions, including density-based clustering [Fig. 8(a)].²³⁵ Density-based clustering defines subpopulations within data such that the highest density datapoints define the cluster for the nearest remaining points.²³⁵ Previous studies have shown that population distribution analysis of autofluorescence lifetimes can classify cell types, drug response, and disease states [Fig. 7(b)].^{32,227–229,236–238} In addition, lifetime distributions can identify objects without prior segmentation. Variations in lifetime distributions have identified other molecular features such as tagged neurons in C. elegans and metabolic activities within tumors.^239,240 Overall, population distribution analysis provides unique insights into sample heterogeneity.

Fig. 7

Heterogeneity analysis of fluorescence lifetime data. (a) Histograms of lifetimes per object are fit to distribution models to describe subpopulations and variability in the data. The $H$-index and $wH$-index are derived from these fits [Eqs. (19) and (20), respectively]. Here, $p_i$ is the proportion of each subpopulation, $d_i$ is the distance between subpopulation median and global median, and ${\sigma }_i$ is the subpopulation standard deviation. (b) Distribution density models fit to cell-level NAD(P)H mean lifetimes can accurately identify distinct breast cancer cell lines (MDA-MB-231 and SKBr3) in mixed cocultures (proportion of mixtures indicated above plots). Errors (Err) in the model predictions for mean ($\overline{x}$) and proportion ($p$) of each population are given within each plot. Adapted with permission from Ref. 227. (c) $H$-index of in vivo FaDu tumor cell NAD(P)H mean lifetime (right) correlates with in vivo treatment response (left). Adapted with permission from Ref. 228. (d) Cell autofluorescence $wH$-index is similar for (left) in vitro organoids derived from primary PyVmT tumors and (right) in vivo PyVmT tumors with vehicle and combination treatment. Adapted with permission from Ref. 229.

Fig. 8

Spatial analysis of fluorescence lifetime distribution. Spatial statistical analyses can quantify spatial heterogeneity in fluorescence lifetimes. Here, spatial heterogeneity in cell-level autofluorescence lifetimes is used as an example. (a) Density-based clustering defines cell subpopulations that are mapped back onto lifetime images. Relative proximity measurements define spatial distributions within (intrapopulation proximity) and between (interpopulation proximity) cell subpopulations. (b) Multivariate spatial heterogeneity is quantified with spatial autocorrelation and spatial principal components analysis. Adapted under CC BY-4.0 with permission from Ref. 234.

Heterogeneity in lifetime measurements is commonly quantified from coefficients of variation (CV).^{179,241–245} The CV is the standard deviation divided by the mean of a measurement, which enables comparisons of variability between samples. However, the CV does not define whether distinct subpopulations exist within the lifetime data. Alternatively, quantitative metrics of heterogeneity can be derived from population density models, so the behavior of subpopulations can be compared between conditions. A heterogeneity index ($H$-index) was previously defined to quantify cell subpopulations of fluorescence lifetimes using population density models of cells in vivo in head and neck cancer.²²⁸ This $H$-index is based on the Shannon diversity index, widely used in ecological studies, and is defined as

These heterogeneity metrics have provided valuable insight into diversity within biological systems. The $H$-index of cell autofluorescence in vivo in tumors shows more homogeneous activity across cells (lower $H$-index) for treatments that significantly reduce tumor volume (combination treatment), and more heterogeneous activity across cells (higher $H$-index) for treatments that do not change tumor volume (cetuximab or cisplatin alone) [Fig. 7(c)]. In addition, $wH$-index values of cell autofluorescence for control and treated conditions are similar between in vitro tumor organoids and in vivo tumors, indicating similar treatment-induced changes in metabolic heterogeneity in vivo and in vitro for the same tumor model²²⁹ [Fig. 7(d)]. Collectively, these studies show that quantitative metrics of fluorescence lifetime heterogeneity provide powerful tools to study diversity in biological systems.

4.1.1.

In vivo autofluorescence FLIM

Numerous sources of molecular contrast make FLIM attractive for in vivo imaging. One of the earliest in vivo FLIM studies was performed with intrinsic sources of contrast in human skin. Koenig et al. investigated changes in autofluorescence and SHG that occur with human skin disease in vivo.³³ SHG is a frequency doubling process with instantaneous lifetime (for a review of SHG, see Ref. 261). Collagen fibers create strong SHG signals in skin. Koenig et al. showed that FLIM resolves changes in the autofluorescence lifetime of skin cells within different layers of the skin. Furthermore, FLIM detected cells that had become diseased due to melanoma or fungal infections.³³ This study pioneered the use of FLIM for clinical applications.

Similarly, the first in vivo FLIM studies in animal models also focused on autofluorescence. Skala et al. identified differences in the autofluorescence lifetime between normal, low-grade, and high-grade precancerous epithelia in the hamster cheek pouch in vivo.^241,262 Later, in vivo FLIM was used to predict treatment response in mouse tumor models. Specifically, NAD(P)H lifetime changes were found to directly correlate to standard tumor response measurements (i.e., tumor volume).²²⁸ Importantly, FLIM detected treatment-induced changes in tumors in vivo only 2 days post-treatment, which is earlier than detectable changes in tumor volume [6 days post-treatment, Fig. 7(c)]. A recent study also demonstrated that in vivo FLIM can measure the efficacy of chemotherapy agents in a mouse model of colorectal cancer.²⁶³ Furthermore, autofluorescence FLIM can capture metabolic features of specific cell types in vivo. Work by Szulczewski et al. indicated that macrophages have a fluorescence lifetime that differs from mammary tumor cells such that macrophages can be identified and monitored in vivo without labels²⁶⁴ [Fig. 9(a)]. Other in vivo applications of autofluorescence FLIM focus on metabolism in the mouse brain. For example, NAD(P)H lifetimes reveal metabolic preferences in the brain using a well-defined set of inhibitors that target-specific metabolic reactions^7,266 [Fig. 9(b)].

Fig. 9

Autofluorescence FLIM applications. (a) NAD(P)H FLIM of a mammary mouse tumor (heatmap) overlaid on an SHG image of collagen (grayscale). $\text{Scale bar}=100\mu \mathrm{m}$. Adapted with permission from Ref. 264 (b) Mean NAD(P)H lifetimes in solution and in the rat cortex in vivo after metabolic inhibition. [2DG, 2-deoxy-d-glucose; IAA, iodoacetic acid; KCNm, potassium cyanide; FCCP, carbonyl cyanide-4-(trifluoromethoxy)phenylhydrazone; BMI, bicuculline methiodide; ETC, electron transport chain.] * indicates significantly different from baseline in vivo measurement; Error bars indicate standard error across all pixels over all measurements. Reproduced with permission from Ref. 7 (c) Optical redox ratio [NAD(P)H/FAD; first row], NAD(P)H ${\tau }_m$ (second row), and FAD ${\tau }_m$ (third row) images of organoids generated from primary human breast tumors obtained from resection surgeries. TNBC, triple negative breast cancer; ER, estrogen receptor. $\text{Scale bar}=100\mu \mathrm{m}$. Adapted with permission from Ref. 265.

In addition, autofluorescence FLIM has been performed in numerous non-mammalian in vivo models to study organs and whole-body processes that are not easily visualized in mammals. For example, the metabolic gradient along the germline of C. elegans was visualized with autofluorescence FLIM,⁶ which provided new insights into metabolic changes with germline differentiation. FLIM has also been performed in plants such as Arabidopsis, where FLIM estimated vacuolar pH inside intact plant cells with the lifetime of anthocyanin.²⁶⁷

4.1.2.

Three-dimensional in vitro autofluorescence FLIM

3-D in vitro cultures, including organoids and cell constructs within microdevices, have also been assessed with FLIM. Optical sectioning techniques such as CLSM and MP-LSM are especially attractive for FLIM of 3-D cultures due to their high spatial resolution and volumetric imaging capabilities. Numerous cancer studies have focused on predicting in vivo drug response using primary tumor organoids. These organoids retain all of the cells of the original tumor in a 3-D matrix so that in vivo cell–cell interactions and relevant gradients of oxygen, nutrients, and drugs are preserved.^268–271 For example, MP-LSM FLIM indicates that autofluorescence lifetimes in primary tumor organoids can predict in vivo response in mouse models across a range of treatment conditions in breast²⁶⁵ and head and neck cancer.²⁷² Furthermore, FLIM can detect differences in the metabolism of primary patient-derived tumor organoids based upon their surface marker expression [Fig. 9(c)]. FLIM has been used to investigate treatment response in patient-derived tumor organoids across multiple cancer types including breast,²⁶⁵ pancreatic,²⁷³ and colorectal cancer.²⁷⁴ In addition, FLIM of colorectal cancer organoids was used to inform a patient treatment regimen.²⁷⁴ Organoids provide important 3-D architecture for in vitro studies, but microdevices improve the relevance of 3-D cultures by mimicking in vivo structures. Specifically, FLIM monitored changes in the metabolism of ductal carcinoma in situ cells during invasion in a lumen microdevice model. FLIM captured changes in metabolism based on the position of a cell within the lumen or invading branch.²⁷⁵

Tissues ex vivo have also been imaged to determine whether FLIM can guide surgical resection of tumors.²⁷⁶ First, Lukina et al. compared NAD(P)H FLIM of in vivo and ex vivo samples using mouse models of colorectal cancer, lung carcinoma, and melanoma to determine optimal tissue maintenance protocols to preserve in vivo signals within ex vivo samples. Then, Lukina et al. used these protocols to perform NAD(P)H FLIM in postoperative samples obtained from colorectal cancer patients and found significant differences in NAD(P)H lifetimes between normal and malignant specimens.

4.1.3.

Autofluorescence FLIM in two-dimensional samples

Autofluorescence FLIM of 2-D in vitro cultures provides simple and repeatable systems to test perturbations of autofluorescence lifetime properties. For example, Walsh et al. showed that NAD(P)H lifetimes can detect metabolic differences due to breast cancer subtype.²³⁶ In addition, the fluorescence lifetime of NAD(P)H correlates with the differentiation potential of neural progenitor and stem cells.¹⁹⁰ Similarly, changes in the relative fluorescence lifetimes of NAD(P)H and lipid droplet associated granules discriminated differentiated and undifferentiated human embryonic stem cells,²⁷⁷ as well as human induced pluripotent stem cell-derived cardiomyocytes under oxidative stress.²⁷⁸ Further autofluorescence FLIM studies in 2-D culture discriminated activation states in multiple types of immune cells including macrophages²⁰⁹ and T cells.²⁵⁹

Finally, autofluorescence FLIM can resolve subcellular features to study intracellular dynamics, including communication between organelles, subcellular features of whole cell processes such as cell division, and bioenergetic demands of different cell types. Mitochondrial organization is often altered to accommodate cellular bioenergetics and biosynthetic demands. Changes in metabolism are also a hallmark of many diseases including cancer. Therefore, mitochondrial imaging has been especially popular for subcellular FLIM applications. Fluorescent dyes such as TMRE can measure mitochondrial membrane potential, which is closely related to cell health.²⁷⁹ However, mitochondrial dyes can alter cellular respiration,²⁸⁰ and therefore label-free methods are in development. NAD(P)H and FAD fluorescence signals are brightest in the mitochondria, which enables label-free visualization of mitochondria. Pouli et al. showed that FLIM of NAD(P)H and FAD can capture rapid changes in mitochondrial spatial dynamics and metabolism using high-resolution imaging of individual mitochondria within cells.²⁸¹

4.2.1.

Exogenous molecular probes for in vivo applications

Numerous optical probes have been developed for both in vivo and in vitro applications to capitalize on the sensitivity of FLIM to physical conditions, including viscosity,²⁸² temperature,²⁸³ acidity,²⁸⁴ and oxygenation.^104,247,285 Additional molecular probes have been generated that allow for FLIM-based monitoring of drug delivery.

Mouse models are widely used for in vivo FLIM studies of exogenous molecular probes. Ardeshirpour et al. detected mouse tumors in vivo that express human epidermal growth factor receptor (HER2) with FLIM of a fluorescent anti-HER2 antibody.²⁸⁶ Similarly, FLIM showed that the near-infrared fluorescence dye cypate localizes to mouse tumors in vivo [Fig. 10(a)]. FLIM of two fluorophores, cypate and bacteriochlorophyll, can identify the unique distribution of each fluorophore in vivo,^239,287,290 which is difficult with intensity-based imaging alone. FLIM has also evaluated renal function in mice using the fluorescence lifetime reporter LS-288, which has a distinct lifetime when free in solution vs. bound to proteins. This approach provides contrast between the protein-rich viscera and the mostly protein-free bladder in mice in vivo [Fig. 10(b)].²⁸⁸ Furthermore, pH-sensitive fluorescence lifetime probes that provide a nonterminal method to quickly determine the acidity of a region in vivo have been developed.²⁹¹ Overall, FLIM in conjunction with the development of these sophisticated probes is promising in cancer detection and other in vivo applications.

Fig. 10

Molecular probes for FLIM and FLIM-FRET. (a) Near-infrared fluorescence lifetime image of Cyp-GRD distribution (heatmap) in an A549-tumor-bearing mouse at 24-h postinjection. Adapted with permission from Ref. 287 (b) Fluorescence lifetime (heatmap) of mouse abdomen acquired 90 min after intravenous injection of LS-288. The low fluorescence lifetime region in the center of the abdomen is the filled urinary bladder. Adapted with permission from Ref. 288. (c) FLIM maps of the weighted mean fluorescence lifetime of T2AMPKAR-NES, a sensor for AMPK activation, in HEK293 spheroids. The blue end of the colormap indicates increased AMPK activation. $\text{Scale bar}=100\mu \mathrm{m}$. Adapted with permission from Ref. 289.

In non-mammalian in vivo models, fluorescence lifetime probes that change with both temperature^292,293 and concentration of ions have been developed. For example, Zhang et al. generated a phosphorescent lifetime probe that is temperature dependent and demonstrated this temperature dependence in vivo in a zebrafish model.²⁸³ Another example of a non-mammalian application of fluorescence lifetime probes includes imaging chloride ion concentrations in cockroach salivary glands done by Hille et al.²⁹⁴

4.2.2.

In vitro molecular probe FLIM

Many fluorescence lifetime probes exist for in vitro applications to measure whole cell changes and localize molecular trafficking within a cell. For example, a fluorescence lifetime probe was developed to track the location and use of ${\mathrm{Zn}}^{2+}$ within a cell. These probes can be localized to understand ion use within individual organelles. Other fluorescence lifetime probes have been developed to detect intracellular prodrug trafficking,²⁹⁵ as well as pH²⁹⁶ and oxygenation changes. Oxygen sensing via phosphorescent lifetime imaging has become a well-established method to monitor intracellular oxygen tension. Furthermore, simultaneous measurement of NAD(P)H FLIM and oxygen sensing by phosphorescence lifetime imaging of Ruthenium tris-(2,2′-bipyridyl) has also been demonstrated in 2-D cell cultures.²⁹⁷

4.3.1.

FLIM-FRET for in vivo applications

Finally, FLIM can be used to better capture extracellular and subcellular interactions on the nanoscale both in vivo and in vitro via FLIM-FRET. FLIM-FRET interactions can be used to measure protein activity, gene regulation, and subcellular dynamics. For example, an activatable FRET probe has been developed with a donor–acceptor pair that can be cleaved by matrix metalloproteinases (MMP). This probe was used in a mouse model of breast cancer to monitor MMP activity.²⁹⁸ FLIM-FRET has also identified patterns in RhoA activity in vivo using a GFP-RFP Raichu-RhoA reporter. These studies found that active RhoA, which is associated with cellular cytoskeleton organization, has subcellular localization to the leading edge of invasion in a pancreatic cancer mouse model.²⁹⁹

FLIM-FRET probes have also been used in non-mammalian in vivo models including plants and zebrafish. In Arabidopsis roots, FLIM-FRET probes have been developed to investigate the role of transcription factors that regulate plant cell fates.³⁰⁰ In zebrafish, FLIM imaged a time-course of apoptosis after radiation treatment in 3-D over the entire zebrafish body using a FRET sensor,³⁰¹ which provided an important whole-body context for the apoptotic process. These are just a few of the many non-mammalian in vivo applications of FLIM-FRET.

4.3.2.

FLIM-FRET for in vitro applications

Subcellular dynamics can also be monitored with FLIM-FRET in vitro. T2AMPKAR-NES is a FLIM-FRET sensor for AMPK activation that was shown to detect spatial changes in the activity of AMPK between the inner and outer layers of human embryonic kidney spheroids in a 3-D culture [Fig. 10(c)].²⁸⁹ FLIM-FRET has also been used in a 2-D culture of mouse pituitary cells to detect dimerization between the transcription factor CAATT and the enhancer binding protein alpha. This dimerization corresponds with increased gene expression and adipogenesis.³⁰² Finally, autofluorescence FLIM-FRET can detect molecular interactions within live cells as well, specifically between NAD(P)H and tryptophan. In these studies, tryptophan is the FRET donor and NAD(P)H is the FRET acceptor [i.e., NAD(P)H quenches tryptophan fluorescence]. These studies found that doxorubicin increases the abundance of FRET interactions between tryptophan and NAD(P)H.³⁰³ New developments in super-resolution FLIM can localize molecular features within smaller structures and is growing in popularity along with other super-resolution techniques.^133,304