1.

Introduction

Fluorescence imaging can provide useful contrast based on a number of measurable parameters (e.g., emission wavelength or spectrum, quantum efficiency, lifetime¹) for many fields including, e.g., medical imaging² and molecular biology.³ When applied to endogenous tissue fluorescence, such contrast may furnish new capabilities for histopathology and has been proposed as a potentially useful clinical tool to identify abnormal or diseased areas of tissue by reporting differences in the composition and distribution of tissue fluorophores. These include the extracellular structural proteins collagen and elastin and the components of intracellular metabolism such as NADH and oxidized flavins. A particularly important application is the identification of lesions that are not readily apparent under observation with white light, such as areas of early cancer or dysplasia.

To date, most clinical use of fluorescence imaging has concentrated on detecting differences in the spectrally resolved fluorescence between abnormal and normal tissue. Changes in fluorescence spectra have been described between normal and dysplastic or cancerous tissue both ex vivo and in vivo in a number of optically accessible organs including the GI tract,^{4, 5} cervix,^{6, 7} bronchial tree,⁸ mouth,⁹ and skin.¹⁰ Such findings are very encouraging as to the utility of fluorescence imaging in the clinical setting and “fluorescence endoscopes,” which detect changes in spectrally resolved fluorescence are now commercially available. However, while spectrally resolved fluorescence imaging has a reasonable sensitivity for the detection of early cancers, it tends to have low specificity and a high rate of false positive findings.¹¹ One difficulty encountered with spectrally resolved fluorescence tissue imaging is the observation that the emission spectra of many tissue fluorophores overlap,¹² which may limit the obtainable contrast. In this regard, the use of temporally resolved fluorescence imaging, where the measured parameter is the rate at which the intensity of the fluorescence decays after excitation, is particularly exciting as many intrinsic tissue fluorophores can be distinguished by their fluorescence lifetime.

Fluorescence lifetime imaging (FLIM) offers the potential for improved fluorophore specificity and, being a ratiometric technique, is inherently robust in the presence of absorption and/or scattering. FLIM can also report on the local fluorophore environment via changes in the radiative or nonradiative decay constants (e.g., Ref. 13). This potential sensitivity to both the function and composition of tissue offers the prospect of contrasting normal and diseased tissue using fluorescence lifetime—even when the observed changes to fluorescence lifetime are not yet well understood. Differences in the fluorescence lifetimes between normal and neoplastic tissue have been demonstrated in the colon,¹⁴ breast tissue,¹⁵ and brain.¹⁶ We previously demonstrated the ability of FLIM to generate contrast between different areas of ex vivo formalin-fixed unstained biological tissue.¹⁷ Recently, we have observed promising intrinsic FLIM contrast (both microscopic and macroscopic) in ex vivo frozen and fresh tissue. For example, Fig. 1 shows the contrast obtained from a frozen $10\text{-}\mu \mathrm{m}$ section of colonic polyp and Fig. 2 shows the contrast obtained with a fresh $200\text{-}\mu \mathrm{m}$ -thick section of mouse spleen. These FLIM images were obtained using a scanning confocal two-photon microscope with a time-correlated single-photon-counting (TCSPC) FLIM system (e.g., Ref. 18), the FLIM system we typically use to study tissue sections. For this system, the total image acquisition times are typically several minutes (achieving photon counts of $>{10}^4$ for the brightest pixels) with frozen sections requiring longer acquisition times compared to fresh tissue. The main source of noise in such TCSPC FLIM images is shot noise.

Fig. 1

(a) Fluorescence intensity and (b) lifetime images of a $10\text{-}\mu \mathrm{m}$ unstained frozen section through a colonic polyp.

Fig. 2

(a) Fluorescence intensity and (b) lifetime images of a fresh $200\text{-}\mu \mathrm{m}$ section of mouse spleen.

Note that the clear contrast observed in these FLIM images is obtained by fitting a monoexponential decay model, although it is well known that biological tissue typically exhibits more complex fluorescence decay profiles. In principle, “better” fits may be obtained by applying multiple exponential decay models—although the improvement in “goodness of fit” often results from the increased number of fitting parameters rather than the actual presence of multiple discrete autofluorescence lifetime components in tissue samples, and the fitting process itself is very sensitive to artifacts associated with the instrument response function and background signals. Furthermore, one requires much higher SNR data for accurate fitting of multiple complex decay components (i.e., one must collect many more detected photons, which entails increased exposure times), and the data processing time to calculate the FLIM images is significantly increased.¹⁹ We demonstrated earlier that in the absence of a priori knowledge of discrete lifetime components, the more realistic assumption of a continuous distribution of autofluorescence lifetimes in a tissue sample provides more robust fitting using the stretched exponential function²⁰ with improved contrast in FLIM images and a much faster data processing rate compared to a double-exponential decay model. Nevertheless, such models also require significantly more collected photons per FLIM image than fitting a single exponential decay and, although the resulting FLIM images may be more “accurate,” it is not clear that they are always significantly more “useful,” particularly in a clinical context where imaging speed is likely to be at a premium. In fact, the confocal-scanning TCSPC FLIM system used to acquire the images shown in Figs. 1 and 2 is too slow for many diagnostic applications due to the way in which pixels are acquired sequentially. Many clinical instruments will require the acquisition and display of fluorescence lifetime data at a frame rate approaching video rate both for ease of use and because of the inherent difficulties of in vivo imaging, where the target may well be moving. Motion artifacts due to relative motion of the target during acquisition can seriously degrade lifetime measurements. Wide-field imaging, for which all the pixels are acquired in parallel, has the potential to greatly reduce data acquisition times as well as being inherently less complex and so well suited for endoscopic application. In principle, if wide-field FLIM technology can be developed with appropriate imaging speed and sensitivity, it should be possible to apply it to clinical modalities such as gastrointestinal endoscopy (e.g., Barrett’s esophagus surveillance, colonic screening), colposcopy (e.g., cervical screening), bronchoscopy, and cystoscopy.

Until now there have been only a few reports of high-speed wide-field FLIM and these have usually been restricted to a reduced number of image pixels. A wide-field frequency-domain FLIM instrument capable of determining fluorescence lifetime images at $25\mathrm{Hz}$ (and resolving lifetime differences at up to $55\mathrm{Hz}$ when pixel-binning down to $164\times 123$ ) has been reported,²¹ although not applied to endogenous tissue fluorescence. Also, a frequency-domain endoscopic FLIM instrument has been reported²² that achieved video-rate FLIM of biological tissue for a limited spatial resolution of $32\times 32\text{pixels}$ . In the time domain, wide-field FLIM of a calcium sensing fluorophore at up to $100\mathrm{Hz}$ with $60\times 120\text{pixels}$ has been realized using a single-shot approach incorporating an optical delay line that provided two time-delayed images at a gated optical intensifier²³ (GOI). We recently reported a real-time full-field $(344\times 256\text{pixels})$ time-domain FLIM apparatus applied to FLIM endoscopy of the endogenous fluorescence of biological tissue.²⁴ FLIM update rates of up to $29\text{frames}∕\mathrm{s}\left(\mathrm{fps}\right)$ were achieved by using a rapidly switchable delay generator with a gated optical image intensifier and by implementing an analytic rapid lifetime determination (RLD) algorithm.^{25, 26}

In this paper, we present results from the most recent developments of this endoscopic FLIM system and report the extension to a setup where the excitation light is fiber delivered via the biopsy channel of a standard clinical endoscope. This represents a vital step toward a practical clinical instrument.

2.

Wide-Field Time-Domain FLIM

The instrumentation used in wide-field time-domain FLIM is described in detail elsewhere.²⁷ Briefly, the fluorescent sample is excited with a very short pulse from a laser source and a GOI coupled to a CCD camera is used to record the intensity of the fluorescence at some chosen time after the excitation pulse. The GOI functions like a normal light intensifier in that each photon incident on the input side from the sample under test results in one or more photons, according to the gain selected, being output and then detected by the CCD. However it is possible to effectively switch the intensifier on and off at very high speed (minimum gate widths of around $\sim 200\mathrm{ps}$ are achievable) to form a gate where photons are only emitted in response to incident photons arriving after a time $t$ from the excitation pulse and before a time $t+Wt$ , where $Wt$ is the gate width.

By varying the delay between the excitation and the opening of the gate a number of “snap-shots” of the exponentially decreasing fluorescence can be recorded. These subimages can be acquired either sequentially or simultaneously (single shot). For most wide-field FLIM systems, the lifetime maps are calculated from subimages that are acquired sequentially in time. In single-shot FLIM systems,^{25, 28} the subimages are separated spatially using a beamsplitter(s) and acquired simultaneously in a single time-gated image acquisition. This has the considerable advantage of being immune to lifetime artifacts caused by motion of the sample at the cost of some experimental and computational complexity to allow for subimage differences caused by having separate optical paths. The overall acquisition time is similar for both cases, being determined by the required photon count (FLIM of tissue autofluorescence can typically achieve a frame rate of up to $10\mathrm{Hz}$ ). For each pixel in the series of time-gated subimages, a lifetime value can be computed using an iterative fitting algorithm like weighted nonlinear least-squares (WNLLS). This approach is flexible, since it can be applied to any model of decay, including the stretched exponential function (e.g., Ref. 20), but it is intrinsically slow due to its iterative nature and the number of subimages necessary to get accurate fitted values.

An alternative approach can be taken where an analytic solution is available for a particular decay model, known as RLD (e.g., Ref. 29).

The simplest decay model is a monoexponential with no offset of the form

The RLD approach has advantages for high-speed FLIM since the acquisition time is shorter (as only the minimum number of subimages are required) and it is noniterative.

As discussed, we note that even though intrinsic fluorescence in tissue tends to show a complex time-dependent emission, a monoexponential decay model may prove useful to observe contrast between different tissue states and/or environments in a real-time diagnostic tool, although it is less appropriate for obtaining absolute lifetime measurements. Note also that the RLD approach (of which a two-gate form is shown in Fig. 3 ) can also be extended to three or more gates, potentially enabling more accurate fitting of fluorescence lifetimes for monoexponential³⁰ and more complicated decay models.³¹

Fig. 3

Two-gate RLD.

3.

Real-Time FLIM Endoscopy

A range of different types of endoscopes are used for different medical applications. For our purposes, these can be broadly categorized into those where the light is transmitted via a series of lenses and those where this is done via optical fibers. The latter type presents some additional challenges. They are less light efficient due to transmission and coupling losses and the fibers will cause dispersion of both the excitation pulse and the imaged fluorescence. We are currently working to quantify the effect this will have on the performance of the system, although it is not expected to be a limiting factor and our preliminary results support this conclusion.

A high-speed FLIM apparatus based on sequential wide-field time-domain FLIM technology that makes use of a rapidly switchable delay generator (Kentech Instruments Ltd.) that can change the electronic temporal delay to the gated imager in only $\sim 2\mathrm{ms}$ has been implemented. This system uses the RLD FLIM approach discussed in the previous section and illustrated in Fig. 3, and has been successfully applied to real-time endoscopic FLIM in two different configurations from the two categories described: rigid and flexible. In each case, we use time gates of typically several nanoseconds in duration, which are relatively long as a proportion of the interpulse duration $\left(12\mathrm{ns}\right)$ but that have steep $(<100\text{-}\mathrm{ps})$ rise and fall times, to maximize the SNR by capturing as much light as possible—as early as possible after the excitation pulse. This choice of long, steep rectangular gates thus improves the photon efficiency but does not degrade the (monoexponential) lifetime calculations.

The excitation source we use for endoscopic FLIM is a frequency tripled $\mathrm{Nd}:\mathrm{Y}\mathrm{V}{\mathrm{O}}_4$ laser (Spectra Physics Inc., Vanguard 350-HMD355), which produces $10\text{-}\mathrm{ps}$ pulses at a repetition rate of $80\mathrm{MHz}$ and a wavelength of $355\mathrm{nm}$ with up to $\sim 350\mathrm{mW}$ average power. This wavelength has been shown to excite a large number of the components of tissue.³² In addition, this ultrafast UV laser is compact and does not require water cooling. The leakage light through one of the dielectric mirrors is detected by a photodiode (Thorlabs, DET210), and is utilized as a trigger signal for the time-gating electronics. A half waveplate (CVI Laser, QWPO-355-05-2-R10) and polarizing beamsplitter cube (CVI Laser, PBSO-355-050) are used to control the laser power. The output from this laser can be sent directly to the sample for test experiments or can be coupled into an optical fiber for delivery, as required.

3.1.

Rigid Lens Guide Endoscope

For this initial experiment, the sample was illuminated directly, using a holographic diffuser (Physical Optics Corporation, LSD60UVTAC30-2) just prior to the sample to produce a cone of illumination with a full angle of $\sim 60\mathrm{deg}$ . In a practical purpose-built device, a separate lens guide excitation path could be engineered into the endoscope to deliver the excitation. The fluorescence emission was transmitted through the rigid endoscope (a commercial Smith & nephew plc, Model 3894 arthroscope) and passed through a $375\text{-}\mathrm{nm}$ -long pass filter (Schott GG375) to block any reflected/scattered excitation light. An image is then formed on the photocathode of the GOI using an $f=100\mathrm{mm}$ lens. Using this system with an ex vivo sample of lamb’s kidney, it was possible to acquire and compute an FLIM image of the tissue autofluorescence in real time with an update rate of $7.2\mathrm{Hz}$ . Figure 4(c) shows one typical fluorescence lifetime map of this fresh kidney tissue from a live sequence. For comparison, a conventional wide-field FLIM image of the same tissue acquired over several seconds and fitted using a WNLLS iterative algorithm is also shown. Clearly the “live” image is noisier but the contrast in fluorescence lifetimes, particularly between the calyces and the medulla is still clear. In fact, showing one image like this exaggerates the problem of noise as, during use, the update rate is high enough that the operators’ persistence of vision tends to perform its own integration of the maps.

Fig. 4

(a) Arthroscope used to obtain endoscopic FLIM images of bisected lamb's kidney using (b) conventional WNLLS at $0.1\mathrm{Hz}$ and (c) real-time $\left(7.2\text{-}\mathrm{Hz}\right)$ RLD FLIM acquisition and processing.

3.2.

Flexible $\mathit{\left(}\mathit{10}\text{-}\mathit{mm}\mathit{\right)}$ Fiber Bundle Endoscope

We recently combined the FLIM system just described with the flexible endoscope shown in Fig. 5(a) to make a “proof of concept” system capable of in vivo imaging. Even though the coupling and transmission losses of the fiber imaging bundle are significantly higher than for the rigid endoscope, this system is still capable of producing lifetime images of the endogenous fluorescence of fresh tissue at several hertz.

Fig. 5

(a) The $10\text{-}\mathrm{mm}$ -diameter flexible endoscope with (b) fluorescence intensity and (c) FLIM images of the mucosal surface of piece of freshly resected normal human stomach viewed enface with $355\text{-}\mathrm{nm}$ excitation.

Fig. 6

Setup for flexible endoscope.

For this system the excitation light was coupled into the lower order modes of a $7\text{-}\mathrm{m}$ -long multimode fiber (Thorlabs, FT-200-UMT) using an aspheric lens (Thorlabs, C280TM-A). This fiber was then introduced through the biopsy port of a flexible endoscope (Endoscan Ltd.), as shown in Fig. 5(a). The fiber was chosen to minimize $355\text{-}\mathrm{nm}$ excited fluorescence in the fiber itself. The excitation light passes through a light shaping diffuser (Physical Optics Corporation, LSD60UVTAC30-2) at the distal end of the endoscope, producing a cone of illumination with a full angle of $\sim 60\mathrm{deg}$ , which is larger than the imaging angle of the endoscope and completely fills the field of view. The power at the sample was approximately $15\mathrm{mW}∕{\mathrm{cm}}^2$ , which had no discernible effect on the tissue. The imaging channel of the flexible endoscope comprises a graded-index objective (NSG, ILW 1.00) with a full viewing angle of $50\mathrm{deg}$ , cemented to a flexible coherent imaging bundle (Sumitomo Electric, IGN-08/30) with $\sim 30000$ fibers. The output from the endoscope is imaged directly onto the photocathode of the GOI.

We initially applied this flexible FLIM endoscope to autofluorescence of fresh human stomach and lamb’s kidney tissue. As an illustration of the potential for endoscopic FLIM contrast, Fig. 5(c) shows a fluorescence lifetime map of human stomach acquired “conventionally” over several seconds and fitted using a WNLLS algorithm. For comparison Fig. 5(b) shows a fluorescence intensity image of the same sample. In the top right corner of the piece of tissue the mucosa has retracted to reveal the underlying collagen-rich muscularis propria. The gastric mucosa demonstrates a homogeneous short lifetime of the order of $1000\mathrm{ps}$ , which is in clear contrast on the FLIM image to the longer lifetime $\left(1600\text{to}1800\mathrm{ps}\right)$ of the muscularis propria. To illustrate the capability of acquiring real-time endoscopic FLIM images, Fig. 7 shows a FLIM image taken from a “live” sequence acquired and updated at $5.5\mathrm{Hz}$ through the same flexible endoscope.

Fig. 7

One of a sequence of live update $\left(5.5\text{-}\mathrm{Hz}\right)$ FLIM images of the mucosal surface of a piece of a bisected lambs kidney viewed en face with $355\text{-}\mathrm{nm}$ excitation through a flexible endoscope.

4.

Improving FLIM Speed and Sensitivity

It can be seen that the images presented in the previous section are somewhat noisy and it is this noise, rather than the time-gated imaging technology, which limits the FLIM image update rate. In general, for FLIM, the major source of noise is shot noise, which scales with the number of photons. The obvious approaches to improve the SNR by increasing the number of photons captured are to increase the intensity of the excitation, to perform the experiment many times and average the results, or to integrate over a long period. For clinical applications, the first approach is likely to be prohibited by photobleaching or safety considerations and the second would involve increased acquisition and processing times and therefore lower update rates. The key to high-speed wide-field FLIM is therefore to use as long CCD integration times as are practical and to make the best use of those photons available in the acquisition time to minimize the noise on the measured fluorescence lifetime image.

For a given number of photons from the sample and for a given lifetime, it is possible to calculate the expected number of photons in each of the time gates. From this, together with knowledge of the statistics of the noise on those measurements, it is possible to calculate the expected error on the calculated lifetime, by standard error propagation, for a given set of experimental parameters and therefore to optimize the experimental fluorescence sampling strategy.

A number of factors can affect the accuracy of a lifetime as computed using this RLD algorithm. The effect of the width of the gates used to record each image and their separation has already been studied. In general, the gate width should be as wide as possible to maximize the number of collected photons and the gate separation should ideally be approximately 2.5 times the expected fluorescence lifetime.³³

A further available degree of freedom arises from the fact that ultrafast lasers typically have high repetition rates on the order of 10s of megahertz, while CCD cameras usually operate at frame rates ranging only up to several hundred hertz. Because there are usually only a small number of photons per pixel emitted after each laser pulse excitation, it is usual to integrate the light from a large number of excitation pulses at a single gate setting into a single recorded image. This approach has the additional benefit of averaging out variations in the intensity of the laser pulses. It is therefore possible and, as we will show desirable, to vary the integration time and consequently the number of excitation pulses sampled for each gate setting so that it is not equal for the two gates. Note that this is equivalent to using a different beamsplitter ratio in a single-shot system and that the discussion that follows also applies in this case.

Figure 8 shows a plot of the expected error on the measured lifetime as a function of both gate separation and the fraction of light going to the first gate for a sample exhibiting a single exponential lifetime of $1400\mathrm{ps}$ and emitting 1000 photons. For this calculation, it has been assumed that both gates are open for $2.5\mathrm{ns}$ and that the first gate opens as soon as possible after the excitation pulse to capture as much light as possible. The gate separation is therefore equal to the opening time of the second gate. It has also been assumed that the only source of noise is photon shot noise.

Fig. 8

Surface plot calculated showing the expected standard deviation of the fitted lifetime as a function of different fractions of light to the first gate and gate separations for a total of 1000 photons and a lifetime of $1.4\mathrm{ns}$ .

It is clear from the figure that the optimum parameters for minimizing the variance in the calculated lifetime do not correspond to half the light going to each gate, but to an increased amount of signal to the second (more delayed) gate, so that it receives three to four times as much light as the first. Thus, the variation in fitted lifetime is lower when the SNR of the two time-gated subimages is more similar. Note that there is no significant change in the mean lifetime observed ( $<1\%$ across the range of parameters studied) and that the optimum gate separation remains at roughly 2.5 times the lifetime, which agrees with the results in Ref. 33 for the case where half the light goes to the first gate. In fact, it can be shown using error propagation that the optimum configuration for an ideal system, for which the only noise is shot noise, is for only $\sim 21\%$ of the light to go to the first gate. This result applies over a very large range of lifetimes provided that the gate separation is set as described.

Note that by changing the ratio of exposure times away from unity, we are actually decreasing the overall number of detected photons but increasing the number of photons being sampled by the later time gate—at the expense of the signal going to the earlier gate—and thereby minimizing the variance in the lifetime.

Although it might appear that we are only able to carry out such an optimization when the expected lifetime is already known, this is not a significant drawback, as can be seen from Fig. 9 , which shows that this sort of optimization gives improved performance across a wide range of lifetimes around that value for which the optimization has been performed. In practice, it would be possible to allow an FLIM user to set the optimization to focus on lifetime ranges of specific interest, perhaps using prior knowledge of tissue fluorophores, or to interactively vary the target lifetime.

Fig. 9

Variation in the expected standard deviation of measured lifetimes found with the RLD method optimized for a lifetime of $1400\mathrm{ps}$ (red) and for the same gate settings but with equal exposure times (blue) for a range of actual lifetimes.

To confirm the numerical results outlined above, a $0.25\text{-}\mathrm{g}∕\mathrm{l}$ solution of Stilbene 3 in ethanol was used as a fluorescent sample with a spatially homogeneous lifetime. This was imaged with a range of different ratios of integration times but with the total acquisition time and gate width kept constant. The histograms of the lifetimes obtained from two of these experiments are shown in Fig. 10 , where we can see that a narrower histogram with a higher central peak is obtained when a longer integration time is used for the second gate with all other experimental parameters kept constant.

Fig. 10

Histogram of measured fluorescence lifetimes of Stibene 3 with (a) equal exposure times for the two gates of the RLD and (b) for the same gate settings and total exposure time but with the exposure time for the second gate three times longer than that for the first. The time gates were $\sim 2.4\mathrm{ns}$ wide.

Figure 11 shows how the standard deviation of the measured lifetimes varies with the ratio of the CCD integration times. For all these acquisitions there is good agreement with the numerical results and a minimum of measured noise is obtained with the CCD integration time for the second gate being approximately three times that of the first—corresponding to 25% of the light going to the first gate. This is not exactly the ratio predicted earlier here, but for this real system, we expect additional sources of noise over and above the shot noise.

Fig. 11

Standard deviation of measured lifetimes of Silbene 3 as a function of the ratio of exposure times for the two gates with all other experimental parameters unchanged (solid line), and the GOI gain adjusted to approach camera saturation (dashed line).

The data points linked by the solid line were taken while keeping all other experimental parameters constant. The data points joined by the dotted line illustrate a second advantage of this approach: the integrated intensity in the two camera frames is more similar if the second gate is longer than the first and therefore the effect of additive noise from, for example, the readout noise of the camera, becomes relatively less important. In addition as noted by Agronskaia ²³ problems with the dynamic range of the detector are reduced. For this data set, the gain of the GOI was adjusted at each point so that the camera approached saturation. With equal exposure times, this is limited by the earlier brighter image. As the exposure time for the earlier gate becomes reduced it is therefore possible to increase the gain. As a result, the amount of light in the second gate is increased by both the increased exposure time and the increased gain, reducing the effect of readout noise and consequently improving the measured standard deviation.

5.

Conclusions

FLIM provides label-free intrinsic contrast in frozen and in fresh tissue that can be superior to what can be obtained from intensity imaging alone. Real-time FLIM of fresh tissue endogenous fluorescence through both rigid and flexible endoscopes was demonstrated using sequential time-gated acquisition in combination with rapidly adjustable electronic delay circuitry. A rapid lifetime determination algorithm was used to provide fast acquisition, computation, and real-time display of the FLIM images, and numerical error analysis was used to optimize some of the parameters for real-time FLIM. Two-gate RLD was shown to exhibit a reduced error in the calculated lifetime if the CCD integration time for the two time-gated subimages is not equal; rather, the CCD integration time for the later gate should be increased such that the number of photons collected for each time-gated image is similar. These observations were confirmed experimentally and could be extended to gating schemes with more than two time gates. Clinical FLIM applications thus appear increasingly feasible and we are working to bring the wide-field time-gated endoscopic FLIM system described here to clinical trials. The development and optimization of high-speed FLIM is clearly relevant to other applications including live cell imaging and high-throughput multiwell plate imaging.