Hardware implementation algorithm and error analysis of high-speed fluorescence lifetime sensing systems using center-of-mass method

Day-Uei Li; Bruce R. Rae; Robin Andrews; Jochen Arlt; Robert K. Henderson

doi:10.1117/1.3309737

1 January 2010 Hardware implementation algorithm and error analysis of high-speed fluorescence lifetime sensing systems using center-of-mass method

Day-Uei Li, Bruce R. Rae, Robin Andrews, Jochen Arlt, Robert K. Henderson

Author Affiliations +

Journal of Biomedical Optics, Vol. 15, Issue 1, 017006 (January 2010). https://doi.org/10.1117/1.3309737

Abstract

A new, simple, high-speed, and hardware-only integration-based fluorescence-lifetime-sensing algorithm using a center-of-mass method (CMM) is proposed to implement lifetime calculations, and its signal-to-noise-ratio based on statistics theory is also deduced. Compared to the commonly used iterative least-squares method or the maximum-likelihood-estimation-based, general purpose fluorescence lifetime imaging microscopy (FLIM) analysis software, the proposed hardware lifetime calculation algorithm with CMM offers direct calculation of fluorescence lifetime based on the collected photon counts and timing information provided by in-pixel circuitry and therefore delivers faster analysis for real-time applications, such as clinical diagnosis. A real-time hardware implementation of this CMM FLIM algorithm suitable for a single-photon avalanche diode array in CMOS imaging technology is now proposed for implementation on field-programmable gate array. The performance of the proposed methods has been tested on Fluorescein, Coumarin 6, and 1,8-anilinonaphthalenesulfonate in water/methanol mixture.

1. Introduction

Time-resolved fluorescence lifetime imaging (FLIM) is widely used in cell-biology research, medical diagnosis, and pharmacological development.^{1, 2, 3} It is based on the measurement of the decay in fluorescence emission across a sample after optical excitation and can be used to quantify physiological parameters, such as pH, ${Ca}^{2 +}$ , $p O_{2}$ , etc., in biological samples. The independence of fluorescence lifetimes from probe concentration makes FLIM more favored than its counterpart—fluorescence intensity imaging. As shown in Fig. 1 , a laboratory FLIM experiment usually contains a Ti-sapphire laser, a photomultiplier tube (PMT), a time-correlated single-photon counting (TCSPC) photon-counting card, fluorescence lifetime analysis software, and a PC graphical user interface (GUI). Available FLIM systems provide excellent time resolution and light sensitivity, although they are quite expensive and cumbersome. Commercial applications increasingly demand compact and portable system-on-chip (SOC) FLIM solutions. Thanks to the progress of semiconductor technology, high-accuracy time resolution, high sensitivity, low cost, and compactness can be achieved by exploiting CMOS single-photon avalanche diode (SPAD) arrays with low dark count rate to replace PMTs^{4, 5, 6, 7} and by bump-bonding AlInGaN UV micropixellated light-emitting diodes to replace lasers⁷ in the general direction of lab on chip. The imager can include a CMOS SPAD array with in-pixel digital counters or time-to-digital converters (TDCs)^{6, 7} that allows recording not just the photon counts but also the raw timing data for detailed scientific analysis. The imager also contains Field-programmable gate arrays (FPGAs) allowing data processing. Figure 1 shows the SOC solution suited to lab-on-chip applications, which is intended to replace the system of Fig. 1. For imaging purposes, a remaining challenge is that the excessive computational demands of available lifetime analysis software such as the iterative least-squares method (LSM) or maximum-likelihood-estimation (MLE)⁸ render real-time imaging impossible. A new FLIM algorithm considering the instrument response based on the Laguerre expansion technique⁹ speeds up lifetime calculations, but the computation time increases with imager size. However, in many applications, such as microfluidic mixing¹⁰ and exploratory biological experiments, it is desirable to monitor the instantaneous biochemical interactions to provide quick feedback to corresponding manipulations. The slow speed of LSM- or MLE-based software analysis tools becomes a bottleneck and has driven the recent development of noniterative, compact, and fast real-time time-domain FLIM systems^{11, 12, 13, 14, 15, 16, 17} and real-time frequency-domain FLIM algorithms and systems.^{18, 19, 20, 21} In the past, rapid lifetime determination methods (RLD) were thought to be the simplest algorithms¹¹ and were used in some previously reported video-rate FLIMs.^{12, 13} In Ref. 12, an optomechanical delay control for RLD was proposed; however, its cumbersome optical setup makes it difficult to image a wide range of fluorophore lifetimes, and an electronically controllable delay would be preferable.¹³ To further achieve compactness for SOC, we can exploit configurable devices such as FPGAs to realize real-time FLIM systems. FPGAs have significantly benefited from the advances of CMOS technology. The latest FPGAs contain over hundreds of millions of transistors and can easily accommodate the output signals from SPAD arrays of growing size. With the ability of configuration, designers can easily reconfigure FPGAs by hardware description language to perform any application-specific logic functions, such as real-time lifetime calculations. We therefore evaluated the possibility of applying RLD either on chip or on FPGA and concluded that RLD can be implemented on FPGA with lookup tables (LUT) of natural logarithmic¹⁴ or other functions if overlap gating techniques are used.^{14, 15} The delay control of RLD can be easily reconfigured by users, and we expect that the system can benefit greatly from the user-friendly features. However, building a LUT on FPGA covering a wide range of lifetimes is inefficient, and it is desirable to develop more FPGA-friendly algorithms that use only additions. The impact will be huge, especially when a large SPAD array is applied. Moreover, a major drawback of RLD is the requirement to choose a proper time delay between two time gates,^{11, 12, 13, 14} which is quite challenging when specimens with a wide range of lifetimes coexist. We therefore propose a more hardware-friendly algorithm for lifetime calculations based on the imager developed in Ref. 6. In this paper, we first introduce the proposed hardware lifetime calculation algorithm by considering a single-exponential decay for simplicity. Although it is possible to implement multiexponential algorithms in hardware in combination with software calculations,¹⁷ it is not economic in terms of hardware resources. The single-exponential assumption allows a proper comparison of various fitting algorithms. Moreover, a single-exponential decay model is still useful to contrast different types of fluorophores. For diagnostic applications, obtaining lifetime contrast is probably more important than calculating the absolute values of lifetimes.¹³ The FPGA implementation and Verilog/Matlab modeling of the center-of-mass (CM) method (CMM) will be introduced. The performance of the proposed algorithm will be tested using Fluorescein, Coumarin 6, and 1,8-anilinonaphthalenesulfonate (ANS) in water/methanol.

Fig. 1

(a) Laboratory FLIM and (b) FLIM system on chip.

2. Theory

2.1.

CM of a Single-Exponential Function

For an object with a continuous distribution of mass density $f (r)$ and total mass $M_{T}$ , its CM is defined as

Eq. 1

CM \equiv \frac{\int r f (r) d V}{\int f (r) d V} = \frac{\int r f (r) d V}{M_{T}} .

For a mass density of a single-exponential function

f (t) = A

\exp (- t ∕ τ)

in the range

0 ⩽ t ⩽ T

, we have

2.

\int_{0}^{T} t f (t) d t = \int_{0}^{T} A t \exp (- t ∕ τ) d t = A τ^{2} (1 - e^{- T ∕ τ}) - A τ T e^{- T ∕ τ} = τ \int_{0}^{T} f (t) d t - A τ T e^{- T ∕ τ},

\Rightarrow CM = \frac{\int_{0}^{T} t f (t) d t}{\int_{0}^{T} f (t) d t} = τ - \frac{T e^{- T ∕ τ}}{1 - e^{- T ∕ τ}} .

As

T > 7 τ

, the center of mass lies at the position with a distance of

τ

from the origin as Fig. 2 shows. If

f (t)

represents a fluorescence histogram, then the denominator of Eq. 2 will be the total photon count, while the numerator is the sum of temporal information of total photon events. To implement Eq. 2 in hardware, denoted as CMM for simplicity, we need to quantize the temporal information by dividing the measurement window into

M

time bins (bin width of

h

), as shown in Fig. 2, using TDCs in the photon counting module. An interesting analog circuit was proposed to calculate Eq. 2 for single-molecule microscopy;¹⁶ however, it did not describe how to remove background noise in the analog domain. In such applications with low fluorescence emission, background-to-signal ratio will be relatively significant. Compared to Ref. 16, CMM works in the digital domain and allows background noise to be removed much more easily.

Fig. 2

(a) Center of mass of a single-exponential function and (b) concept of single-exponential CMM.

2.2.

Error Analysis of CMM

In Ref. 22, it was shown that Eq. 2 is equivalent to MLE when $M \to \infty$ ; therefore, CMM can be viewed as hardware version of MLE. When the ratio of the full width at half maximum (FWHM) of the instrumental response function (IRF) over the lifetime is $⪡ 1$ , we can assume the fluorescence decay histogram $f (t) = A$ $\exp (- t ∕ τ)$ with $τ$ being the lifetime.¹⁷ For the usual measurement setup in a lab, the FWHM of the IRF is on the order of hundreds of picoseconds; thus, it is reasonable to target lifetimes of $> 500 ps$ . With the assumption of single-exponential decay, the lifetime $τ$ is related to the decay function as

Eq. 3

τ ≅ \frac{\int Δ t f (t) d t}{\int f (t) d t} = \frac{\int_{t_{0}}^{t_{1}} (t - t_{0}) f (t) d t + \dots + \int_{t_{M - 1}}^{t_{M}} (t - t_{0}) f (t) d t}{\int_{t_{0}}^{t_{M}} f (t) d t} ≅ \frac{\int_{t_{0}}^{t_{1}} (\frac{t_{0} + t_{1}}{2} - t_{0}) f (t) d t + \dots + \int_{t_{M - 1}}^{t_{M}} (\frac{t_{M - 1} + t_{M}}{2} - t_{0}) f (t) d t}{\int_{t_{0}}^{t_{M}} A \exp (- t ∕ τ) d t} = \frac{\sum_{j = 0}^{M - 1} Δ t_{j} \int_{t_{j}}^{t_{j + 1}} f (t) d t}{N_{c}} = \frac{\sum_{j = 0}^{M - 1} Δ t_{j} N_{j}}{N_{c}} = \frac{\frac{N_{c}}{2} + \sum_{j = 0}^{M - 1} j N_{j}}{N_{c}} h = (\frac{\sum_{j = 0}^{M - 1} j N_{j}}{N_{c}} + \frac{1}{2}) h = R h,

where

Δ t_{j} = t_{j} - t_{0} + h ∕ 2 = (j + 1 - 1 ∕ 2) h

and

N_{j}

is the number of recorded counts in the

j

’th time bin (

j = 0

, 1,…,

M - 1

), and

N_{c} = A τ [1 - \exp (- M h ∕ τ)]

is the total effective signal count. The recorded variables

N_{j}

are independently Poisson distributed with respective mean value

E N_{j} = \int_{j h}^{(j + 1) h} f (t) d t

and standard deviation

σ N_{j} = {(E N_{j})}^{1 ∕ 2}

, and we thus have

Eq. 4

N_{j} = E N_{j} + σ N_{j} = N_{c} x^{j} (1 - x) {(1 - x^{M})}^{- 1} + σ N_{j},

where

x = \exp (- h ∕ τ)

. Substituting Eq. 4 into Eq. 3, we have

Eq. 5

τ_{CMM} ≅ \frac{\sum_{j = 0}^{M - 1} Δ t_{j} (E N_{j} + σ N_{j})}{\sum_{j = 0}^{M - 1} (E N_{j} + σ N_{j})} = \frac{\sum_{j = 0}^{M - 1} Δ t_{j} E N_{j} + \sum_{j = 0}^{M - 1} Δ t_{j} σ N_{j}}{\sum_{j = 0}^{M - 1} E N_{j} + \sum_{j = 0}^{M - 1} σ N_{j}} = \frac{U + σ u}{V + σ v} = \frac{U}{V} \frac{1 + σ u ∕ U}{1 + σ v ∕ V} = τ (1 + \frac{Δ τ}{τ}) (1 + \frac{σ u}{U} - \frac{σ v}{V}),

where

Eq. 6

U = \sum_{j = 0}^{M - 1} Δ t_{j} E N_{j}, V = N_{c}, σ u = \sum_{j = 0}^{M - 1} Δ t_{j} σ N_{j}, σ v = \sum_{j = 0}^{M - 1} σ N_{j} .

From Eqs. 4, 6, we have

\frac{U}{V} = \frac{h (1 - x)}{1 - x^{M}} \sum_{j = 0}^{M - 1} (j + 1 - \frac{1}{2}) x^{j} = \frac{h (1 - x)}{1 - x^{M}} {(x + \dots + x^{M})}^{'} - \frac{h}{2} = h [\frac{1 - (M + 1) x^{M} + M x^{M + 1}}{(1 - x) (1 - x^{M})} - \frac{1}{2}] = h G (x) = τ (1 + \frac{Δ τ}{τ});

therefore, we have the accuracy equation

Eq. 7

\Rightarrow \frac{Δ τ_{CMM}}{τ_{CMM}} = \frac{h}{τ} G (x) - 1,

Eq. 8

G (x) = \frac{1 + x - (2 M + 1) x^{M} + (2 M - 1) x^{M + 1}}{2 (1 - x) (1 - x^{M})} .

From Eqs. 5, 6, 7, we have the precision equation

Eq. 9

\frac{σ τ_{CMM}}{τ_{CMM}} = (1 + \frac{Δ τ}{τ}) (\frac{σ u}{U} - \frac{σ v}{V}) = \frac{h}{τ} G (x) (\frac{σ u}{U} - \frac{σ v}{V}),

Eq. 10

\frac{σ u}{U} - \frac{σ v}{V} = \sqrt{\sum_{j = 0}^{M - 1} [{(\frac{Δ t_{j}}{U} - \frac{1}{N_{c}})}^{2} σ N_{j}^{2}]} = \frac{1}{\sqrt{N_{c}} G (x)} \sqrt{(\frac{1 - x}{1 - x^{M}}) \sum_{j = 0}^{M - 1} {[j + \frac{1}{2} - G (x)]}^{2} x^{j}} = \frac{1}{\sqrt{N_{c}}} \frac{\sqrt{P (x)}}{(1 - x) (1 - x^{M}) G (x)},

Eq. 11

P (x) = x - M^{2} x^{M} + (2 M^{2} - 2) x^{M + 1} - M^{2} x^{M + 2} + x^{2 M + 1} .

The accuracy of the CMM lifetime estimator is determined by the quantization error in Eq. 3. It is usually predictable and can be calibrated by software,¹⁷ whereas the precision (normalized standard deviation) mainly comes from Poisson noise and can be improved only through increasing photon count. From Eqs. 2, 3, we can also calibrate the lifetime by

Eq. 12

τ_{CMM, Cal} ≅ τ_{CMM} + \frac{T e^{- M h ∕ τ_{CMM}}}{1 - e^{- M h ∕ τ_{CMM}}} = (R + \frac{M e^{- M ∕ R}}{1 - e^{- M ∕ R}}) h .

This calibration can be easily done by software and can improve the accuracy further to

T > 4 τ

.

Figure 3 shows the inverse accuracy and precision curves of Eqs. 7, 9 (for easily transferred to decibels) for $M = 1024$ and $N_{c} = 2^{17}$ . The theoretical results marked as solid lines are compared to Monte Carlo simulations marked with crosses, giving good agreement and proving the correctness of Eqs. 7, 9. Theoretical precision curves of 1024-bin MLE, and 2-gate RLD (with gate width $w_{g} = M h ∕ 2 = 512 h$ ) are also provided for comparison. From Fig. 3, the optimal window for RLD is from $M h ∕ τ = 1 - 5$ , whereas that for CMM is from $M h ∕ τ = 7 - 100$ . Here, we define a new precision value for CMM as

Eq. 13

Precision \equiv \frac{τ_{CMM}}{\sqrt{σ τ_{CMM}^{2} + Δ τ_{CMM}^{2}}} .

In applications, there is no need to define this new precision as long as the accuracy can be enhanced by software calibration as described above. For simplicity, we assume there is no software calibration available. The new precision definition facilitates end users to familiarize themselves with the sensing system and easily choose a proper parameter. The new precision curve is also shown in Fig. 3. Its optimal window is the same as that of MLE from

M h ∕ τ = 10 - 100

.

Fig. 3

Inverse precision and accuracy curves for the 1024-bin CMM, 1024-bin MLE, and 2-gate RLD with $N_{c} = 2^{17}$ (measurement $window = M h = 2 w_{g}$ ).

In some applications, we need to know the range of lifetimes that a predictor can resolve when the laser repetition rate (LPR) or the measurement window (MW) is fixed. We use the $F$ value introduced in Ref. 23 to quantify the performance of a lifetime imaging technique. The $F$ value is defined as $F = N_{c}^{1 ∕ 2} σ τ ∕ τ$ , where $σ τ$ is the standard deviation of repeated measurements of the lifetime value $τ$ . Figure 4 shows $F$ curves for 1024-bin CMM, 4096-bin CMM, 1024-bin MLE, 15-bin integration for extraction method (IEM),¹⁷ and 2-gate RLD in terms of $τ$ normalized by measurement window $(MW = M h)$ . The MLE demonstrates the best resolvability range. However, it is not possible to implement it in hardware. For the other three methods, only CMM has a flat optimal $F$ response. Taking $LPR = 5 MHz$ as an example and assuming the TDC full range is equivalent to the measurement window, $MW = 200 ns$ . If $N_{c} = 2^{17}$ , then the lifetime range with a precision of $40 dB$ $(F \sim 4)$ for 2-gate RLD, 15-bin IEM, 1024-bin CMM, and 1024-bin MLE are 16–200, 12–140, 0.6–30, and $0.02 - 320 ns$ , respectively. For RLD, the optimal window can be chosen by selecting proper delays, mechanically or electronically.^{12, 13} For CMM and IEM, the optimal window can be easily set on FPGA by choosing a proper $M$ . In theory, the lower bound of CMM and MLE can be further reduced by increasing $M$ . However, it is limited by the FWHM of the system IRF, which can be several hundreds of picoseconds, considering jitter contributed by the SPADs, laser, and TDCs. Therefore, increasing $M$ further for real-time, single-exponential lifetime estimation is not sensible. A comparison summary for the CMM, IEM, RLD, and the MLE algorithms is provided in Table 1 . It clearly shows the merits of the CMM in terms of $F$ value, lifetime resolvability, and on-chip feasibility. From Fig. 4, the advantage of CMM is that its photon collecting efficiency, for a given precision, is 2.5-fold larger than RLD-2 and IEM. The measurement window should be about $10 \times$ the lifetime (or $7 \times$ with software calibration) to achieve high sensitivity (good photon economy). In this respect, RLD imaging can use a higher laser repetition rate and achieve a better duty cycle at measurement $window = 1 - 5 τ$ . However, if complete raw arrival time data are needed, the laser repetition rate cannot be too low. Moreover, for most fluorophores, the measurement window is $> 4 τ$ to avoid nonidealities such as bleed through. Although one might argue that for mono-exponential decays the bleed through does not matter too much, but it undermines background correction. This means our CMM detector system has a slightly lower duty-cycle performance in order to maintain the sensitivity. However, our system contains TDCs and can provide both raw data and lifetime data output.

Fig. 4

$F$ -value curves for the 1024-bin CMM, 1024-bin MLE, 4096-bin CMM, 15-bin IEM,¹⁷ and 2-gate RLD with $N_{c} = 2^{17}$ (measurement $window = M h = 2 w_{g}$ ).

Table 1

Comparison summary of the CMM, IEM, RLD, and MLE.

Method	Fmin at h (τ)	F<4 resolvability	On-chipfeasibility
StandardRLD-2	1.5at 2.5	$0.08 < τ ∕ MW < 1$	Yes/LUT2
IEMw/oCalibration¹⁷	1.6at 0.671	$0.06 < τ ∕ MW < 0.7$ for $M = 15$	Yes
IEMwithCalibration¹⁷	1.2at 1.67	$0.03 < τ ∕ MW < 0.7$ for $M = 15$	Yes
MLE $M = 1024$	1.0at [0.01–0.5]	$1 \times 10^{- 4} < τ ∕ MW < 1.6$	No
CMM $M = 1024$	1.0at [0.01–0.1]	$3 \times 10^{- 3} < τ ∕ MW < 0.15$	Yes
CMM $M = 4096$	1.0at [0.003–0.08]	$7 \times 10^{- 4} < τ ∕ MW < 0.15$	Yes

^aThe optimal h of IEM is independent of M .

^bOn a small detector array.

2.3.

Error Analysis of CMM with Background Correction

In most practical lifetime analysis tools, background is taken into account by the subtraction of a dc background value $C_{0}$ ( $= N_{b} ∕ M$ , $N_{b}$ is total background noise within a measurement window of $M h$ ) from the measured histogram, and along with lifetime calculations, this is done by software. However, for faster real-time imaging it is desirable that a hardware calibration technique can be integrated into the system by generating the required $C_{0}$ using the available counts. Figure 5 shows a typical measured histogram (of $1 μ M$ Fluorescein, detailed in Section 4). There always exists a flat response before the peak of the histogram decided by the delay between systems enable signal and laser excitation. A dc value of $C_{0}$ can be obtained by averaging the counts of several bins on the flat response. Suppose we have a white background noise response, and we can therefore obtain the background count as $N_{b} = M C_{0}$ , from Eq. 5, and by subtracting $C_{0}$ from the count in each bin, we have

Eq. 14

\frac{τ_{CMM, corr}}{h} = \frac{\sum_{j = 0}^{M - 1} \frac{Δ t_{j}}{h} (N_{j} - C_{0})}{N_{total} - M C_{0}} = \frac{\sum_{j = 0}^{M - 1} (j + \frac{1}{2}) N_{j} - \frac{M N_{b}}{2}}{N_{c}},

where

N_{total} = N_{c} + N_{b}

,

15.

N_{j} = E N_{j} + σ N_{j} = N_{c} \frac{x^{j} (1 - x)}{1 - x^{M}} + \frac{N_{b}}{M} + σ N_{j},

E N_{j} = \int_{j h}^{(j + 1) h} [f (t) + N_{b} ∕ (M h)] d t, and σ N_{j} = {(E N_{j})}^{1 ∕ 2} .

The error equations can be obtained by replacing Eq. 15 into Eq. 14 and following the same procedure as Eqs. 4, 5, 6, 7, 8, 9, 10, 11.

Fig. 5

Measured fluorescence histogram of Fluorescein by a CMOS SPAD.

3. Hardware Implementation Method and Modeling

3.1.

FPGA Implementation Method

We rewrite Eq. 3 as

Eq. 16

\frac{τ_{CMM}}{h} = \frac{\sum_{i = 1}^{N_{c}} {\bar{D}}_{i}}{N_{c}} + \frac{1}{2},

where

{\bar{D}}_{i}

is the

K

-bit TDC output of the

i

’th captured photon. It is quite easy to implement Eq. 16 on FPGA. It can also be readily modified to implement background correction of Eq. 14. For simplicity, we introduce the FPGA implementation of Eq. 16. The first term on the right-hand side can be implemented by an adder for the numerator and a counter for the denominator. In Fig. 6 , a

(K + \log_{2} N_{c})

-bit register is used to store the results from the adder and sends them back to the adder for adding with the TDC code of the next photon. The lifetime can be updated by latching the results when the counter storing

N_{c}

reaches a value of

Eq. 17

N_{c} = 2^{L}, L is an integer .

When this condition is reached, a trigger signal is sent to latch the latest

τ_{CMM} ∕ h

and reset the register to perform the next calculation and keep updating the lifetime. By this arrangement, we do not even need digital division by only taking the first

K

most significant bit (MSB) bits of the register or more than

K

MSB bits for decimal accuracy. The second term

1 ∕ 2

on the right-hand side can be kept in mind or simply merging Eqs. 16, 17 as

Eq. 18

\frac{τ_{CMM}}{h} = \frac{\sum_{i = 1}^{N_{c}} {\bar{D}}_{i} + 2^{L - 1}}{2^{L}},

with only a minor effort on the FGPA resource because it is a global correction term for the whole SPAD array. From the discussion above, in the optimal window or lifetime resolving range for CMM, we have

Eq. 19

F = \frac{N_{c}^{1 ∕ 2} σ τ}{τ} = \frac{2^{L ∕ 2} σ τ}{τ} = 1.0 \Rightarrow \frac{τ}{σ τ} = 2^{L ∕ 2} or signal-to-noise ratio (SNR) ≅ 3 L (dB) .

This is a very convenient formula for end users. By selecting a proper

L

via the GUI, one can easily set the accuracy of images. For example, if a precision of

SNR = 30 dB

(σ τ ∕ τ = 3 %)

is required for the system, the total count within the measurement window is

2^{SNR ∕ 3} = 1024

. The expression of Eq. 19 is the same with that of MLE in the optimal range. For imaging purposes, CMM can be applied to a column of SPAD pixels, and the hardware implementation can be extended accordingly. For video-rate applications, we have to keep the lifetime update time to

Eq. 20

t_{update} = \frac{2^{L}}{PCR} ⩽ 30 ms,

where PCR is the photon count rate. For example, for a precision of

SNR = 30 dB

with a lifetime update rate of

> 33 fps

, the PCR should be larger than

33 kHz

, which is not a difficult task at all for the latest CMOS SPAD detectors.^{4, 5, 6, 7} End users can choose a proper

L

to maintain an acceptable accuracy while keeping the lifetime update rate. All parameters can be set by end users via the GUI considering the image contrast, accuracy, and update speed. Other functions, such as automatic histogram peak finding, imaging filtering, dark area marking, homogeneous lifetime updating, and background correction, can also be implemented on a FPGA. With this arrangement, video-rate lifetime images can be generated and the dynamics of interactions between fluorophores and the microenvironment, such as microfluidic mixing, can be easily observed.

Fig. 6

FPGA implementation of CMM.

3.2.

SPAD Detection Model and Verilog/Matlab Simulations

Before employing CMM with a SPAD array, we first built a detection model of a SPAD pixel in order to verify the efficiency of the algorithm on the FPGA. Figure 7 shows the diagram for the detection model of a SPAD pixel. The $8 - bit$ signal coming from a TDC in a SPAD pixel cell can be modeled by a $31 - bit$ pseudo-random bit sequence (PRBS) generator and a lookup table used for generating a photon-emission probability function. The threshold values $V_{th, j}$ are related as

Eq. 21

V_{th, j + 1} - V_{th, j} \propto N_{j}, j = 0, \dots, M - 1 .

The overall jitter of the SPAD and laser (assumed as a Gaussian distribution), and the laser excitation delay between the electrical excitation signal and laser pulse is built right after the exponential lookup table. The SPAD detection model then feeds the output data into a serializer to model the signal coming from the SPAD pixel and then via a deserializer to the CMM lifetime calculation module in Fig. 6. The polynomial for generating the

31 - bit

PRBS is

g (x) = 1 + x^{28} + x^{31}

.²⁴ The maximum length of the bit sequence is

2^{31} - 1 = 2.15 \times 10^{9}

, which is much larger than the total photon count of usual TCSPC measurements for a single pixel. For comparison to RLD algorithms and for simplicity, we built a lookup table of digital division inside the CMM implementation block; although in practice, there is no need to do so according to Eqs. 16, 17. Taking a single decay function

f (t) = A \exp [- t ∕ (21 h)]

as an example, if

h

is of

200 ps

(the full range of the TDC is

256 h = 51.2 ns

, which is equivalent to a

LPR = 20 MHz

). The lifetime

τ = 21 h = 4.2 ns

is much larger than a typical jitter of

300 ps

such that only tail fitting is applied to extract the lifetime without digital deconvolution.¹⁷ A measurement window of

M = 200

(44th to 243rd bin) from the peak of the histogram is chosen for lifetime calculations. Figure 8 shows the decay histogram

N_{r}

obtained by the model and the fitted curve

N_{f}

by CMM with background correction of Eq. 14 using Verilog. The calculated lifetimes with four extra bits for decimal accuracy obtained by CMM and RLD with/without background correction are listed, respectively. The reduced chi-squared is 1.10 showing a good fit, and Fig. 8 also shows the normalized residual plot of

(N_{r} - N_{f}) ∕ N_{f}^{1 ∕ 2}

, which is well distributed, implying that the model is Poisson distributed as in real cases.

Fig. 7

SPAD detection model implemented on FPGA.

Fig. 8

Decay and fitted curves obtained by the model and CMM using Verilog.

The second example is $f (t) = A \exp [- t ∕ (2 h)]$ , with a lifetime $τ = 2 h = 400 ps$ at $LPR = 20 MHz$ . We are comparing CMM to other algorithms with $M = 200$ . CMM is not sensitive to the timing jitter. For RLD, it is a challenging task to resolve lifetimes, much less than the effective measurement window ( $τ ⪡ 200 h$ in this case). Thus, we use MLE⁸ instead to calculate the lifetime with software using

Eq. 22

1 + {(x^{- 1} - 1)}^{- 1} - M {(x^{- M} - 1)}^{- 1} = N_{c}^{- 1} \sum_{j = 0}^{M - 1} (j + 1) N_{j},

where

x = \exp (- h ∕ τ)

. In this example, all algorithms are performed on the computer for a fair comparison, and therefore, there is no digital quantization error for CMM. The reduced chi-squared is 0.94, and Fig. 9 shows the decay histogram and residual obtained by the model and the fitted curve by CMM with background correction using Matlab. The calculated lifetimes for CMM and MLE are also listed. For CMM and MLE, it is necessary to apply background correction when resolving lifetime much less than the measurement window. It is also interesting to note that the behavior of CMM and MLE are almost identical in terms of precision and sensitivity to background noise in the optimal lifetime range; therefore, CMM can be viewed as a hardware implementation algorithm of MLE although their physical definitions are not the same. Figure 10 shows inverse precision curves versus total count for 800-bin CMM and 2-gate RLD (

2 w_{g} = 800 h = 1 τ

or

4 τ

) with Monte Carlo simulations for different lifetimes. CMM displays its uniform performance and higher photon-counting efficiency over a wide range of lifetimes. Therefore, CMM is suitable for low light detection.

Fig. 9

Decay and fitted curves obtained by the model and CMM using Matlab.

Fig. 10

Inverse precision curves versus total count of 800-bin CMM and 2-gate RLD for different lifetimes.

4. Experimental Results

4.1.

Measurements of Fluorescein and Coumarin 6 Using SPADs

Measurements of the decays of Fluorescein and Coumarin 6 mounted on microcavity slides have been made to test the proposed CMM hardware lifetime calculation algorithm. Table 2 lists the fluorophores under test in terms of solvent, concentration, excitation and emission wavelengths, typical lifetimes provided by the manufacturers, and the calculated lifetimes using CMM, MLE, RLD-2, and Edinburgh Instruments F900 software. Then, $45 μ L$ of each sample was pipetted into a single-cavity ( $15 - mm$ diam) glass microscope slide (Fisher Scientific, United Kingdom, MNK-140-010A) and sealed with a $0.12 - mm$ -thick borosilicate glass coverslip (Fisher Scientific, MNJ-300-020T). The LPR (PicoQuant pulsed diode laser with wavelength of $470 nm$ ) is $10 MHz$ , and the average output power is $0.12 mW$ . Fluorescence decay curves were recorded on a time scale of $100 ns$ , resolved into 1024 time bins (i.e., $h \sim 0.098 ns$ ). With LPR of $10 MHz$ , there is no bleed through observed on measured histograms. The fluorescence emission is captured by a SPAD array fabricated in a $0.35 - μ m$ CMOS high-voltage process mounted on a daughter board. Figure 5 shows the measured histogram of Fluorescein, and Fig. 11 shows the logarithmic plot for the measured histogram $N_{r}$ , starting from the bin with the peak intensity and the fitted curve $N_{f}$ by CMM with background correction, and also the normalized residual count. The reduced chi-squared is 1.40. The last three rows of Table 2 show the calculated lifetimes with background correction for CMM, MLE, RLD, and Edinburgh Instruments F900 software, respectively. Measurement windows of $5 - 20 ∕ 30 τ$ , $0.5 - 20 ∕ 30 τ$ , $1 - 5 τ$ , and $1 - 20 ∕ 30 τ$ are chosen for CMM, MLE, RLD-2, and F900, respectively. The mean lifetimes of Fluorescein and Coumarin 6 calculated by CMM, are 4.15 and $2.42 ns$ , respectively, in good agreement with the data provided by the manufacturers and are also comparable to other algorithms.

Fig. 11

Measured fluorescence histogram of Fluorescein and fitted curve obtained by CMM.

Table 2

Summary of fluorophores used.

Fluorophore	Fluorescein	Coumarin 6
Solvent	Ethanol	Methanol
Concentration $(μ M)$	1	1
Excitation wavelength (nm)	495	460
Peak emission wavelength (nm)	517	505
Typical lifetime (ns)	4.1	2.5
Calculated lifetime (ns) usingCMM $h = 0.098 ns$	$4.15 \pm 0.15$ $5 τ < Mh < 20 τ$ 1	$2.42 \pm 0.08$ $5 τ < Mh < 30 τ$ 2
Calculated lifetime (ns) usingMLE $h = 0.098 ns$	$4.2 \pm 0.20$ $0.5 τ < Mh < 20 τ$	$2.4 \pm 0.1$ $0.5 τ < Mh < 30 τ$
Calculated lifetime (ns) usingRLD-2 $1 τ < 2 w_{g} < 5 τ$	$4.4 \pm 0.08$	$2.42 \pm 0.05$
Calculated lifetime (ns) usingEdinburgh instruments F900 $h = 0.098 ns$	$4.38 \pm 0.06$ $1 τ < Mh < 20 τ$	$2.41 \pm 0.03$ $1 τ < Mh < 30 τ$

^aMaximum measurement window for Fluorescein ∼20×4.1 (∼800h) .

^bMaximum measurement window for Coumarin 6∼30×2.5 (∼800h) .

4.2.

Measurements of ANS in Water/Methanol Using PMTs

Fluorescent dye ANS is widely used in biological experiments due to its extreme sensitivity to the composition of water/methanol mixtures, showing a drastic variation in lifetime from $250 ps$ in pure water to $6 ns$ in pure methanol.²⁵ The excitation light comes from a Ti-sapphire laser ( $LPR = 4.75 MHz$ using a pulse picker) with a laser power of $0.1 mW$ . The concentration of ANS is $1 mM$ . The ANS in the water/methanol mixture with a concentration of water of 0, 10, 20, 30, and 100% $ν ∕ ν$ , respectively, is measured by a PMT. Figure 12 shows the measured and fitted fluorescence histograms obtained by the PMT and CMM, respectively. The fluorescence histograms of ANS display a single-exponential decay as stated in the previously reported literature,^{10, 25} making it an ideal probe of solvent composition. The calculated lifetimes in terms of water concentration obtained by CMM, the prediction function in Ref. 10, and Edinburgh Instruments F900 software are listed in Table 3 . They are in a good agreement with one another. The reduced chi-squared is also listed in Table 3.

Fig. 12

Measured and fitted fluorescence histograms obtained by a PMT and CMM for different concentration of water (measured in percent $ν ∕ ν$ ).

Table 3

Comparison of calculated lifetimes of ANS between CMM, prediction function,10 and Edinburgh Instruments F900 software.

Percentage ν∕ν (%)	CMM (ns)/reduced chi-squared	Ref. 10 (ns)	F900 (ns)
0	6.08/1.7	5.97	6.13
10	4.87/1.01	4.76	4.86
20	3.49/1.10	3.38	3.47
30	2.49/1.02	2.4	2.48
100	0.28/1.22	0.25	0.27

5. Conclusion

We have proposed a very simple FLIM algorithm called CMM [Eq. 3] for real-time applications and derived the theoretical error equations [Eqs. 7, 9] for easily obtaining the best recording parameters, such as measurement window, width of a time bin, and bit resolution of the TDC. The method has the potential of using the available photons efficiently, provided that the recording parameters are correctly optimized. For single-exponential lifetime imaging, the algorithm provides the same precision level as the MLE in the optimal window with $F$ of 1.0. An interesting result of our study is that the optimum performance of $F \sim 1.0$ can be obtained at

Eq. 23

10 h < τ < 0.1 M h (or 10 h < τ < 0.14 M h, with software calibration)

or for

F \sim 1.5

comparable to RLD-2

(w_{g} \sim 2.5 τ)

Eq. 24

5 h < τ < 0.35 M h .

The advantage of CMM over the other hardware algorithms is that CMM has higher photon-collecting efficiency

(> \times 2.5)

than RLD/IEM. For CMM of Eq. 3, without any differential term similar to that of IEM,¹⁷ the design specifications of in-pixel TDCs can be more relaxed. The FPGA implementation of this FLIM algorithm is proposed for the first time as Eqs. 17, 18. Hardware implementation of CMM with background correction can also be easily implemented on FPGA with Eq. 14. The performance of CMM is successfully tested not only on Verilog/Matlab synthetic data but also on real data collected by CMOS SPAD pixels and PMTs. CMM on the latest developed CMOS SPAD arrays has single-photon sensitivity and provides an efficient way of video-rate FLIM implementations; it is promising for imaging applications.

Acknowledgments

This work has been supported by the European Community within the Sixth Framework Programme of the Information Science Technologies, Future and Emerging Technologies Open MEGAFRAME project (Contract No. 029217-2, www.megaframe.eu). R.A. was funded by the Biotechnology and Biological Sciences Research Council RASOR Grant No. BB/C511599/1. The measurements have been performed using the COSMIC laboratory facilities with help from Trevor Whittley and David Dryden. The authors express gratitude to them. This publication reflects only the authors’ views. The European Community is not liable for any use that may be made of the information contained herein.

References

1.

J. R. Lakowicz, Principles of Fluorescence Spectroscopy, 3rd ed.Kluwer Academic/Plenum Publishers, New York (2006). Google Scholar

2.

P. I. H. Bastiaens and A. Squire, “Fluorescence lifetime imaging microscopy: spatial resolution of biochemical processes in the cell,” Trends Cell Biol., 9 48 –52 (1999). https://doi.org/10.1016/S0962-8924(98)01410-X 0962-8924 Google Scholar

3.

R. Sanders, A. Draaijer, H. C. Gerritsen, P. M. Houpt, and Y. K. Levine, “Quantitative pH imaging in cells using confocal fluorescence lifetime imaging microscopy,” Anal. Biochem., 227 302 –308 (1995). https://doi.org/10.1006/abio.1995.1285 0003-2697 Google Scholar

4.

M. Gersbach, J. Richardson, E. Mazaleyrat, S. Hardillier, C. Niclass, R. Henderson, L. Grant, and E. Charbon, “A low-noise single-photon detector implemented in a

130 nm

CMOS imaging process,” Solid-State Electron., 53 803 –808 (2009). https://doi.org/10.1016/j.sse.2009.02.014 0038-1101 Google Scholar

5.

J. A. Richardson, L. A. Grant, and R. K. Henderson, “Low dark count single-photon avalanche diode structure compatible with standard nanometer scale CMOS technology,” IEEE Photon. Technol. Lett., 21 1020 –1022 (2009). https://doi.org/10.1109/LPT.2009.2022059 1041-1135 Google Scholar

6.

J. Richardson, R. Walker, L. Grant, D. Stoppa, F. Borghetti, E. Charbon, M. Gersbach, and R. Henderson, “A

32 \times 32

50 ps

resolution

10 bit

time to digital converter array in

130 nm

CMOS for time correlated imaging,” (2009). Google Scholar

7.

B. Rae, C. Griffin, K. Muir, J. Girkin, E. Gu, D. Renshaw, E. Charbon, M. Dawson, and R. Henderson, “A microsystem for time-resolved fluorescence analysis using CMOS single-photon avalanche diodes and micro-LEDs,” 166 –167 (2008). Google Scholar

8.

P. Hall and B. Selinger, “Better estimates of exponential decay parameters,” J. Chem. Phys., 85 2941 –2946 (1981). https://doi.org/10.1021/j150620a019 0021-9606 Google Scholar

9.

J. A. Jo, Q. Fang, and L. Marcu, “Ultrafast method for the analysis of fluorescence lifetime imaging microscopy data based on the Laguerre expansion technique,” IEEE J. Sel. Top. Quantum Electron., 11 835 –845 (2005). https://doi.org/10.1109/JSTQE.2005.857685 1077-260X Google Scholar

10.

S. W. Magennis, E. M. Graham, and A. C. Jones, “Quantitative spatial mapping of mixing in microfluidic systems,” Angew. Chem., Int. Ed., 44 6512 –6516 (2005). https://doi.org/10.1002/anie.200500558 1433-7851 Google Scholar

11.

R. M. Ballew and J. N. Demas, “An error analysis of the rapid lifetime determination method for the evaluation of single exponential decays,” Anal. Chem., 61 30 –33 (1989). https://doi.org/10.1021/ac00176a007 0003-2700 Google Scholar

12.

A. V. Agronskaia, L. Tertoolen, and H. C. Gerritsen, “High frame rate fluorescence lifetime imaging,” J. Phys. D, 36 1655 –1662 (2003). https://doi.org/10.1088/0022-3727/36/14/301 0022-3727 Google Scholar

13.

D. S. Elson, I. Munro, J. Requejo-Isidro, J. McGinty, C. Dunsby, N. Galletly, G. W. Stamp, M. A. A. Neil, M. J. Lever, P. A. Kellett, A. Dymoke-Bradshaw, J. Hares, and P. M. W. French, “Real-time time-domain fluorescence lifetime imaging including single-shot acquisition with a segmented optical image intensifier,” New J. Phys., 6 1 –13 (2004). https://doi.org/10.1088/1367-2630/6/1/180 1367-2630 Google Scholar

14.

D.-U. Li, B. Rae, E. Bonnist, D. Renshaw, and R. Henderson, “On-chip fluorescence lifetime extraction using synchronous gating scheme—theoretical error analysis and practical implementation,” 171 –176 (2008). Google Scholar

15.

C. Moore, S. P. Chan, J. N. Demas, and B. A. Degraff, “Comparison of methods for rapid evaluation of lifetime of exponential decays,” Appl. Spectrosc., 58 603 –607 (2004). https://doi.org/10.1366/000370204774103444 0003-7028 Google Scholar

16.

W. Trabesinger, C. G. Hübner, B. Hecht, and T. P. Wild, “Continuous real-time measurement of fluorescence lifetime,” Rev. Sci. Instrum., 73 3122 –3124 (2002). https://doi.org/10.1063/1.1488676 0034-6748 Google Scholar

17.

D.-U. Li, E. Bonnist, D. Renshaw, and R. Henderson, “On-chip, time-correlated, fluorescence lifetime extraction algorithms and error analysis,” J. Opt. Soc. Am. A, 25 1190 –1198 (2008). https://doi.org/10.1364/JOSAA.25.001190 0740-3232 Google Scholar

18.

H. P. Good, A. J. Kallir, and U. P. Wild, “Comparison of fluorescence lifetime fitting techniques,” J. Phys. Chem., 89 5435 –5441 (1984). https://doi.org/10.1021/j150666a066 0022-3654 Google Scholar

19.

P. C. Schneider and R. M. Clegg, “Rapid acquisition, analysis, and display of fluorescence lifetime-resolved images for real-time applications,” Rev. Sci. Instrum., 68 4107 –4119 (1997). https://doi.org/10.1063/1.1148354 0034-6748 Google Scholar

20.

J. Mizeret, T. Stepinac, M. Hansroul, A. Studzinski, H. van den Bergh, and G. Wagnières, “Instrumentation for real-time fluorescence lifetime imaging in endoscopy,” Rev. Sci. Instrum., 70 4689 –4701 (1999). https://doi.org/10.1063/1.1150132 0034-6748 Google Scholar

21.

R. A. Colyer, C. Lee, and E. Gratton, “A novel fluorescence lifetime imaging system that optimizes photon efficiency,” Microsc. Res. Tech., 71 201 –213 (2008). https://doi.org/10.1002/jemt.20540 1059-910X Google Scholar

22.

J. Tellinghuisen, C. W. Wilkerson Jr., “Bias and precision in the estimation of exponential decay parameters from sparse data,” Anal. Chem., 65 1240 –1246 (1993). https://doi.org/10.1021/ac00057a022 0003-2700 Google Scholar

23.

A. Draaijer, R. Sanders, and H. C. Gerritsen, “Fluorescence lifetime imaging, a new tool in confocal microscopy,” Handbook of Biological Confocal Microscopy, 491 –505 Plenum Publishing, New York (1995). Google Scholar

24.

P. Alfke, “Efficient shift registers, LFSR counters, and long pseudo-random sequence generators,” (1996) Google Scholar

25.

L. Dougan, J. Crain, H. Vass, and S. W. Magennis, “Probing the liquid state structure and dynamics of aqueous solutions using fluorescence spectroscopy,” J. Fluoresc., 14 91 –97 (2004). https://doi.org/10.1023/B:JOFL.0000014666.03818.a4 1053-0509 Google Scholar

Citation Download Citation

Day-Uei Li, Bruce R. Rae, Robin Andrews, Jochen Arlt, and Robert K. Henderson "Hardware implementation algorithm and error analysis of high-speed fluorescence lifetime sensing systems using center-of-mass method," Journal of Biomedical Optics 15(1), 017006 (1 January 2010). https://doi.org/10.1117/1.3309737

Published: 1 January 2010

Access the abstract

JOURNAL ARTICLE
10 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

CITATIONS

Cited by 54 scholarly publications and 4 patents.

Explore citations on Lens.org

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Luminescence

Field programmable gate arrays

Fluorescence lifetime imaging

Error analysis

Sensing systems

Photon counting

Lithium

1.

Introduction

Fig. 1

2.

Theory

2.1.

CM of a Single-Exponential Function

Eq. 1

2.

Fig. 2

2.2.

Error Analysis of CMM

Eq. 3

Eq. 4

Eq. 5

Eq. 6

Eq. 7

Eq. 8

Eq. 9

Eq. 10

Eq. 11

Eq. 12

Eq. 13

Fig. 3

Fig. 4

Table 1

2.3.

Error Analysis of CMM with Background Correction

Eq. 14

15.

Fig. 5

3.

Hardware Implementation Method and Modeling

3.1.

FPGA Implementation Method

Eq. 16

Eq. 17

Eq. 18

Eq. 19

Eq. 20

Fig. 6

3.2.

SPAD Detection Model and Verilog/Matlab Simulations

Eq. 21

Fig. 7

Fig. 8

Eq. 22

Fig. 9

Fig. 10

4.

Experimental Results

4.1.

Measurements of Fluorescein and Coumarin 6 Using SPADs

Fig. 11

Table 2

4.2.

Measurements of ANS in Water/Methanol Using PMTs

Fig. 12

Table 3

5.

Conclusion

Eq. 23

Eq. 24

Acknowledgments

References

Show All Keywords

Keywords/Phrases

Search In:

Publication Years