1.

Introduction

An important challenge when imaging developing embryos is the continual motion of the heart. This presents considerable challenges for 3-D optically sectioned imaging, laser targeting of individual cells or cell clusters, and tracking of individual cells throughout the development of the organism. The motion of the heart itself is also of real physiological interest.

The specific problem with 3-D optically sectioned imaging of the living, beating heart is that every slice must be acquired when the heart is at the same point in its cycle. Without synchronization to the motion of the heart, the individual image slices cannot be combined together to form a consistent 3-D model of the heart.

This gating problem has previously been addressed in optical microscopy by using postprocessing approaches: continuous images of the beating heart in the desired imaging modality (e.g., confocal fluorescence imaging) are acquired at a high frame-rate over the course of one or more heartbeats, and subsequent postacquisition analysis identifies the frame that is closest to the desired point in the heart's cycle^{1, 2, 3} (e.g., selecting frames corresponding to ventricular end diastole). These “best choice” images are then used in the 3-D reconstruction. Such an approach results in considerable exposure of the sample to fluorescence excitation light, at least an order of magnitude greater than that which is strictly necessary for acquiring a single gated fluorescence image per z slice, as well as requiring particularly high-specification cameras to acquire reasonable quality fluorescence images at high frame rates. By imaging over multiple successive heartbeats, it is possible to increase the probability of obtaining a frame close to the desired position in the heart's cycle, but at the cost of increased acquisition time and exposure of the sample to laser light.

Our approach eliminates these problems by performing real-time prospective gating to generate a trigger signal when the heart is in a position selected by the user, causing a single fluorescence image to be acquired. Our algorithm analyzes frame data streamed from a bright-field imaging camera in real time in order to calculate when this trigger should be sent. In this way, our synchronization system ensures that a high-quality frame is acquired at the correct point in the heart's cycle rather than requiring continuous high-frame rate acquisition to make sure that one of the frames acquired will be at approximately the correct position in the heart's cycle.

Such an approach also opens up the future possibility of triggering localized photoactivation, transfection, or ablation within the beating embryonic heart. These are all attractive techniques in developmental biology,^{4, 5, 6, 7, 8} but in cardiac applications it is necessary to trigger the laser when the heart is at a known point in its cycle or, otherwise, the (moving) target cells will not be at the focus of the laser beam.

Existing prospective gating techniques are not well suited to studies involving early embryos. In humans and other large organisms an electrocardiogram can be used, but this is not practical here due to the small physical size of young mouse and zebrafish embryos. In magnetic resonance imaging (MRI), the electrocardiogram can be used,⁹ although it is liable to be distorted by the magnetic field. Alternative approaches that do not suffer from this issue include optical postacquisition processing using a simple metric based on the amplitude at the k-space center,¹⁰ and gating using a fiber-optic stethoscope.¹¹ However, despite the continued growth in computing power, such interventional approaches remain challenging, and high-resolution “prospective” gating performed in real time has only recently been demonstrated,^{12, 13} with latencies that are far too great to give the high-precision results required for optical microscopy imaging, particularly in small animal embryos. Prospective gating in MRI remains an open problem.¹⁴ In robotic heart surgery, sophisticated video-based gating methods are used, but these rely on highly complex control systems that must still make assumptions about the regularity of the heartbeat, limiting the precision with which they can track irregularities in the heart's periodicity and rendering them incapable of resolving rapid, small changes in the heart's motion.¹⁵

Despite the high image resolution required, in the optical imaging of small animal embryos the challenges are considerably reduced in many ways. In particular, a full-frame bright-field image can be acquired at frame rates on the order of 100 Hz, there is no respiratory motion to compensate for, and the challenges present in robotic heart surgery (such as occlusions and specular reflections) are not an issue. This means that with a well-designed and efficient implementation it is realistic to envision a low-cost real-time synchronization system with extremely precise gating within the heart cycle.

This real-time operation necessarily requires that our image processing is less computationally demanding than that previously reported by other authors for postacquisition processing.¹ Furthermore, our design calls for the use of trigger signals with millisecond precision, which requires some careful technical consideration (see Sec. 3.3). Our present system has a closed-loop response time that can be as little as 10 ms for acquisition of gated fluorescence images in real time (with the response time mostly limited by the bright-field camera readout speed). This ability to send appropriate trigger signals in real time has many important advantages, as follows:

One important feature of postprocessing techniques, which our results as presented here do not offer, is the ability to retrospectively reconstruct images of any or all positions in the heart's cycle. This is a significant advantage, making it possible, for example, to show an animation of the full heartbeat cycle. One possible way of addressing this shortfall in the technique we have demonstrated here would be to develop a hybrid approach. In postprocessing, a free-running camera acquires images continuously, whereas in our technique one frame per cycle is triggered. There is no fundamental reason why our algorithm cannot generate triggers at multiple different points within the heart's cycle. This could for example generate precisely 20 exposures equally spaced in phase over one heartbeat, and in the case of 3-D imaging these exposures would be mutually consistent across z slices. This would yield data over the whole heart cycle while still minimizing exposure to excitation light. Outwardly, this would appear very much like the free-running data acquisition used in postprocessing approaches, but with the trigger signals providing slight “adjustments” to the precise exposure times to ensure consistency between the single cycles of data acquired for each z slice.

This paper first gives an overview of our real-time heart synchronization platform (Sec. 2). We then discuss the details of the algorithm it uses (Sec. 3) and demonstrate the use of our system for triggered acquisition of sectioned fluorescence images (Sec. 4).

2.

A Real-Time Heart Synchronization Platform

Figure 1 shows a schematic diagram of how the technique can be integrated into a complete imaging system. One free-running monitoring camera acquires bright-field images at a relatively high frame rate of 50–100 Hz, for input into the heart-gating algorithm. This camera does not necessarily need to be a high-cost, extremely high-performance camera; indeed, we have used a low-cost Prosilica GS650 camera with great success. As each frame is received from the camera by the PC, it is analyzed (see Sec. 3) in order to quantify its phase (position in the heart's cycle), extrapolate forwards in time and determine when the heart's spatial structure will next be as required.

Fig. 1

Schematic diagram of a complete synchronized imaging system. A free-running camera acquires bright-field images continuously, and a second fluorescence camera is triggered to acquire gated frames only at the appropriate calculated times. The excitation laser is also triggered, only being turned on when the fluorescence camera is exposing a frame.

The second camera is triggered at this predicted time, acquiring the high-quality images that are of biological interest (which may be fluorescence images). Only when this second camera is triggered is the fluorescence excitation laser turned on, thus minimizing photobleaching effects.

The monitoring camera acquires bright-field images even in experiments where gated fluorescence images are desired from the second camera. The camera used for fluorescence imaging does not need to operate at high frame rates. As well as having cost implications, this advantage enables us to tailor our choice of cameras to the two very different requirements of the imaging problem. The gating problem requires image data at reasonably high frame rates for motion analysis, but these frames do not require fine structural detail or exceptional quality. On the other hand the ultimate output from the problem, optically sectioned fluorescence images for 3-D reconstruction, calls for maximization of image quality. This is best provided by a high quantum efficiency image sensor with relatively slow readout electronics. By dividing the task between two cameras, both these aspects of the problem can be addressed in an optimal manner.

The timing error associated with an individual trigger signal is largely determined by two factors. The first is the accuracy with which a phase can be assigned to each frame as it is received from the camera (which will be discussed in Sec. 3). The second is the accuracy with which these phases can be extrapolated forward in time to determine the required trigger time. The length of this forward extrapolation is closely tied to the latency of the system: the minimum possible interval between the acquisition of a frame by the monitoring camera and a calculated electrical trigger signal being sent. This interval will include the time taken to read out the frame from the CCD, the time required to analyze the frame data, and communication delays. Note though that with good forward prediction the trigger precision in the case of a stably beating heart (we have achieved 0.25 ms jitter relative to the hypothetical “correct” trigger time, see Sec. 4) may be considerably smaller than the latency (10–15 ms in our case).

As well as being used to trigger the imaging camera and a fluorescence excitation light source, the trigger signal could for example also be used to integrate the system seamlessly into a commercial laser ablation workstation. The user would interact with their existing laser control computer interface as normal, but the gating system would intercept the trigger signals sent to the ablation laser, delaying the signals until the appropriate point in the heart cycle. This opens up the possibility of precision targeting of moving cardiac cells without resorting to a “trial-and-error” approach involving a large number of samples.

3.

Algorithm

This section discusses the algorithms and control systems used to implement the synchronization platform. The key to the synchronization technique is in quantifying the relationship between the spatial and temporal properties of the heart. The algorithm interprets still images of the heart (spatial properties/shape) in order to build up a precise model of the temporal behavior of the heart (heart rate and instantaneous stage in the heart's cycle). This temporal model is then used to predict when the heart will next be in the required position when an image should be acquired (spatial structure again).

The gating algorithm consists of a number of building blocks, as follows:

The first three steps are performed on a desktop computer and the final step is performed with the help of a low-cost, custom hardware timing controller.

3.1.

Frame Comparison

The techniques described here rely on the ability to compare two frames taken by the same camera from the same viewpoint and to quantify their similarity. All that is required is a scalar comparison metric ∂_fg representing the similarity between two frames f and g. Two commonly used and closely related metrics that perform surprisingly well given their simplicity are sum of absolute differences (SAD) and variance (Var).¹⁷ Very similar metrics have been used by other authors for motion detection in brightfield heart images, see for example.¹⁸

For two frames f and g each consisting of n pixels f _i and g _i, the metrics are defined as follows:

Eq. 1

[TeX:] \documentclass[12pt]{minimal}\begin{document} \begin{eqnarray} \rm{SAD:\ \ \ } \partial _{fg} &=& \sum _{i=1}^n |f_i - g_i|, \nonumber \\ \rm{Var:\ \ \ } \partial _{fg} &=& \sum _{i=1}^n (f_i - g_i)^2. \end{eqnarray} \end{document} $$\begin{array}{ccc}\hfill \mathrm{SAD}:{\partial }_{\mathrm{fg}}& =& \sum _{i=1}^n|f_i-g_i|,\hfill \\ \hfill \mathrm{Var}:{\partial }_{\mathrm{fg}}& =& \sum _{i=1}^n{(f_i-g_i)}^2.\hfill \end{array}$$

In both these cases, a small value of ∂_fg indicates frames that are very similar to each other. The simplest application of the metric ∂_fg is to determine the instantaneous period of the heartbeat. We define the instantaneous period τ(t) as the time interval such that the heart at time t “looks the same” as it did at the earlier time t − τ(t). The more physiologically familiar measure, heart rate, is 60/τ(t) bpm. If the heart follows exactly the same trajectory for every beat, but with a speed that may vary over time, then our definition of τ(t) is a precise one. In practice, there will be some variation in trajectory between beats and we must select the previous point in time when the heart looks most like it does at the present moment in time.

Figure 2 shows the SAD between a specific frame and the frames preceding it. It can be seen that there is a strong periodicity to this signal, from which the period τ(t) can be determined. It is necessary to estimate the period in a manner that is robust in the face of realistic SAD signals. In particular, a “double-dip” effect is often observed (see Fig. 3), where due to the complex motion of the heart there is a temporary small dip in the SAD signal that does not indicate one complete period. It is preferable that the algorithm should not require any a priori knowledge of the heart period, and Fourier analysis is not appropriate because we may be interested in tracking a varying heart rate and/or have only one full heartbeat's worth of data available. We use a heuristic trough-finding algorithm to find the period.

Fig. 2

SAD between a specific frame and the frames preceding it, for a bright-field image sequence of a beating zebrafish heart. The crosses indicate differences between frames sampled at 50 fps. For reference, the continuous line shows the differences calculated from a 150-fps video, as a close approximation to the “continuum” variation of the heart's appearance over time. A small value for the SAD indicates frames that are similar in structure; the sharp troughs in the plot represent those points in time when the heart is in the same position as it is in the frame at Δt = 0 (i.e., the same phase in previous heartbeat cycles).

Fig. 3

Trough-finding heuristic with no a priori knowledge of the period. The heuristic (defined in the main text) looks for a minimum in the comparison metric (frame i) that lies below the lower threshold level and is followed by a rise in the metric above the upper threshold level (both threshold levels, defined in the main text, are marked as dotted horizontal lines on the diagram). Because the thresholds are a function of the frame indices h, i, and j (among others), local minima such as that seen to the right of h in the diagram will not satisfy the criteria and will not be erroneously identified as matching.

The double dip mentioned previously is caused by the fact that, for example, an image of the half expanded heart looks similar to that of the half-contracted heart. In the simplified case of a rhythmically expanding and contracting sphere, this would cause problems for the algorithm, because two different points in the cycle would look entirely identical. However, in practice the heart undergoes a more complex motion (either two chambers or a peristaltic tube, depending on the stage of development) and while images from some points in the cycle may be more similar than others, this only causes a small double-dip effect rather than more serious problems. This effect is reduced by using a sufficiently large depth of field for the bright-field imaging that both chambers of the heart can be seen simultaneously. We find that the form of this curve changes little with position in the heart cycle, other than the fact that the position of the double dip (if any) changes.

The trough-finding heuristic is presented graphically in Fig. 3. It identifies the period through a procedure based on identifying frames whose comparison metrics ∂_fg lie above or below two threshold values [defined by Eqs. 2, 3, respectively], which in turn depend on the value of frame index j (as defined below). The design of the heuristic makes it effective at rejecting the double-dip effects discussed earlier, which we found to be the greatest obstacle to a robust determination of the period with no a priori information and a potentially limited frame history for analysis.

The heuristic is formally defined in terms of the comparison metric between the most recent frame f and a series of earlier frames {∂_fj ... ∂_{f, f−1}}. First, the most recent frame j satisfying the following criteria is identified:

Eq. 2

[TeX:] \documentclass[12pt]{minimal}\begin{document} \begin{eqnarray} \partial _{fi} &<& \partial _{f,f-1} + c_1 \,\times\, (\partial _{fh} - \partial _{f,f-1}), \end{eqnarray} \end{document} $$\begin{array}{ccc}\hfill {\partial }_{fi}& <& {\partial }_{f,f-1}+c_1\times ({\partial }_{fh}-{\partial }_{f,f-1}),\hfill \end{array}$$

Eq. 3

[TeX:] \documentclass[12pt]{minimal}\begin{document} \begin{eqnarray} \partial _{fj} > \partial _{fm} + c_2 \,\times\, (\partial _{fh} - \partial _{fm}), \end{eqnarray} \end{document} $$\begin{array}{c}\hfill {\partial }_{fj}>{\partial }_{fm}+c_2\times ({\partial }_{fh}-{\partial }_{fm}),\end{array}$$

where

[TeX:] \documentclass[12pt]{minimal}\begin{document} \begin{eqnarray*} && f > h > i > j,\\ \partial _{fh} &=& {\rm max} \lbrace \partial _{fj}, \ldots \partial _{f,f-1} \rbrace,\\ \partial _{fi}\break &=& {\rm min} \lbrace \partial _{fj} \ldots\partial _{fh} \rbrace,\\ \partial _{fm} &=& {\rm min} \lbrace \partial _{fj} \ldots \partial _{f,f-1} \rbrace. \end{eqnarray*} \end{document} $$\begin{array}{ccc}& & f>h>i>j,\hfill \\ \hfill {\partial }_{fh}& =& \max \{{\partial }_{fj},...{\partial }_{f,f-1}\},\hfill \\ \hfill {\partial }_{fi}& =& \min \{{\partial }_{fj}...{\partial }_{fh}\},\hfill \\ \hfill {\partial }_{fm}& =& \min \{{\partial }_{fj}...{\partial }_{f,f-1}\}.\hfill \end{array}$$

Here, the two inequalities define the lower and upper thresholds shown in Fig. 3. Frame h appears in the expressions for the threshold levels and has the largest comparison metric value ∂_fh of all frames between f and j, frame i has the lowest comparison metric value ∂_fi of all frames between h and j, while frame m is defined slightly differently, having the lowest value of all frames between f and i, and is used in determining the threshold levels. All three are dependent on the value of j [i.e., h = h(j), i = i(j), m = m(j)].

The closest matching frame is then i, and the output of the algorithm is τ_est = t _f − t _i. In the above criteria c ₁ and c ₂ are constants between 0 and 1. We have found the values c ₁ = 0.5 and c ₂ = 0.75 to work well with a wide range of samples, viewpoints, and frame rates. It should be noted that the above criteria cannot necessarily be satisfied for an arbitrary data set. However we have not encountered any problems at all when applying the heuristic as described to video of a range of early zebrafish embryos.

This heuristic can determine the period to the nearest integer number of frames, but a greater accuracy than this is required for precise prospective gating. In order to achieve this, it is necessary to interpolate between the available frames. For this a three-point fit to a V function is used (see Fig. 4). The three points are samples of the continuously time-varying function ∂_cont representing the difference between the current appearance of the heart and its appearance as depicted by our reference frame. We make the good approximation that that underlying function has a symmetric V profile around its minimum, and therefore that a three-point V interpolation will provide a very good estimate of where the current frame fits into the sequence of reference frames (to subinteger accuracy within the reference frame list). We have found that the profile of this underlying function is very close to a symmetric V profile for both bright-field and selective plane illumination microscopy (SPIM) fluorescence images, which is why we have chosen this candidate function as opposed to alternative functions such as a quadratic function.

Fig. 4

V fit used to estimate the subframe period of the heart. The ten black dots show the SAD values for each frame (acquired at 50 Hz), which sample a continuously time-varying function like the experimental data shown in Fig. 3. The red solid line shows the three-point V fit used to determine the subframe period. For reference, the dotted (blue) line shows the SAD values for actual experimental data acquired at 150 Hz, to serve as a close approximation to the continuum variation of the heart's appearance over time. Note that the form of the dotted blue line confirms that the temporal variation of the signal is well suited to interpolation using a V model.

This three-point fit is formally defined as follows: for three values u ₀, u ₁ and u ₂ at times −Δt, 0, and Δt, respectively (where u ₁ has the smallest value, ∂_fi), the interpolated period is

Eq. 4

[TeX:] \documentclass[12pt]{minimal}\begin{document} \begin{equation} \tau = \tau _{\rm est} + \frac{\Delta t}{2} \frac{u_0 - u_2}{{\rm max}(u_0, u_2) - u_1}. \end{equation} \end{document} $$\tau ={\tau }_{\mathrm{est}}+\frac{\Delta t}{2}\frac{u_0-u_2}{\max (u_0,u_2)-u_1}.$$

This subframe interpolation assumes that ∂_cont can indeed be locally modeled as a symmetric V function. Nonperiodic motion in the image (such as motion of red blood cells) is liable to reduce the effectiveness of this model. The effect of random noise on the input values for Eq. 4 is to bias the result away from zero, meaning that random noise will introduce small systematic errors to the frame comparisons. More sophisticated approaches might prove more noise tolerant, but the current accuracy appears to be more than sufficient for practical purposes, judging by the consistency of the results we present in Sec. 4. We emphasize, however, that the algorithm we have used has been developed specifically for application to cardiac imaging of zebrafish embryos, and we are not suggesting that it is universally applicable or that it could not be improved.

3.3.

Real-Time Triggering

Until now we have assumed that we can schedule an electrical trigger signal to be sent at a precisely specified time. However, this is not an easy thing to do on a desktop computer because a consumer operating system is not structured to guarantee that operations will take place at a precise moment in time. In order to address this problem, we implemented a custom hardware solution [using the Xilinx Spartan-3 Field-Programmable Gate Array (FPGA) chip¹⁹] to fulfill the real-time requirements of the imaging system. Our custom computer software (written in Objective C + +) receives and analyzes time-stamped images from the camera and anticipates the time at which the camera should be triggered. It notifies the FPGA timing controller (via an RS-232 data link) that a trigger should be sent at a time of precisely t _est. The timing controller will then send an electrical trigger pulse when that time is reached. In this way, all the image-processing requirements of the system can be implemented on a standard desktop computer while still meeting strict real-time timing constraints (see Fig. 5).

Fig. 5

Schematic timeline showing how the timing controller enables trigger signals to be scheduled at precise times despite the image processing being hosted on a (non–real-time) consumer desktop computer. As long as the commands are received in time, there are no strict timing requirements on the desktop computer.

The software must commit to choosing a trigger time t _est sufficiently far in advance that enough time remains to communicate that request to the timing controller. The trigger will then be sent at the precise moment calculated to match the heart position selected in advance by the user, and the only effect of communication/computational latencies within the system is a potential reduction in the quality with which that estimated time is determined (because in the presence of longer communication latencies it will be necessary to extrapolate further into the future to make the estimate, which may mean that that estimate is likely to be somewhat less reliable).

The analysis software must be configured to expect a latency that is sufficiently large that the communication will almost never take longer than the configured value. Occasionally, this expectation may be violated but this situation can be identified when it occurs and is reported by the timing controller as an error, with no trigger being sent. Such a protocol ensures that either a fluorescence image will be acquired (or laser trigger signal sent, etc.) at precisely the desired trigger time, or (occasionally) the trigger will not be sent for this particular heartbeat.

4.

Results Showing Real-Time Gated Heart Imaging

Here we present and analyze results obtained on a SPIM system. The design of the microscope we have constructed is based on that described in Ref. 20 and a schematic diagram of our system is shown in Fig. 6. SPIM enables high speed sectioned fluorescence image acquisition with high depth penetration, while minimizing the exposure of the sample to the excitation laser light.²¹ We demonstrate the use of images streamed from a bright-field camera to trigger a fluorescence excitation laser and cause a second fluorescence imaging camera to acquire a frame. The laser is only turned on and the sample exposed to the laser light when the fluorescence imaging camera is acquiring a frame. This is a proof of concept of our system, demonstrating a considerable reduction in laser exposure of the sample compared to that required for approaches involving postacquisition selection. We also show initial results for gated 3-D acquisition in the beating zebrafish heart.

Fig. 6

Schematic diagram of our SPIM system. The illumination light sheet originates from a Point Source IFLEX 488-nm laser and is shaped and steered using a sequence of lenses (see Ref. 20) before being launched into the sample through a Nikon CFI Plan Fluor 10xW 0.3NA water-dipping objective (Obj1). The fluorescence light is imaged using a Nikon CFI75 LWD 16xW 0.8NA water-dipping objective (Obj2) and a tube lens system (of effective focal length 425 mm) onto a QImaging QICam 12-bit digital camera (CCD1). Transmitted bright-field light is imaged onto a Prosilica GS650 digital camera (CCD2), using the same objective and a separate tube lens of focal length 100 mm. These bright-field images are processed by our gating algorithm in order to generate trigger signals for the QImaging camera. The laser and fluorescence camera are triggered using the timing controller described in Sec. 3.

All the images shown here are of transgenic four-day-old zebrafish embryos expressing green fluorescent protein within its cardiomyocytes tg(myl7:gfp).²² The embryos were immobilized with ethyl 3-aminobenzoate methanesulfonic acid (Tricaine^®) to reduce movement and contained within an index-matched tube of FEP polymer.²³ Although our intention is not to provide comprehensive biological analysis of the results here, we note that we have obtained good synchronization performance from a range of zebrafish samples in the range of 2–5 dpf in both sagittal and coronal orientations, in both commercial microscopes and our own SPIM system.

Figure 7 and Video 1 show simultaneous acquisition of bright field and fluorescence images. The low-resolution bright-field images of the beating heart are streamed continuously from the bright-field camera. They are analyzed by our synchronization system, which generates trigger signals to turn on the excitation laser and cause the fluorescence camera to acquire an image at the appropriate point in the heart's cycle.

Fig. 7

Still frame from video showing simultaneous acquisition of bright-field and (gated) fluorescence images. The bright-field images, acquired continuously at 100 fps, are shown on the left. These are processed by the gating algorithm in order to calculate when fluorescence image acquisition should be triggered to coincide with ventricular enddiastole. The triggered fluorescence images (one per heartbeat) are shown on the right of the image. The scale bars represent 25 μm. It can be seen from the video that there is no discernible variation in the heart's position from one acquired image to the next. For comparison, the video also shows frames identified by postacquisition selection of the best frame from each heartbeat for fluorescence images acquired at 36 fps. (Video 1, QuickTime, 5.3 MB)

This also illustrates the alternative technique of postacquisition processing, in which a single camera acquires continuous fluorescence images at a rate limited by the capabilities of the camera. These images are then analyzed, and the closest frame to the desired point in the heart's cycle is automatically selected for each complete heartbeat. As expected, there is inevitable variation between these frames (as noted earlier, it would be possible to increase acquisition time—and sample exposure—to span multiple heartbeats in order to improve accuracy). For this demonstration, a relatively low frame rate of 36 fps has been used to emphasize the contrast between the techniques. A practical application of the postprocessing approach would probably use a higher frame rate, but note that even at 36 fps the cumulative laser exposure is over ten times that of our prospectively gated approach.

Figure 8 shows kymographs illustrating the effectiveness of our gated fluorescence image acquisition. They show the variation in the heart's position in the “best” frames identified by postprocessing of a continuous stream of fluorescence images, and show that this variation is eliminated when frames are instead triggered by our gating algorithm.

Fig. 8

Kymographs showing effectiveness of gated fluorescence image acquisition. (a) Still fluorescence image of the heart. The vertical white bar indicates the profile used in the kymographs, and the horizontal scale bar represents 20 μm. (b) Kymograph of images acquired continuously at 36 fps over a period of approximately 10 heartbeats, showing amplitude of motion over the full heart cycle. (c) Kymograph of images selected by postprocessing from images acquired continuously at 36 fps (closest frame from each of 150 successive heartbeats is selected and shown in the kymograph). This shows the inevitable variation due to the free-running camera not acquiring a frame at quite the right point in the heartbeat, represented by a zigzag form of the kymograph. (d) Kymograph of images triggered on 150 successive heartbeats using our real-time optical gating system. The kymograph takes the form of an unchanging horizontal line, indicating that there is no discernible variation in the heart's position from one acquired image to the next.

Figure 9 shows a histogram representing the variation in calculated phase for the frames that were returned by postprocessing and real-time gating algorithms during successive heart imaging sessions. As expected, the frames identified through postprocessing show considerable variation because the closest available frame to the user-selected position will vary between half a frame early and half a frame late relative to that desired point in the heart's cycle. In contrast, the histogram for our real-time triggered approach shows an extremely small spread in calculated phase for the frames obtained. The width of this histogram represents a “jitter” of ∼0.25 ms in our trigger signals relative to the requested position of the heart (with the trigger signals generated through analysis of 100 fps bright-field images). In order to perform this analysis, we required an appropriate set of reference fluorescence images covering one full heartbeat, against which we could compare the triggered frames. Because the camera is only triggered once per heartbeat such frames are not directly available, and for this analysis we used the reference frames that were automatically extracted from the sequence used for the postprocessing experiment.

Fig. 9

Histograms showing variation within the heart's cycle for a sequence of frames at ventricular end systole returned for 180 successive heartbeats, as obtained using postprocessing and real-time gating algorithms. The frames obtained by postprocessing were extracted, one per heartbeat, from a video sequence of fluorescence images acquired at 36 fps. The fluorescence images obtained using our real-time gating system were triggered as a result of real-time analysis of bright-field images acquired at 100 fps. In each case, the jitter was analyzed by using our comparison algorithm to assign a phase in the heart's cycle to each frame. Given the extreme contrast between the very low jitter on the triggered images as compared to the postprocessed frames, different vertical scales have been used for each histogram in order to show them both clearly on the same graph.

Finally, Figure 10 shows initial results for gated 3-D imaging of the beating heart. This work is still ongoing, but these results serve to demonstrate that it is possible to use our technique for gated acquisition of a consistent 3-D image stack of the living, beating heart. The plane of the fluorescence image slices is scanned through the sample by moving the sample holder. At the same time, the tube lens for the bright-field imaging camera is automatically moved to maintain focus of the bright-field images at a constant z depth so that the synchronization analysis can continue unaffected by the sample motion. The slight change in magnification that this causes is corrected in software prior to the bright-field image analysis.

Fig. 10

Still frame from video showing 3-D reconstruction of the beating zebrafish heart at ventricular end systole. 120 separate z slices were acquired, with each exposure gated to the motion of the heart to ensure that a consistent 3-D stack was obtained. One image was acquired per heartbeat, meaning that the total dataset took 50 s to acquire. (Video 2, QuickTime, 0.5 MB)

5.

Conclusions

We have demonstrated the use of a real-time optical cardiac gating system for fluorescence imaging of live samples such as early zebrafish embryos. It is capable of exceptionally precise gating (we have demonstrated jitter as low as 0.25 ms). We have described, in detail, the heuristic algorithms we have used for image analysis and prospective gating. We have shown them to be extremely effective for synchronization based on bright-field images of the beating heart in early zebrafish embryos. The exact heuristics we used were specifically developed to perform well on images of this type, and while they might perform well with other modalities we have not yet investigated this.

We have presented initial results showing the application of this technique for prospectively gated 3-D fluorescence tomography of a living, beating zebrafish heart. The technique reduces the exposure of the sample to fluorescence excitation light by at least an order of magnitude compared to postprocessing approaches. Furthermore, the system has the potential to be used to trigger external equipment, such as laser ablation systems for targeting moving heart tissue, something that is only possible with such a real-time system.