Over the past decade, wide-field, noncontact biomedical imaging has made tremendous progress toward solving relevant in vivo clinical problems. Straightforward techniques such as continuous wave (CW) fluorescence imaging have already been translated to the clinic and can provide surgeons with real-time guidance during surgical procedures.1, 2, 3, 4, 5, 6, 7 More complex techniques employing time-varying sources and image collection have been used to image fluorescence lifetime and endogenous tissue optical properties [i.e., absorption and reduced scattering coefficients] in vivo, and over large fields of view (FOVs).8, 9, 10, 11, 12 Spatially modulated (i.e., structured) illumination has also been successfully used to image tissue optical properties in vivo, and over large FOVs.13, 14, 15, 16 When combined with multiwavelength analysis, the imaging of optical properties can be used to characterize the biochemical composition of tissues and thus provide a useful tool for noncontact, wide-field tumor delineation.17, 18
Although the results obtained using these techniques are quite promising, the field is still struggling with their translation into clinical environments. One problem of major impact to clinical translation is motion. This problem, also faced by other clinical imaging techniques such as magnetic resonance imaging (MRI), positron emission tomography (PET), computed tomography (CT), and optical coherence tomography (OCT), leads to random image location in space, impairs proper focusing, and generates blurred images when image acquisition time is long. This results in considerable error when performing qualitative localization of structures in a single image, and significant processing errors when applying quantitative techniques to a number of images acquired over time. For example, in the context of quantitative optical imaging, techniques such as fluorescence lifetime measurement and optical property extraction require that sequential images be acquired and co-registered in space. Not fulfilling co-registration renders the techniques unusable and prevents translation to a clinically realistic environment.
Several techniques have been developed to reduce motion-induced image noise, with most based on the acquisition and processing of physiological signals such as from the electrocardiogram (ECG) and/or plethysmograph (i.e., respiration). Solutions have been described and validated for MRI, CT, PET, and OCT.19, 20, 21, 22, 23 Different approaches have been described for the processing of the acquired signal(s) and the generation of a reliable gating phase and are reviewed in Ref. 24. One such software solution was initially implemented by our group,25 but did not provide the desired accuracy due to overwhelming of the processor with multiple camera streaming and computationally intensive signal processing. Other techniques involve the acquisition of several images at random times followed by postacquisition filtration to remove the undesired nonfocused images.22, 26 While this approach has its usefulness, it is obviously not suited for real-time qualitative and quantitative optical imaging. Hardware solutions are therefore preferred for real-time imaging. Although motion-gating capabilities are included in many commercially available MRI and CT systems, there are currently few off-the-shelf hardware devices available that could be used for in vivo optical imaging from mouse to human. Of special concern to in vivo optical imaging are the relatively long exposure times, usually greater than for fluorescence, and the wide range of physiological frequencies ranging from human respiration to mouse heartbeat .
In this study, we hypothesized that it should be possible to construct a hardware-only gating device that could accommodate all of the common pseudo-periodic and nonperiodic signals arising in a typical clinical environment and to produce a precise acquisition window compatible with any type of optical imaging application.
Materials and Methods
Gating Algorithm and Circuit
The gating algorithm was designed to provide a synchronized acquisition gate, or “acquisition window,” derived from an incoming signal. Here, we describe the synchronized acquisition of images captured by a camera with respect to an incoming physiological signal. The physiological signal was assumed to be pseudo-periodic and directly linked to the motion of the image being acquired. Thus, one can use the previous period to predict the width of the following period. It was then possible to acquire the phase and period of the physiological input signal and to generate a user-controlled acquisition window in real time. Various techniques are available for performing this operation and are reviewed in Ref. 24 The most commonly used techniques are periodic threshold detection coupled with differentiation filtering, digital filters, or pattern recognition.
The following is an overview of the electronic circuit used by our gating device, with more detailed information available at www.frangionilab.org. First, the incoming analog signal ( maximum) was conditioned using a precision differential instrumentation amplifier, low-pass filter, tunable high-pass filter, and tunable gain to provide input to the circuit. Phase was then assessed through threshold detection and differentiation filtering. Threshold detection ensured that no other extrema were incorrectly identified as the phase of the signal. Threshold detection was also automatically scaled to the previous period’s maximum to accommodate various types of incoming signals. It is important to note that a single-threshold approach does not guarantee a completely accurate time detection of the phase due to changes in the shape of the signal. The addition of a differentiation filter ensures an accurate and repeatable detection of the signal’s phase by the means of extremum detection [in the case of ECG, the peak of the R wave in the QRS complex; Fig. 1a ]. Last, the threshold and extremum detection signals were combined together into a single transistor-transistor logic (TTL) pulse per period indicative of the incoming signal’s phase. The phase pulse was then used to reset a timing sequence, which controls the delay and width of the acquisition window [Figs. 1a and 1b].
The timing sequence was generated using a field-programmable gate array (FPGA; Xilinx Spartan 3AN). The schematic of the timing sequence within the FPGA (programmed by the Boston University Electronics Design Facility, Boston, Massachusetts) is shown in Fig. 1c. The phase detection pulse was used to launch a timing sequence generated by a user-controlled timing clock. In order to accommodate a wide range of signal frequencies with high timing accuracy, two counters were used: one defining the start of the acquisition window (referred to as the “delay” relative to the phase signal) and the other defining the end of the acquisition window (referred to as the “width”). The desired delay and width could be entered as absolute values, representative of the units of the timing clock, or could be scaled automatically as a percentage of the previous period. The latter assumes that signal period and physical displacement are proportional, which permits compensation for variability in frequency (e.g., with respiration), as can occur with a normal ECG. The final acquisition window was a jumper-selectable TTL or complementary metal-oxide semiconductor (CMOS)-compatible pulse.
Gating device components were from DigiKey (Thief River Falls, Minnesota) and Mouser (Mansfield, Texas). Four-layer printed circuit boards (PCBs) were fabricated by Nashua Circuits (Nashua, New Hampshire) and assembled by Sure Design (Farmingdale, New Jersey). Gerber files for PCB manufacturing, a parts list, assembly instructions, and a user manual are available at www.frangionilab.org.
In Vitro Validation Experiments
We performed two types of in vitro validation of the gating device. Electrical validation was performed to test production of a reliable acquisition window from one or more periodic input signals, while functional validation was performed to test camera acquisition of an image in cyclic motion.
Input signals were generated using a Hewlett Packard (HP) model HP3245A dual-channel signal generator. One channel was used to produce a higher frequency triangular wave (e.g., ECG-like) with frequency , peak-to-peak (Vpp), and 10% duty cycle, while the other was used to produce a lower frequency sine wave (e.g., plethysmograph-like) with frequency , , and 50% duty cycle. The lower frequency signal was also added to the higher frequency signal using a model ZFRSC-42 voltage combiner (Mini-Circuits, Brooklyn, New York) to simulate the type of slow baseline drift typically seen in clinical tracings. Testing frequencies ranged from 60 to 600 beats per minute (bpm). Circuit inputs and outputs were monitored using a LeCroy WaveSurfer 44Xs, digital oscilloscope. Power for the gating device was provided by a , model WM220-1 (Elpac, Irvine, California) power supply.
To test functionality of the gating device, we constructed a motion simulator capable of moving an optical phantom using an arbitrary analog voltage input. The motion simulator consisted of a custom vertical translation stage (Microvideo Instruments, Avon, Massachusetts) that could be tilted to produce , , and motion, a model VL23-020D-ZAA DC servo motor (Applied Motion, Watsonville, California), and a model BLU-200 (Applied Motion) motor controller. Gated image acquisition was performed on a single PC with the acquisition window triggering a National Instruments (Austin, Texas) PCI-6229 multifunction board. Images were acquired, displayed, and saved using custom LabView software (National Instruments) capable of simultaneously acquiring two Firewire cameras [color and near-infrared (NIR) fluorescence] with triggering from the multifunction board. The cameras used were an Imitech (Seoul, Korea) IMC-80F for color video and a Hamamatsu (Bridgewater, New Jersey) Orca-AG for NIR fluorescence. Cameras were mounted to custom optics as described in detail previously.27 Color camera exposure time was and NIR camera exposure time was . The optical phantom consisted of a 384-well plate filled with either indocyanine green (ICG; NIR fluorescent) in dimethylsulfoxide or 2% skim milk (visible with color camera). White and NIR fluorescence excitation light was provided by custom LED modules.28
In Vivo Validation Experiments
In vivo validation of the device was performed during NIR fluorescence angiography of the beating swine heart using intravenously injected ICG ( per injection) as the contrast agent. Animals were studied under the supervision of an approved institutional protocol. Yorkshire pigs weighing and of either sex were purchased from E. M. Parsons and Sons (Hadley, Massachusetts). Anesthesia was induced using intramuscular Telazol (Fort Dodge Labs, Fort Dodge, Iowa), and maintained through a endotracheal tube with 1.5% isoflurane/balance at . The heart was exposed for imaging by a midline sternotomy. After each study, anesthetized pigs were euthanized by rapid intravenous injection of of Fatal-Plus (Vortech Pharmaceuticals, Dearborn, Michigan).
The ECG signal was obtained through standard three-lead electrodes attached to a Datex/Ohmeda model S/5 vital signs monitor. The acquisition window from the gating device was used to trigger image acquisition by custom LabView software. Color camera exposure time was , and NIR camera exposure time was . Color and NIR images were acquired with gating on and off, and regions of interest over the left anterior descending artery and the great cardiac vein were quantified over time for NIR fluorescence intensity.
An assembled gating device is shown in Fig. 2a . The assembled PCB could be mounted in an enclosure, controlled via manual switches, and operated as a stand-alone device or operated in a fully automated fashion using the integrated USB port. Indeed, the gating device was designed with dimensions of H, permitting up to two stacked devices to be mounted in a CA-1000 enclosure (National Instruments) and thus connected seamlessly to a multifunction board. The total price for a single prototype device was $1,000, which included $200 for board fabrication, $300 for components, and $500 for manual board assembly. When manufactured in larger quantities, the price per device fell to less than $300.
To produce a single acquisition window from multiple simultaneous signals, such as from simultaneous cardiac and respiratory gating, devices were designed to be stackable [Fig. 2b]. When stacked, the acquisition window from the lowest device in the stack outputs the logical AND of the acquisition windows from all gating devices above it [Fig. 3a ], although individual acquisition windows are also available if needed. Each stacked device has its own selectable input frequency range (Table 1 ) chosen from (cpm), , and . These frequency ranges cover the entire spectrum from mouse ECG to human plethysmograph. In addition, each board has a TTL- and CMOS-compatible input to suppress the acquisition window in the setting of nonperiodic signals, such as from involuntary or voluntary muscle movement detected using an accelerometer [Fig. 2a]. As suggested in the figure, such signals might need to be inverted depending on the sensor output.
Setting the gating circuit frequency range based on the species of interest and source of motion-induced noise.
|Frequency rangeselection (F1F2 bits)||Working range(cycle perminute)||Species||Signal type|
In Vitro Validation Experiments
By selecting the appropriate frequency range (Table 1), we tested operation of the gating device with input frequencies from (cpm). Electrical outputs from a typical experiment are shown in Fig. 3b. Notice that a feature built into the circuit is the ability to remove slow baseline drift during signal processing. For each type of input signal, a stable and reliable acquisition window could be generated and ANDed to produce an acquisition window that fit multiple criteria. For all frequency ranges, timing resolution was , and jitter was of the input signal’s period. This high precision was the result of employing a timing clock and counters within the FPGA, which reduced synchronization jitter between the phase pulse and timing clock.
To simulate a beating heart, a motion simulator [Fig. 4a ] that moves a stage vertically and laterally in response to an analog signal was constructed. To simulate an ECG, the input signal was a triangular wave of and 10% duty cycle [Fig. 4b]. As shown in Fig. 4c and Videos 1 and 2 (gating off and gating on, respectively), the gating device dramatically improved both color video and NIR fluorescence image acquisition. Indeed, with gating on, the images appeared crisp, co-registered, and motionless except for an occasional slight phase shift due to software overhead.10.1117/1.3275473.110.1117/1.3275473.2
In Vivo Validation Experiments
The beating heart of a pig is the closest approximation to a human heart and represents a worst case scenario for optical imaging since each systole results in over of vertical, lateral, and horizontal displacement. Nevertheless, as shown in Fig. 5 (top) and Videos 3 and 4 (gating off and gating on, respectively), the gating device was able to “freeze” image acquisition during NIR fluorescence angiography such that even the smallest arteries and veins were visible and could be imaged as if they were immobile. In fact, the only blurring that occurred was the result of the exposure time necessary for NIR fluorescence imaging.10.1117/1.3275473.310.1117/1.3275473.4
Importantly, in addition to qualitative improvements in optical imaging, the gating device also enabled quantitation of the NIR fluorescence signal emanating from the arteries and veins of the heart. Shown in Fig. 5 (bottom) is the ratio of the NIR fluorescence signal arising from the left anterior descending coronary artery and the great cardiac vein. In the absence of gating, considerable noise exists in this measurement due to blurring and is seen as a signal of standard deviation superimposed on the artery/vein ratio. However, in the presence of gating, the noise of the measurement is reduced to standard deviation, and the various phases of vascular filling can be identified readily without interference from the heartbeat.
The gating device we describe in this study has been shown to provide reliable and stable acquisition windows from one or more pseudo-periodic input signals. Although the device adapts for slow changes in period and amplitude, fast and large variations in period render the incoming signal nonperiodic and thus not interpretable to the device. Special care should be given to deliver a clean, spike-free analog input signal by avoiding moving the electrode wires, or using electrocautery, during gated image acquisition. Additional software modifications can also be used to automate frequency range and gain selection via the TTL error signals provided by the device.
As described earlier, the gating device accommodates slow periodic variations by stretching the acquisition window proportionally to the incoming signal’s prior period. This approach assumes that the electrical signal and mechanical motion stretch proportionally with period, a reasonable assumption with respect to electrophysiological signals such as ECG and respiration.
One critical point when using the gating device is optimization of the camera acquisition software. The simple LabView code used in this study was not able to acquire two or more cameras with perfect simultaneity, which resulted in occasional phase shifts and image blurring. Newer software developed for our Fluorescence-Assisted Resection and Exploration [FLARE™] image-guided surgery system is capable of truly simultaneous acquisition of up to three Firewire cameras7 and is expected to solve this problem. Whenever possible, camera exposure time should be as short as possible relative to the physiological signal period (typically lower than 10%), which will help eliminate blurring and improve quantitation. Stacked gating devices and multiple input signal ANDing can also help reduce noise by removing superimposed mechanical motions. Last, hardware triggering of camera acquisition using the gating device window status bit (1: acquire; 0: do not acquire) is always preferred. Using this mode, motion-gated imaging is limited only by the minimum exposure time of a typical NIR camera (e.g., Hamamatsu ORCA C4742-80-12AG) since the gating device provides a timing resolution and a jitter of the input signal’s period. In all circumstances, it is the camera readout time ( for the ORCA with binning) that will be rate limiting.
This study will hopefully contribute to the clinical translation of real-time quantitative optical imaging techniques that are now limited by periodic or pseudo-periodic motion artifacts. For example, as fast parallel computing of image pixels using graphics processing units (GPUs) becomes more commonplace, CCD camera acquisition becomes the critical parameter for permitting real-time imaging. Motion-free acquisition of these images should enable improved quantitation.
We envision that the gating device could be employed to improve quantitation of virtually any optical imaging methodology. One technology that could benefit immediately is NIR fluorescence for image-guided surgery. Although this technique has already been translated to the clinic,1, 2, 3, 4, 5, 6, 7 it is presently limited by the harsh mechanical motions encountered in the operating room. These include those caused by the beating heart, ventilated chest, and inadvertent “bumps” from caregivers. The stackable gating device we describe can suppress all such contributors to image noise since an unlimited number of pseudo-periodic and nonperiodic signals can be combined to control the acquisition window, greatly improving the visualization of fine structures of interest. It should also be emphasized that no part of the human body is immune from physiological motion, and that every surface of the body, including the toes, moves a perceptible and measurable distance with every heartbeat. The higher the resolution and narrower the depth of field of the optical imaging technique employed, the more such movement contributes to image noise.
Gated acquisition might also be useful for contact point measurements. Since point measurements can be influenced by applied pressure29 and blood volume effects due to pulsatile arterial flow, the combination of both motion gating for heartbeat and pressure-level gating using the gating device’s provided nonperiodic signal input could improve noise reduction in such measurements.
Last, our gating device is not limited to the field of biomedical optics since it provides considerable versatility for use in most synchronized acquisition measurements. For example, processing filters can be changed during assembly to accommodate other types of periodic signals, phase detection can be provided externally as needed, and timing clock frequency can be changed within the FPGA code or provided externally (see www.frangionilab.org).
We have developed a low-cost, computer-controlled, and stackable gating device that permits an unlimited number of pseudo-periodic and nonperiodic signals to be combined to generate a precision acquisition window for optical imaging and image-guided surgery. Performance of the device has been validated in vitro using the extremes of frequency encountered during small animal imaging and image-guided surgery, and in vivo using beating hearts that match the electrical, mechanical, and optical properties of humans.
We thank Jerome Mertz (Boston University Department of Biomedical Engineering) for many helpful discussions; Eric Hazen and Paul Bohn (Boston University Electronics Design Facility) for FPGA programming; Summer L. Gibbs-Strauss, Aya Matsui, and Rita Laurence (Beth Israel Deaconess Medical Center) for assistance with animal experiments; Summer L. Gibbs-Strauss for editing; Jim Cuthbertson (Nashua Circuits) for PCB fabrication; Ken Thomas and Fernando Irizarry (Sure Design) for PCB assembly; Mike Paszak and Victor Laronga (Microvideo Instruments) for construction of the motion simulator; and Eugenia Trabucchi and Lorissa A. Moffitt for administrative assistance. This work was supported by NIH Grant No. R01-CA-115296 to Dr. Frangioni.