2.1.1.

Source design

The light source used in this system combines a laser diode based NIR source with a plasma-based white light source for simultaneous illumination for both color and NIR imaging. The laser module houses four 2.5 W laser diodes operating at 760 nm (LDX-3215-760, LDX Optronics, Maryville, Tennessee) and operated using current and thermo-electric cooler controllers (ITC300, Thorlabs, Newton, New Jersey). Alternatively, the system can also use four 1 W laser diodes operating at 660 nm (LDX-3115-660, LDX Optronics, TN, USA) imaging NIR fluorophores with the emissions at shorter NIR wavelengths (e.g., MB). The white light illumination is generated by a high-power solid state plasma light source (HPLS-30-02, Thorlabs, NJ, USA) with an output power of 10.2 W (at 5 mm focal spot, 0.5 NA) in the visible wavelengths.

The two sources are combined using a custom-designed light coupler (mechanically mounted on the plasma light source) that shapes the output from the two sources before injection into a fiber-optic light guide with 4.8-mm core diameter (0.54 NA). The design of the light coupler is shown in Fig. 2. It should be noted that a dichroic mirror (T673lpxxr, Chroma Technologies, Bellows Falls, Vermont) was used in order to combine the two light sources. Furthermore, the light coupler includes a hot mirror (21002A, Chroma Technologies, VT, USA) and a 650 nm short pass filter (E650SP, Chroma, VT, USA) in order to filter out the plasma source output outside the visible window, thereby reducing the light leakage in the NIR images. The NIR laser module is connected to the light coupler via an SMA terminated 2 mm multimode fiber optic cable. The output port of the light coupler accepts a standard off-the-shelf 1.8-m long fiber optic light guide with a 4.8-mm core diameter (C3278, Conmed, Utica, New York). The use of a fiber optimized for NIR transmission could further improve the efficiency of the system. The light coupler operates at a coupling efficiency of 87% for the NIR light source and 35% for the white light source.

Fig. 2

Source coupler design. (a) Mechanical layout of the source coupler mounted on the custom white light source chassis. F1: IR filter. F2: 650SP filter. L1: 30 mm biconvex lens. L2: 25 mm biconvex lens. Di: T673LP dichroic mirror. L3: 25 mm biconvex lens. (b) Optical design of source coupler.

2.1.2.

Endoscope design

The system described here uses a custom rigid rod-lens based NIR optimized endoscope (Schoelly Fiberoptic GmBH, Denzlingen, Germany). The endoscope is 340-mm long and has a 10-mm bore diameter with a 5 mm imaging area and a 30 deg direction of view. The fiber optic light guide from the source coupler is connected to the endoscope via an Olympus endoscope fitting (7453, Conmed, FL, USA). The endoscope was determined to have a transmission efficiency of 53% and 42% at 660 nm and 760 nm, respectively, making it more efficient for fluorescence imaging than standard, nonoptimized endoscopes.³⁹ Finally, the endoscope is attached to the camera via a C-mount adapter with a focal length of 28 mm (#20-0215, Myriad Fiber Optics, Dudley, Massachusetts).

2.1.3.

Dual-channel camera design

The simultaneous dual-channel acquisition in this system is accomplished using a custom-made prototype prism-based 2-CCD camera (AD-130GE, Jai Inc., Yokohama, Kanagawa). The camera includes both color and monochrome 1/3 in. Sony ICX447 CCD image sensors with custom filters. The color sensor operates in 24-bit RGB mode and the NIR sensor acquires 12-bit images, both with a resolution of $1296\times 966\text{pixels}$. The use of the same image sensors in this integrated camera package has two important advantages in the design of a clinical endoscopic imaging system. First, the camera design ensures that the imaging system is sufficiently small and lightweight that it can be operated easily by one hand during minimally invasive procedures. Second, the use of identical sensors for both channels allows the acquisition of color and NIR channels of exactly the same dimensions with built-in robustness and accurate registration. The images acquired by the two sensors are transmitted to the computer via two independent Gigabit Ethernet (GigE) channels. The acquired data for each channel is then cropped to $960\times 960\text{pixels}$ for display on the screen and the images are currently resampled to $512\times 512\text{pixels}$ when data is saved to the disk. The camera also allows the independent control of exposure (from 1 ms to 2 s) and gain (up to 23 dB) on each sensor. However, in order to maintain a fluid refresh rate, the exposure time values should preferably be lower than 100 ms (10 fps) with an acceptable upper limit of 150 ms (6.8 fps). Slower refresh rates will strongly limit the potential for ergonomic use due to the movement artifacts during minimally invasive procedures.

In order to use the camera for NIR fluorescence imaging, the prism optics in the camera are customized with the specific filter coatings to allow the acquisition of color images (400 to 650 nm) and monochrome images for two selected NIR windows above 685 nm (Chroma, VT, USA). Figure 3(a) shows the design of the prism-based dual-channel acquisition. Specifically, the camera includes two prism blocks wherein a dichroic filter coating on the long edge of one block reflects the NIR wavelengths greater than 650 nm while transmitting all the shorter wavelengths. The emission filters are positioned in front of each of the image sensors to isolate the signals of interest. In order to improve the versatility of this device, the fluorescence emission filter (in front of the NIR sensor) is designed to have dual band-pass characteristics allowing the imaging of fluorophores emitting in two distinct wavelength windows. The first window (referred to as 700-nm channel), centered at 710 nm with a 50-nm bandwidth, acquires a fluorescence signal from agents excited at 660 nm (e.g., MB). The second window (referred to as 800-nm channel), acquires fluorescence signal at all the wavelengths above 780 nm, typically from agents excited at 760 nm (e.g., ICG). It should be noted that the single monochrome sensor design allows the sequential (nonconcurrent) acquisition of NIR fluorescence images in either channel (700 or 800 nm) concurrently with color images. To achieve dual NIR channel imaging (i.e., 700 and 800 nm fluorescence concurrently with color), a software acquisition mode synchronizes NIR excitation light with camera acquisition to guarantee that the two NIR fluorescence channels are separated. Actual transmission spectra of the filter coatings described are shown in Fig. 3(b).

Fig. 3

Camera design. (a) Prism-based 2-CCD camera design. Light enters the prism and is separated into two channels using a dichroic coating between the two prism blocks. Emission filters are placed between the prism output and the image sensors. (b) Transmission characteristics of custom filter coatings for dual-channel NIR and color image acquisition. This includes a dichroic coating separating the two channels and two emission filters, one for the color and the other for the NIR fluorescence.

2.2.1.

Optical characteristics

The spatial resolution of the system was determined using a USAF 1951 target (#38-710, Edmund Optics, Barrington, New Jersey), and measured using the slanted edge method. The test target was imaged at a distance of 5 cm and the resolution was determined using the color image. Finally, the accuracy of registration of the two imaging channels was determined by imaging four tubes of ICG in dimethyl sulfoxide (DMSO) (at 1, 0.5, 0.25, and 0.125 μM concentration). The fluence was measured using a calibrated Orion power meter (PD300-3W, Ophir Optronics, North Logan, UT, USA).

2.2.2.

Noise characterization

The image noise (background noise) in a dual-channel fluorescence imaging system can arise from the dark noise of the CCD sensor and the excitation light leakage due to inefficient filtration. In order to characterize the background noise of this system, the optical signal from nonfluorescent backgrounds has been measured at 100 ms exposure time as a function of distance (from 2 to 9 cm) and gain (up to 23 dB/100% gain). Three different backgrounds were used:

It should be noted that each NIR channel was characterized independently with the respective laser excitation and white light illumination at their maximum output. The average photon counts for the entire image recorded for each setting was used to determine the background noise.

2.2.3.

Sensitivity

The sensitivity of the system for the 800 nm fluorescence channel was determined by imaging the signal from the five samples of ICG diluted in DMSO at varying concentrations ranging from 50 to 1000 nM. The measurements were acquired at eight working distances expected in vivo (ranging from 2 to 9 cm at 1 cm steps) at 100% gain settings and exposure times of 50, 100, and 150 ms. The above procedure was repeated for the same working distances and camera settings to determine the sensitivity of the 700 nm fluorescence channel using dilutions of MB in DMSO at concentrations ranging from 100 to 1000 nM.

The average signal recorded over a $50\times 50\text{pixel}$ region of interest was used to estimate the corresponding photon counts to establish the linearity of the detector and determine the sensitivity of the system at different working distances. Furthermore, the range of working distances over which the signal in both 700- and 800-nm channels is detectable over the noise floor for three concentrations at 100 ms exposure time is analyzed.

3.1.1.

Endoscopic imaging system

Figure 4 shows a picture of the dual-channel clinical endoscopic imaging system. The system is comprised of a white light and NIR laser source, a dual-channel camera, a rigid endoscope, and a computer for data acquisition and system control housed in a small form-factor cart for easy transportation. Note the two displays, one for the system operator and the other for the surgeon. The custom endoscope made by Schoelly has been fabricated following the ISO 9001 and 13485 standards and can be sterilized using standard clinical methods (Autoclave, EtO, STERIS STERRAD, and STERIS System). In addition, the system uses a standard endoscope sterile drape that connects at the junction between the endoscope and the camera coupler to guarantee complete sterility near the surgical field. The complete system specifications are shown in Table 1.

Fig. 4

The NIR-optimized dual-channel clinical endoscopic imaging system. The system comprises of a white light and NIR laser source, a dual-channel camera, a rigid endoscope, and a computer for data acquisition and system control housed in a small form-factor cart for easy transportation. Note the two displays, one for the system operator and the other for the surgeon.

Table 1

System specifications at 5 cm nominal working distance.

3.1.2.

Optical characteristics

Figure 5(a) shows the image of the USAF 1951 test target acquired on the color channel. Using the slanted edge method, the spatial resolution was measured to be 110 μm at a distance of 5 cm. Figures 5(b) and 5(c) show the color and NIR images of the ICG samples and Fig. 5(d) shows the overlay of the two channels with the fluorescence information in green on the top of the color image (merge image). It should be noted that the two channels are accurately coregistered with no observable misalignment of the two sensors.

Fig. 5

Bench testing for the imaging system. (a) Color image of USAF 1951 resolution test target acquired at 5 cm working distance. (b–d) Color, NIR, and merge images of indocyanine green (ICG) in dimethyl sulfoxide (DMSO), respectively, at 1.0 μM, 0.5 μM, 0.25 μM, and 0.125 μM (left to right). Note the accurate registration of images from the color and NIR channels.

3.1.3.

Illumination and laser safety

The total laser power output from the endoscope was limited to 26 and 156 mW at 660 and 760 nm, respectively. Because the laser Class 3R accessible emission limits for this particular endoscope at these wavelengths are 400 and 160 mW (IEC 60825-1: 2007), this device operates as a Class 3R device. This translates to fluence of 14.0, 2.3, $0.7\mathrm{mW}/{\mathrm{cm}}^2$ at 660 nm and 41.2, 12.9, $2.8\mathrm{mW}/{\mathrm{cm}}^2$ at 760 nm at the working distances of 2, 5, and 9 cm, respectively. Furthermore, the white light illuminance at the 5 cm working distance was measured to be 33 klux.

3.1.4.

Background noise

Figures 6(a) and 6(b) show the variation in background noise arising from light leakage and dark noise in the 700 nm emission channel as a function of working distance and gain, respectively, for the three different reflectance materials (black ABS plastic, tissue phantom, Spectralon). Figures 6(c) and 6(d) show the same measurements for the 800 nm emission channel. As expected, background noise increases when increasing the gain or reducing the working distance. It is interesting to note that in the worst possible measurement condition (100% gain at 2 cm), the maximum background noise is limited to 350, 216, and 147 counts for the 700 nm emission channel and 418, 256, and 152 counts for the 800 nm emission channel, for Spectralon, tissue mimicking phantom and black ABS plastic backgrounds, respectively. For the 12-bit NIR sensor employed in this camera (4095 gray levels), this translates to 8.5%, 5.3%, and 3.6% of the dynamic range at 700 nm and 10.2%, 6.3%, and 3.7% of the dynamic range at 800 nm, indicating a highly efficient filtration scheme.

Fig. 6

Background noise evaluation using three different reflectance materials (black ABS plastic, tissue phantom, and Spectralon). (a and b) Background noise measured in the 700-nm channel under 660-nm excitation and white light at their maximum output, as a function of working distance and gain. (c and d) Background noise measured in the 800-nm channel under 760-nm excitation and white light at their maximum output, as a function of working distance and gain.

3.1.5.

Sensitivity

The average fluorescence signal measured in the 700-nm channel at 3 cm working distance using MB for the three exposure times investigated herein is plotted in Fig. 7(a). The linear fit to the data shows excellent detector linearity over the range of concentration. Furthermore, the variation in measured signal intensity (at 100 ms exposure time) with the working distance is plotted in Fig. 7(b). Similarly, Figs. 7(c) and 7(d) show the corresponding measurements of system sensitivity and signal variation over distance for the 800-nm channel using ICG.

Fig. 7

Sensitivity performance evaluation. (a) Average fluorescence signal from various concentrations of methylene blue (MB) in DMSO at 3 cm working distance. (b) Photon counts measured from MB in DMSO at three concentrations at varying working distances. (c) Average fluorescence signal from various concentrations of ICG in DMSO at 3 cm working distance. (d) Photon counts measured from ICG in DMSO at three concentrations at varying working distances. BG, background.

As expected, both channels demonstrate higher sensitivity at longer exposure times. The measured photon counts for both the 700- and 800-nm channels are above their respective noise floor ($\sim 150$ counts) for all the three concentrations considered. It is worth noting that these measurements are acquired at a 100 ms exposure time which translates to a 10 fps acquisition. From these measurements, we define detectability based on the SNR of at least 2 (signal level more than twice the noise floor). Extrapolating from the three concentrations measured, we establish that $\sim 100$ to 185 nM can be detected at 3 cm working distance for both the 700 and 800-nm channels, depending on the background optical properties (here from black plastic to tissue simulating phantom).