Translator Disclaimer
20 December 2013 Design and characterization of an optimized simultaneous color and near-infrared fluorescence rigid endoscopic imaging system
Author Affiliations +
Abstract
We report the design, characterization, and validation of an optimized simultaneous color and near-infrared (NIR) fluorescence rigid endoscopic imaging system for minimally invasive surgery. This system is optimized for illumination and collection of NIR wavelengths allowing the simultaneous acquisition of both color and NIR fluorescence at frame rates higher than 6.8 fps with high sensitivity. The system employs a custom 10-mm diameter rigid endoscope optimized for NIR transmission. A dual-channel light source compatible with the constraints of an endoscope was built and includes a plasma source for white light illumination and NIR laser diodes for fluorescence excitation. A prism-based 2-CCD camera was customized for simultaneous color and NIR detection with a highly efficient filtration scheme for fluorescence imaging of both 700- and 800-nm emission dyes. The performance characterization studies indicate that the endoscope can efficiently detect fluorescence signal from both indocyanine green and methylene blue in dimethyl sulfoxide at the concentrations of 100 to 185 nM depending on the background optical properties. Finally, we performed the validation of this imaging system in vivo during a minimally invasive procedure for thoracic sentinel lymph node mapping in a porcine model.

1.

Introduction

The translation of optical imaging methods to clinical applications has rapidly grown in recent years, as evidenced in the field of image-guided surgery (IGS).1 IGS refers to the use of an imaging modality to provide the surgeon with the additional information (e.g., functional or structural) allowing more accurate, objective, and efficient surgical procedures.23.4.5 One promising application of optical imaging in IGS is based on the fluorescence signal detection for the identification of anatomical features or suspicious tissue during the procedure.67.8 Specifically, the fluorescence signal may arise from endogenous fluorophores [e.g., NADH,9 protoporphyrin IX,10,11] or may be externally administered via intravenous, intratumoral, or peritumoral injection [e.g., indocyanine green (ICG), methylene blue (MB)12,13]. In terms of device developments, there have been significant advances in designing optical imaging systems used for IGS resulting in compact, ergonomic, and sensitive imaging systems.1415.16 These systems have, however, been designed for invasive, open surgical procedures with lens, and illumination designs allowing imaging over wide fields of view (typically in the hundreds of centimeters squared). Many fluorescence imaging systems have successfully reached the clinic and have been used during first-in-human trials relying on off-label use of U.S. FDA approved contrast agents.1,17,18 In particular, MB and ICG have been extensively used to investigate applications of fluorescence IGS since they can delineate some tumors through passive accumulation or highlight flow in lymphatics and vasculature. Typical applications include sentinel lymph node mapping and lymphatic imaging,1920.21 angiography,22,23 intraoperative tumor detection,8,2425.26 and ureters and bile duct imaging.27,28

In recent years, surgical procedures witnessed a significant shift toward the use of minimally invasive methods owing to minimal blood loss, faster recuperation, and less postoperative pain. Today, a variety of procedures such as gastrointestinal surgery,29 thoracic surgery,30,31 colonoscopy,32 and arthroscopy33 are primarily performed minimally invasively. The translation of optical IGS techniques to minimally invasive procedures can therefore play a major role in improving the quality of patient care. Endoscopic procedures incorporating fluorescence detection for guiding surgery have typically relied on the use of fluorophores excited in ultraviolet and far blue wavelengths (350 to 500 nm).3435.36 On the other hand, the use of fluorophores operating in the near-infrared (NIR) wavelengths (650 to 1000 nm) offering deeper contrast and reduced auto fluorescence in living tissues has not been widely explored during minimally invasive procedures. A number of studies demonstrated the use of NIR fluorescence-guided endoscopic imaging systems incorporating NIR detection capabilities with standard endoscopic imaging systems.37,38 However, the poor transmission of NIR light through standard clinical endoscopes and the limited sensitivity of standard camera systems at longer NIR wavelengths can result in fluorescence images with lower signal-to-noise ratio (SNR) and long exposure times.39 In turn, the increase in exposure times during NIR fluorescence detection resulted in low acquisition frame rates making this approach impractical for real-time ergonomic clinical use. Furthermore, the potential of NIR fluorescence-guided endoscopic imaging is highlighted by the development of commercial clinically compatible systems optimized for NIR fluorescence imaging (e.g., Storz FL series40 and Novadaq spyscope41). These systems are, however, either designed for single-channel imaging (i.e., either color or NIR), or present a significant amount of fluorescence background. From our experience in IGS, the benefit of having co-registration between the color image and a bright, background-free NIR fluorescence signal is of paramount importance since it provides both anatomical landmarks and clear fluorescence localization simultaneously to the surgeon. In addition, there is a growing clinical interest in the field for using either a 700-nm dye (e.g., MB), or an 800-nm dye (e.g., ICG) or both simultaneously42,43 that requires one to incorporate dual NIR channel capability on a single imaging system. Together, all these limitations demand the development of a custom endoscopic imaging system optimized for operation in the NIR wavelengths.

The development of an NIR optimized endoscopic imaging system is primarily based on increasing throughput and efficiency. For this purpose, two different components can be optimized: the endoscope and the detector.

  • 1. The endoscope is responsible for the transmission of the illumination light as well as relaying images back to the detector. The use of a nonoptimized endoscope for NIR imaging can result in significant losses in both illumination and collection.44 A recent study comparing NIR performances of standard and NIR optimized endoscopes identified up to a 6-fold improvement in global throughput (1.5-fold in illumination and 4-fold in collection) when using an optimized clinical endoscope such as the one used in this study.39

  • 2. The detector used in a dual-channel (color and NIR) endoscopic imaging system is comprised of filters for separating the color and NIR optical signals and corresponding image sensors for image acquisition. Due to the inherently low NIR fluorescence signal, the sensitivity of the image sensor and the efficiency of the filtration scheme play a critical role in the performance of a fluorescence imaging system. The image sensor should be highly sensitive, with a high quantum yield for fast NIR imaging, while allowing minimal camera head footprint for ergonomic use in minimally invasive surgery. Previous approaches have relied on the use of higher sensitivity cameras (such as EMCCD) or an image intensifier to improve the detection sensitivity42,43 but pose significant challenges in terms of integration within the clinical workflow and/or noise and resolution degradation.45

In this study, we describe the design, characterization, and validation of a clinically compatible endoscopic imaging system optimized for simultaneous color and NIR fluorescence imaging. First, the design and specification of the primary system components (source, endoscope, and camera) are detailed. The imaging system is then characterized under typical clinical settings. Finally, the performance of this system for in vivo imaging is validated during a minimally invasive procedure for sentinel lymph node mapping of the lung in a porcine animal model.

2.

Materials and Methods

2.1.

System Design

Figure 1 depicts a schematic of the dual-channel clinical endoscopic imaging system. The system comprises of a white light and NIR laser source, a dual-channel camera, fibers for light transmission, and a rigid endoscope. In the following sections, the primary system components are described in detail.

Fig. 1

Schematic of a typical dual-channel near-infrared (NIR) and color endoscopic imaging system. Light from a filtered white light source is combined with the light from an NIR excitation source and injected into the endoscope through the illumination port. Light is collected through the endoscope imaging channel, split using a dichroic mirror (Di) and filtered in two channels, one for the color imaging and the other for the NIR fluorescence imaging.

JBO_18_12_126018_f001.png

2.1.1.

Source design

The light source used in this system combines a laser diode based NIR source with a plasma-based white light source for simultaneous illumination for both color and NIR imaging. The laser module houses four 2.5 W laser diodes operating at 760 nm (LDX-3215-760, LDX Optronics, Maryville, Tennessee) and operated using current and thermo-electric cooler controllers (ITC300, Thorlabs, Newton, New Jersey). Alternatively, the system can also use four 1 W laser diodes operating at 660 nm (LDX-3115-660, LDX Optronics, TN, USA) imaging NIR fluorophores with the emissions at shorter NIR wavelengths (e.g., MB). The white light illumination is generated by a high-power solid state plasma light source (HPLS-30-02, Thorlabs, NJ, USA) with an output power of 10.2 W (at 5 mm focal spot, 0.5 NA) in the visible wavelengths.

The two sources are combined using a custom-designed light coupler (mechanically mounted on the plasma light source) that shapes the output from the two sources before injection into a fiber-optic light guide with 4.8-mm core diameter (0.54 NA). The design of the light coupler is shown in Fig. 2. It should be noted that a dichroic mirror (T673lpxxr, Chroma Technologies, Bellows Falls, Vermont) was used in order to combine the two light sources. Furthermore, the light coupler includes a hot mirror (21002A, Chroma Technologies, VT, USA) and a 650 nm short pass filter (E650SP, Chroma, VT, USA) in order to filter out the plasma source output outside the visible window, thereby reducing the light leakage in the NIR images. The NIR laser module is connected to the light coupler via an SMA terminated 2 mm multimode fiber optic cable. The output port of the light coupler accepts a standard off-the-shelf 1.8-m long fiber optic light guide with a 4.8-mm core diameter (C3278, Conmed, Utica, New York). The use of a fiber optimized for NIR transmission could further improve the efficiency of the system. The light coupler operates at a coupling efficiency of 87% for the NIR light source and 35% for the white light source.

Fig. 2

Source coupler design. (a) Mechanical layout of the source coupler mounted on the custom white light source chassis. F1: IR filter. F2: 650SP filter. L1: 30 mm biconvex lens. L2: 25 mm biconvex lens. Di: T673LP dichroic mirror. L3: 25 mm biconvex lens. (b) Optical design of source coupler.

JBO_18_12_126018_f002.png

2.1.2.

Endoscope design

The system described here uses a custom rigid rod-lens based NIR optimized endoscope (Schoelly Fiberoptic GmBH, Denzlingen, Germany). The endoscope is 340-mm long and has a 10-mm bore diameter with a 5 mm imaging area and a 30 deg direction of view. The fiber optic light guide from the source coupler is connected to the endoscope via an Olympus endoscope fitting (7453, Conmed, FL, USA). The endoscope was determined to have a transmission efficiency of 53% and 42% at 660 nm and 760 nm, respectively, making it more efficient for fluorescence imaging than standard, nonoptimized endoscopes.39 Finally, the endoscope is attached to the camera via a C-mount adapter with a focal length of 28 mm (#20-0215, Myriad Fiber Optics, Dudley, Massachusetts).

2.1.3.

Dual-channel camera design

The simultaneous dual-channel acquisition in this system is accomplished using a custom-made prototype prism-based 2-CCD camera (AD-130GE, Jai Inc., Yokohama, Kanagawa). The camera includes both color and monochrome 1/3 in. Sony ICX447 CCD image sensors with custom filters. The color sensor operates in 24-bit RGB mode and the NIR sensor acquires 12-bit images, both with a resolution of 1296×966pixels. The use of the same image sensors in this integrated camera package has two important advantages in the design of a clinical endoscopic imaging system. First, the camera design ensures that the imaging system is sufficiently small and lightweight that it can be operated easily by one hand during minimally invasive procedures. Second, the use of identical sensors for both channels allows the acquisition of color and NIR channels of exactly the same dimensions with built-in robustness and accurate registration. The images acquired by the two sensors are transmitted to the computer via two independent Gigabit Ethernet (GigE) channels. The acquired data for each channel is then cropped to 960×960pixels for display on the screen and the images are currently resampled to 512×512pixels when data is saved to the disk. The camera also allows the independent control of exposure (from 1 ms to 2 s) and gain (up to 23 dB) on each sensor. However, in order to maintain a fluid refresh rate, the exposure time values should preferably be lower than 100 ms (10 fps) with an acceptable upper limit of 150 ms (6.8 fps). Slower refresh rates will strongly limit the potential for ergonomic use due to the movement artifacts during minimally invasive procedures.

In order to use the camera for NIR fluorescence imaging, the prism optics in the camera are customized with the specific filter coatings to allow the acquisition of color images (400 to 650 nm) and monochrome images for two selected NIR windows above 685 nm (Chroma, VT, USA). Figure 3(a) shows the design of the prism-based dual-channel acquisition. Specifically, the camera includes two prism blocks wherein a dichroic filter coating on the long edge of one block reflects the NIR wavelengths greater than 650 nm while transmitting all the shorter wavelengths. The emission filters are positioned in front of each of the image sensors to isolate the signals of interest. In order to improve the versatility of this device, the fluorescence emission filter (in front of the NIR sensor) is designed to have dual band-pass characteristics allowing the imaging of fluorophores emitting in two distinct wavelength windows. The first window (referred to as 700-nm channel), centered at 710 nm with a 50-nm bandwidth, acquires a fluorescence signal from agents excited at 660 nm (e.g., MB). The second window (referred to as 800-nm channel), acquires fluorescence signal at all the wavelengths above 780 nm, typically from agents excited at 760 nm (e.g., ICG). It should be noted that the single monochrome sensor design allows the sequential (nonconcurrent) acquisition of NIR fluorescence images in either channel (700 or 800 nm) concurrently with color images. To achieve dual NIR channel imaging (i.e., 700 and 800 nm fluorescence concurrently with color), a software acquisition mode synchronizes NIR excitation light with camera acquisition to guarantee that the two NIR fluorescence channels are separated. Actual transmission spectra of the filter coatings described are shown in Fig. 3(b).

Fig. 3

Camera design. (a) Prism-based 2-CCD camera design. Light enters the prism and is separated into two channels using a dichroic coating between the two prism blocks. Emission filters are placed between the prism output and the image sensors. (b) Transmission characteristics of custom filter coatings for dual-channel NIR and color image acquisition. This includes a dichroic coating separating the two channels and two emission filters, one for the color and the other for the NIR fluorescence.

JBO_18_12_126018_f003.png

2.2.

System Characterization

2.2.1.

Optical characteristics

The spatial resolution of the system was determined using a USAF 1951 target (#38-710, Edmund Optics, Barrington, New Jersey), and measured using the slanted edge method. The test target was imaged at a distance of 5 cm and the resolution was determined using the color image. Finally, the accuracy of registration of the two imaging channels was determined by imaging four tubes of ICG in dimethyl sulfoxide (DMSO) (at 1, 0.5, 0.25, and 0.125 μM concentration). The fluence was measured using a calibrated Orion power meter (PD300-3W, Ophir Optronics, North Logan, UT, USA).

2.2.2.

Noise characterization

The image noise (background noise) in a dual-channel fluorescence imaging system can arise from the dark noise of the CCD sensor and the excitation light leakage due to inefficient filtration. In order to characterize the background noise of this system, the optical signal from nonfluorescent backgrounds has been measured at 100 ms exposure time as a function of distance (from 2 to 9 cm) and gain (up to 23 dB/100% gain). Three different backgrounds were used:

  • A 12in.×12in. piece of ABS plastic, nonfluorescent, having a rough surface exhibiting a Lambertian reflectance.

  • A 8.5in.×8.5in. tissue mimicking phantom made of silicon and having an absorption value of 0.05mm1 and a reduced scattering value of 0.97mm1 at 650 nm, as measured using our spatial frequency domain imaging system.46,47

  • A 10in.×10in. Spectralon diffuse reflectance target having a 99% reflectance value (SRT-99-100, Labsphere Inc., North Sutton, NH, USA).

It should be noted that each NIR channel was characterized independently with the respective laser excitation and white light illumination at their maximum output. The average photon counts for the entire image recorded for each setting was used to determine the background noise.

2.2.3.

Sensitivity

The sensitivity of the system for the 800 nm fluorescence channel was determined by imaging the signal from the five samples of ICG diluted in DMSO at varying concentrations ranging from 50 to 1000 nM. The measurements were acquired at eight working distances expected in vivo (ranging from 2 to 9 cm at 1 cm steps) at 100% gain settings and exposure times of 50, 100, and 150 ms. The above procedure was repeated for the same working distances and camera settings to determine the sensitivity of the 700 nm fluorescence channel using dilutions of MB in DMSO at concentrations ranging from 100 to 1000 nM.

The average signal recorded over a 50×50pixel region of interest was used to estimate the corresponding photon counts to establish the linearity of the detector and determine the sensitivity of the system at different working distances. Furthermore, the range of working distances over which the signal in both 700- and 800-nm channels is detectable over the noise floor for three concentrations at 100 ms exposure time is analyzed.

2.3.

Animal Model

The performance of the NIR optimized endoscopic imaging system in vivo was validated during lung sentinel lymph node detection and resection in a porcine model. The animal studies were performed under the supervision of Beth Israel Deaconess Medical Center’s Institutional Animal Care and Use Committee in accordance with the approved institutional protocols. Female Yorkshire pigs (E.M. Parsons and Sons, Hadley, MA, USA) averaging 37 kg were induced with intramuscular Telazol™ (Fort Dodge Labs, Fort Dodge, IA, USA), incubated and maintained with 2% isoflurane (Baxter Healthcare Corp., Deerfield, IL, USA). Physiological parameters (heart rate, oxygen saturation, and body temperature) were monitored during the procedure. In this study, 0.5 cc of 3.2 mM ICG in saline buffer was superficially injected (2 to 3-mm deep) in the lung parenchyma on the left posterior upper lobe. The localization of the dye in the mediastinal and hilar lymph nodes was investigated 5 min postinjection. Once identified, the node expressing ICG fluorescence signal was resected. The color camera was operated at an exposure time of 21 ms and the NIR camera exposure time was adjusted between 20 and 150 ms (based on the signal level) at maximum gain setting. The entire procedure was recorded on the video for analysis.

3.

Results

3.1.

System Characteristics

3.1.1.

Endoscopic imaging system

Figure 4 shows a picture of the dual-channel clinical endoscopic imaging system. The system is comprised of a white light and NIR laser source, a dual-channel camera, a rigid endoscope, and a computer for data acquisition and system control housed in a small form-factor cart for easy transportation. Note the two displays, one for the system operator and the other for the surgeon. The custom endoscope made by Schoelly has been fabricated following the ISO 9001 and 13485 standards and can be sterilized using standard clinical methods (Autoclave, EtO, STERIS STERRAD, and STERIS System). In addition, the system uses a standard endoscope sterile drape that connects at the junction between the endoscope and the camera coupler to guarantee complete sterility near the surgical field. The complete system specifications are shown in Table 1.

Fig. 4

The NIR-optimized dual-channel clinical endoscopic imaging system. The system comprises of a white light and NIR laser source, a dual-channel camera, a rigid endoscope, and a computer for data acquisition and system control housed in a small form-factor cart for easy transportation. Note the two displays, one for the system operator and the other for the surgeon.

JBO_18_12_126018_f004.png

Table 1

System specifications at 5 cm nominal working distance.

CategorySpecificationDescription
PhysicalEndoscope length340 mm
Direction of view30 deg
Diameter10 mm
Camera size5.5×5.5×9.0  cm3
Weight726 g
NIR lightTypeSMA coupled laser diode
Wavelength660 nm and 760 nm
Power26 mW (660 nm) 156 mW (760 nm)
White lightTypePlasma white light source
Power (Visible)10.25 W
Numerical aperture0.5
Light guideType4.8 mm active area fiber optic bundle 1.8 m In length.
Numerical aperture0.54
IlluminationFluence2.3  mW/cm2 (660 nm) 12.9  mW/cm2 (760 nm)
Illuminance33 kLux
SafetyClass 3R
CollectionWorking distance5 cm
Optical resolution110 μm
Channels2
Camera resolution960×960 px (display) 512×512 px (recording)
NIR exposure time1 ms–150 ms

3.1.2.

Optical characteristics

Figure 5(a) shows the image of the USAF 1951 test target acquired on the color channel. Using the slanted edge method, the spatial resolution was measured to be 110 μm at a distance of 5 cm. Figures 5(b) and 5(c) show the color and NIR images of the ICG samples and Fig. 5(d) shows the overlay of the two channels with the fluorescence information in green on the top of the color image (merge image). It should be noted that the two channels are accurately coregistered with no observable misalignment of the two sensors.

Fig. 5

Bench testing for the imaging system. (a) Color image of USAF 1951 resolution test target acquired at 5 cm working distance. (b–d) Color, NIR, and merge images of indocyanine green (ICG) in dimethyl sulfoxide (DMSO), respectively, at 1.0 μM, 0.5 μM, 0.25 μM, and 0.125 μM (left to right). Note the accurate registration of images from the color and NIR channels.

JBO_18_12_126018_f005.png

3.1.3.

Illumination and laser safety

The total laser power output from the endoscope was limited to 26 and 156 mW at 660 and 760 nm, respectively. Because the laser Class 3R accessible emission limits for this particular endoscope at these wavelengths are 400 and 160 mW (IEC 60825-1: 2007), this device operates as a Class 3R device. This translates to fluence of 14.0, 2.3, 0.7mW/cm2 at 660 nm and 41.2, 12.9, 2.8mW/cm2 at 760 nm at the working distances of 2, 5, and 9 cm, respectively. Furthermore, the white light illuminance at the 5 cm working distance was measured to be 33 klux.

3.1.4.

Background noise

Figures 6(a) and 6(b) show the variation in background noise arising from light leakage and dark noise in the 700 nm emission channel as a function of working distance and gain, respectively, for the three different reflectance materials (black ABS plastic, tissue phantom, Spectralon). Figures 6(c) and 6(d) show the same measurements for the 800 nm emission channel. As expected, background noise increases when increasing the gain or reducing the working distance. It is interesting to note that in the worst possible measurement condition (100% gain at 2 cm), the maximum background noise is limited to 350, 216, and 147 counts for the 700 nm emission channel and 418, 256, and 152 counts for the 800 nm emission channel, for Spectralon, tissue mimicking phantom and black ABS plastic backgrounds, respectively. For the 12-bit NIR sensor employed in this camera (4095 gray levels), this translates to 8.5%, 5.3%, and 3.6% of the dynamic range at 700 nm and 10.2%, 6.3%, and 3.7% of the dynamic range at 800 nm, indicating a highly efficient filtration scheme.

Fig. 6

Background noise evaluation using three different reflectance materials (black ABS plastic, tissue phantom, and Spectralon). (a and b) Background noise measured in the 700-nm channel under 660-nm excitation and white light at their maximum output, as a function of working distance and gain. (c and d) Background noise measured in the 800-nm channel under 760-nm excitation and white light at their maximum output, as a function of working distance and gain.

JBO_18_12_126018_f006.png

3.1.5.

Sensitivity

The average fluorescence signal measured in the 700-nm channel at 3 cm working distance using MB for the three exposure times investigated herein is plotted in Fig. 7(a). The linear fit to the data shows excellent detector linearity over the range of concentration. Furthermore, the variation in measured signal intensity (at 100 ms exposure time) with the working distance is plotted in Fig. 7(b). Similarly, Figs. 7(c) and 7(d) show the corresponding measurements of system sensitivity and signal variation over distance for the 800-nm channel using ICG.

Fig. 7

Sensitivity performance evaluation. (a) Average fluorescence signal from various concentrations of methylene blue (MB) in DMSO at 3 cm working distance. (b) Photon counts measured from MB in DMSO at three concentrations at varying working distances. (c) Average fluorescence signal from various concentrations of ICG in DMSO at 3 cm working distance. (d) Photon counts measured from ICG in DMSO at three concentrations at varying working distances. BG, background.

JBO_18_12_126018_f007.png

As expected, both channels demonstrate higher sensitivity at longer exposure times. The measured photon counts for both the 700- and 800-nm channels are above their respective noise floor (150 counts) for all the three concentrations considered. It is worth noting that these measurements are acquired at a 100 ms exposure time which translates to a 10 fps acquisition. From these measurements, we define detectability based on the SNR of at least 2 (signal level more than twice the noise floor). Extrapolating from the three concentrations measured, we establish that 100 to 185 nM can be detected at 3 cm working distance for both the 700 and 800-nm channels, depending on the background optical properties (here from black plastic to tissue simulating phantom).

3.2.

In Vivo Imaging Results

Figures 8(a) and 8(b) show the color, NIR, and merge images acquired minimally invasively in vivo during the detection and resection of a lung sentinel lymph node using ICG. Figure 8(a) shows the image acquired 5 min postinjection and Fig. 8(b) shows the snapshot of the exposed node. The images shown were acquired at an exposure time of 100 ms at 100% gain settings with the real-time display acquiring at a frame rate of 10fps. The fluorescence signal in the neighborhood of the lymph node (on the chest wall) was defined as the background. The signal to background ratio (SBR) in these images was determined to be 5.2, demonstrating the highly sensitive detection of fluorescence in vivo using the endoscopic imaging system. Finally, Fig. 8(c) shows an image of the resected lymph node acquired using the endoscope ex vivo. Video 1 shows an example of the data collected in real time using the dual-channel endoscopic system highlighting the functionality of the imaging system operating at 6.8 fps and capable of efficiently acquiring both color and NIR fluorescence information.

Fig. 8

In vivo evaluation of the endoscopic imaging system: snapshots. (a) Lymph node detected 5 min after injection of ICG. (b) Exposed lymph node prior to resection. (c) Excised lymph node imaged ex vivo.

JBO_18_12_126018_f008.png

Video 1

In vivo evaluation of the endoscopic imaging system: video. In this study, 0.5 cc of 3.2 mM ICG in saline buffer was superficially injected (2 to 3 mm deep) in the lung parenchyma on the left posterior upper lobe. The localization of the dye in the mediastinal and hilar lymph nodes was investigated 5 min postinjection. Exposure time on the NIR camera was 150 ms. (MPEG, 8.06 MB) [URL: http://dx.doi.org/10.1117/1.JBO.18.12.126018.1].

JBO_18_12_126018_f009.png

4.

Discussion

With the rapid growth of minimally invasive surgical procedures, the development of endoscopic imaging systems for fluorescence-guided surgery presents an important potential application for optical imaging methods. Concurrently, the advances in molecular imaging contrast agents operating in the NIR window encourages the development of intraoperative imaging systems optimized for NIR fluorescence imaging.8,48 In light of these developments, the potential advantages of a clinical endoscopic imaging system optimized for NIR fluorescence imaging cannot be overstated. The fundamental design principle guiding the development of a dual-channel NIR and color endoscopic imaging system is the capability of imaging fluorescence with high sensitivity and high resolution in real time. The design of such systems for clinical applications introduces additional requirements of clinical regulation compliance and integration with the existing clinical workflow. Such systems must have a small form factor (comparable to existing endoscopic imaging systems) and must be capable of operating at high frame rates without any motion blurring.

In this article, we introduce a clinically compatible endoscopic imaging system where every component (including the source, the camera, and the endoscope) has been optimized to meet the requirements for realistic clinical use. For instance, the use of a highly sensitive 2-CCD camera system customized for NIR measurements ensured simultaneous acquisition of color and NIR measurements while retaining the small form factor of the device. The optimization of the endoscope for operation in the NIR wavelengths coupled with the camera allowed the acquisition of high-resolution fluorescence signal with high SNR. Overall, the characterization studies presented here demonstrate that the system can detect ICG and MB fluorescence in DMSO at concentrations as low as 100 nM at a distance of 3 cm and 100 ms exposure time (10 fps). Furthermore, this system was validated in vivo with the successful identification of lung lymph nodes in Yorkshire pigs with high sensitivity while acquiring images at frame rates higher than 6.8 fps.

The main challenge in designing a fluorescence endoscopic imaging system is to be able to perform fluorescence imaging with enough sensitivity to guarantee a high frame rate display in vivo, while complying with the constraints of endoscopy and clinical workflow. Because of the regulatory illumination power limitations, the camera and the NIR transmission of the optics are the components that can be optimized for increasing throughput. Although NIR transmission of the optics can be relatively easily optimized, increasing camera head sensitivity usually goes along with the footprint and weight. In this work, our approach was to prioritize the standard clinical workflow and to provide a small size and lightweight camera similar to currently used endoscopic cameras. Other approaches have used larger and heavier cameras, including EMCCDs or image intensifiers, either directly plugged onto the endoscope or using fiber-bundle relays.42,44,49 In either case, the entire imaging system should be designed to permit high quality fluorescent imaging at high frame rates in vivo. With a size of 5cm×5cm×9cm and a weight of 726 g, this system is easily held and manipulated with a single hand by the surgeon. In addition, the endoscope can be left in place in a laparoscopic entry port without other mechanical support if needed. Together our approach targets fast translation of the technology to explore clinical applications.

The downside of our approach is the intrinsic low sensitivity of CCD cameras having such a small form factor. However, from the surgeons working with our open surgery and endoscopic systems, we determined that the minimum frame rate for surgical guidance using nondynamic fluorescence processes (e.g., angiography, sentinel lymph node mapping, tumor delineation, ureters and bile ducts localization) to be higher than 6.8 fps. Our collaborators being typically early adopters of novel technologies, other clinicians may desire a faster frame rate minimum, which will happen following technological progress. In any case, this indicates that anything slower than 6.8 fps is not acceptable.

The evaluation of the performance metrics for NIR fluorescence imaging in particular, and diffuse imaging in general, is a very active area of research. In this work, we introduce the measurement of background noise on various nonfluorescent reflectance surfaces, including a Spectralon standard. Although background noise measurement is of paramount importance for the NIR fluorescence imaging, it is difficult to define an appropriate material for this purpose. Black ABS plastic, while commonly used in the literature, does not provide a realistic value due to its very low reflectance properties. Similarly, Spectralon is not realistic either due to its very high reflectance properties, but is the most accessible and reliable method for comparing performances. Tissue mimicking phantoms seem to be the most appropriate material due to their optical similarity to living tissues, but are not accessible to all researchers. Finally, the most realistic measurement is during in vivo experiments, but suffers from a great variability due to the variation of reflectance properties and optical properties.

The design of the system described here presents two singular advantages. First, the incorporation of a dual band-pass emission filter on the NIR sensor allows the acquisition of two NIR channels, one for 700 nm and the other for 800 nm emission dyes. The characterization of the two channels presented here indicates a high sensitivity when acquiring images in both wavelength windows. This offers a unique opportunity to investigate the advantages of dual-tracer imaging during minimally invasive procedures. Second, the endoscopic imaging system has been designed to meet the requirements for future clinical translation. Specifically, the components and operation settings were characterized to ensure compliance with guidelines set for clinical medical imaging devices in IEC 60601, laser safety in IEC 60825, as well as mechanical safety and documentation.

Finally, the imaging platform presented in this study is readily adaptable to NIR fluorescence imaging with flexible endoscopes.49,50 The camera and source can be mounted in the same manner to a flexible endoscope, used on the same platforms and controlled through the same software. It should be noted, however, that the manipulation of flexible endoscopes differ significantly from rigid endoscopes and the system we present may no longer be ergonomic for this purpose. In addition, flexible endoscopy for the fluorescence imaging is typically based on imaging fiber bundles, degrading resolution, and transmission. More work is required to achieve high frame rate imaging in fluorescence flexible endoscopy.

5.

Conclusion

In summary, we presented the design, characterization, and validation of an optimized dual-channel endoscopic imaging system capable of simultaneous real-time imaging of NIR fluorescence and color video during minimally invasive procedures. The feasibility and performance of this system were demonstrated in preclinical studies. Future investigations will focus on the translation of this system for clinical use during minimally invasive procedures. Overall, this study lays the foundations for the design of clinical compatible dual color and NIR fluorescence endoscopic imaging systems and their translation to the clinic.

Acknowledgments

The authors would like to thank Kelly Stockwell at Chroma Technologies or her help with custom filter fabrication, Christoph Herdrich and Niels Lemke at Schoelly Fiberoptics GMBH for the custom endoscope design, and Jon Chouinard, Steve Kinney, Wesley Okeke, and Gordon Rice at JAI Inc. for their support with customization of the 2-CCD camera. We would also like to thank David Burrington for editing and Eugenia Trabucchi for administrative assistance. This work was supported by the NIH/NCATS Award No. 8 UL1 TR000170-05 to Harvard Clinical and Translational Science Center, collaboration Award No. 027343.386541.05226, NIH/NCI Award No. R01-CA-115296, and NIH/NIDDK Award No. K01-DK-093603.

References

1. 

S. GiouxH. S. ChoiJ. V. Frangioni, “Image-guided surgery using invisible near-infrared light: fundamentals of clinical translation,” Mol. Imaging, 9 (5), 237 –255 (2010). MIOMBP 1535-3508 Google Scholar

2. 

S. Keereweeret al., “Optical image-guided surgery—where do we stand?,” Mol. Imaging Biol., 13 (2), 199 –207 (2011). https://doi.org/http://dx.doi.org/10.1007/s11307-010-0373-2 1536-1632 Google Scholar

3. 

F. A. Joleszet al., “MR imaging-controlled focused ultrasound ablation: a noninvasive image-guided surgery,” Magn. Reson. Imaging Clin. N. Am., 13 (3), 545 –560 (2005). https://doi.org/http://dx.doi.org/10.1016/j.mric.2005.04.008 1064-9689 Google Scholar

4. 

S. B. Sobottkaet al., “Comparison of functional brain PET images and intraoperative brain-mapping data using image-guided surgery,” Comput. Aided Surg., 7 (6), 317 –325 (2002). https://doi.org/http://dx.doi.org/10.3109/10929080209146520 1092-9088 Google Scholar

5. 

K. ClearyT. M. Peters, “Image-guided interventions: technology review and clinical applications,” Ann. Rev. Biomed. Eng., 12 119 –142 (2010). https://doi.org/http://dx.doi.org/10.1146/annurev-bioeng-070909-105249 ARBEF7 1523-9829 Google Scholar

6. 

J. V. Frangioni, “In vivo near-infrared fluorescence imaging,” Curr. Opin. Chem. Biol., 7 (5), 626 –634 (2003). https://doi.org/http://dx.doi.org/10.1016/j.cbpa.2003.08.007 COCBF4 1367-5931 Google Scholar

7. 

E. M. Sevick-Muraca, “Translation of near-infrared fluorescence imaging technologies: emerging clinical applications,” Annu. Rev. Med., 63 217 –231 (2012). https://doi.org/http://dx.doi.org/10.1146/annurev-med-070910-083323 0066-4219 Google Scholar

8. 

G. M. van Damet al., “Intraoperative tumor-specific fluorescence imaging in ovarian cancer by folate receptor-alpha targeting: first in-human results,” Nat. Med., 17 (10), 1315 –1319 (2011). https://doi.org/http://dx.doi.org/10.1038/nm.2472 1078-8956 Google Scholar

9. 

A. MayevskyG. G. Rogatsky, “Mitochondrial function in vivo evaluated by NADH fluorescence: from animal models to human studies,” Am. J. Physiol. Cell Physiol., 292 (2), C615 –C640 (2007). https://doi.org/http://dx.doi.org/10.1152/ajpcell.00249.2006 0363-6143 Google Scholar

10. 

M. A. D'HallewinL. BezdetnayaF. Guillemin, “Fluorescence detection of bladder cancer: a review,” Eur. Urol., 42 (5), 417 –425 (2002). https://doi.org/http://dx.doi.org/10.1016/S0302-2838(02)00402-5 EUURAV 0302-2838 Google Scholar

11. 

J. C. KennedyR. H. Pottier, “New trends in photobiology: endogenous protoporphyrin IX, a clinically useful photosensitizer for photodynamic therapy,” Photochem. Photobiol., 14 (4), 275 –292 (1992). https://doi.org/http://dx.doi.org/10.1016/1011-1344(92)85108-7 PHCBAP 0031-8655 Google Scholar

12. 

S. L. Gibbs, “Near infrared fluorescence for image-guided surgery,” Quant. Imaging Med. Surg., 2 (3), 177 –187 (2012). Google Scholar

13. 

S. Luoet al., “A review of NIR dyes in cancer targeting and imaging,” Biomaterials, 32 (29), 7127 –7138 (2011). https://doi.org/http://dx.doi.org/10.1016/j.biomaterials.2011.06.024 BIMADU 0142-9612 Google Scholar

14. 

S. Giouxet al., “FluoSTIC: miniaturized fluorescence image-guided surgery system,” J. Biomed. Opt., 17 (10), 106014 (2012). https://doi.org/http://dx.doi.org/10.1117/1.JBO.17.10.106014 JBOPFO 1083-3668 Google Scholar

15. 

Y. Liuet al., “Hands-free, wireless goggles for near-infrared fluorescence and real-time image-guided surgery,” Surgery, 149 (5), 689 –698 (2011). https://doi.org/http://dx.doi.org/10.1016/j.surg.2011.02.007 SURGAZ 0039-6060 Google Scholar

16. 

X. Wanget al., “Compact instrument for fluorescence image-guided surgery,” J. Biomed. Opt., 15 (2), 020509 (2010). https://doi.org/http://dx.doi.org/10.1117/1.3378128 JBOPFO 1083-3668 Google Scholar

17. 

M. V. Marshallet al., “Near-infrared fluorescence imaging in humans with indocyanine green: a review and update,” Open Surg. Oncol. J., 2 (2), 12 –25 (2010). https://doi.org/http://dx.doi.org/10.2174/1876504101002020012 1876-5041 Google Scholar

18. 

B. E. Schaafsmaet al., “The clinical use of indocyanine green as a near-infrared fluorescent contrast agent for image-guided oncologic surgery,” J. Surg. Oncol., 104 (3), 323 –332 (2011). https://doi.org/http://dx.doi.org/10.1002/jso.v104.3 JSONAU 0022-4790 Google Scholar

19. 

J. C. Rasmussenet al., “Lymphatic imaging in humans with near-infrared fluorescence,” Curr. Opin. Biotechnol., 20 (1), 74 –82 (2009). https://doi.org/http://dx.doi.org/10.1016/j.copbio.2009.01.009 CUOBE3 0958-1669 Google Scholar

20. 

Y. Tajimaet al., “Sentinel node mapping guided by indocyanine green fluorescence imaging in gastric cancer,” Ann. Surg., 249 (1), 58 –62 (2009). https://doi.org/http://dx.doi.org/10.1097/SLA.0b013e3181927267 ANSUA5 0003-4932 Google Scholar

21. 

S. L. Troyanet al., “The FLARE intraoperative near-infrared fluorescence imaging system: a first-in-human clinical trial in breast cancer sentinel lymph node mapping,” Ann. Surg. Oncol., 16 (10), 2943 –2952 (2009). https://doi.org/http://dx.doi.org/10.1245/s10434-009-0594-2 1068-9265 Google Scholar

22. 

B. T. Leeet al., “The FLARE intraoperative near-infrared fluorescence imaging system: a first-in-human clinical trial in perforator flap breast reconstruction,” Plast. Reconstr. Surg., 126 (5), 1472 –1481 (2010). https://doi.org/http://dx.doi.org/10.1097/PRS.0b013e3181f059c7 PRSUAS 0032-1052 Google Scholar

23. 

D. Z. Liuet al., “The application of indocyanine green fluorescence angiography in plastic surgery,” J. Reconstr. Microsurg., 27 (6), 355 –364 (2011). https://doi.org/http://dx.doi.org/10.1055/s-0031-1281515 JRMIE2 0743-684X Google Scholar

24. 

S. Keereweeret al., “Image-guided surgery in head and neck cancer: current practice and future directions of optical imaging,” Head Neck, 34 (1), 120 –126 (2012). https://doi.org/http://dx.doi.org/10.1002/hed.v34.1 HEANEE 1097-0347 Google Scholar

25. 

Y. Sunet al., “Fluorescence lifetime imaging microscopy for brain tumor image-guided surgery,” J. Biomed. Opt., 15 (5), 056022 (2010). https://doi.org/http://dx.doi.org/10.1117/1.3486612 JBOPFO 1083-3668 Google Scholar

26. 

J. R. van der Vorstet al., “Near-infrared fluorescence-guided resection of colorectal liver metastases,” Cancer, 119 (18), 3411 –3418 (2013). https://doi.org/http://dx.doi.org/10.1002/cncr.28203 Google Scholar

27. 

F. P. Verbeeket al., “Intraoperative near infrared fluorescence guided identification of the ureters using low dose methylene blue: a first in human experience,” J. Urol., 190 (2), 574 –579 (2013). https://doi.org/http://dx.doi.org/10.1016/j.juro.2013.02.3187 JOURDD 0248-0018 Google Scholar

28. 

M. Huttemanet al., “Near-infrared fluorescence imaging in patients undergoing pancreaticoduodenectomy,” Eur. Surg. Res., 47 (2), 90 –97 (2011). https://doi.org/http://dx.doi.org/10.1159/000329411 EUSRBM 0014-312X Google Scholar

29. 

M. J. Pucciet al., “Laparoscopic approaches to gastric gastrointestinal stromal tumors: an institutional review of 57 cases,” Surg. Endosc., 26 (12), 3509 –3514 (2012). https://doi.org/http://dx.doi.org/10.1007/s00464-012-2374-5 SUREEX 1432-2218 Google Scholar

30. 

S. WatanabeH. Asamura, “Lymph node dissection for lung cancer: significance, strategy, and technique,” J. Thorac. Oncol., 4 (5), 652 –657 (2009). https://doi.org/http://dx.doi.org/10.1097/JTO.0b013e31819cce50 JTOOB7 1556-0864 Google Scholar

31. 

B. WitteM. Hurtgen, “Video-assisted mediastinoscopic lymphadenectomy (VAMLA),” J. Thorac. Oncol., 2 (4), 367 –369 (2007). https://doi.org/http://dx.doi.org/10.1097/01.JTO.0000263725.89512.d7 JTOOB7 1556-0864 Google Scholar

32. 

A. Adleret al., “Narrow-band versus white-light high definition television endoscopic imaging for screening colonoscopy: a prospective randomized trial,” Gastroenterology, 136 (2), 410 –416 (2009). https://doi.org/http://dx.doi.org/10.1053/j.gastro.2008.10.022 GASTAB 0016-5085 Google Scholar

33. 

P. A. de LeeuwM. N. van SterkenburgC. N. van Dijk, “Arthroscopy and endoscopy of the ankle and hindfoot,” Sports Med. Arthrosc., 17 (3), 175 –184 (2009). https://doi.org/http://dx.doi.org/10.1097/JSA.0b013e3181a5ce78 1062-8592 Google Scholar

34. 

M. Loninget al., “Laparoscopic fluorescence detection of ovarian carcinoma metastases using 5-aminolevulinic acid-induced protoporphyrin IX,” Cancer, 100 (8), 1650 –1656 (2004). https://doi.org/http://dx.doi.org/10.1002/(ISSN)1097-0142 60IXAH 0008-543X Google Scholar

35. 

D. C. Grayet al., “Dual-mode laparoscopic fluorescence image-guided surgery using a single camera,” Biomed. Opt. Express, 3 (8), 1880 –1890 (2012). https://doi.org/http://dx.doi.org/10.1364/BOE.3.001880 BOEICL 2156-7085 Google Scholar

36. 

M. Mitsunagaet al., “In vivo longitudinal imaging of experimental human papillomavirus infection in mice with a multicolor fluorescence mini-endoscopy system,” Cancer Prev. Res., 4 (5), 767 –773 (2011). https://doi.org/http://dx.doi.org/10.1158/1940-6207.CAPR-10-0334 CPRACC 1940-6207 Google Scholar

37. 

D. M. GilmoreO. V. KhullarY. L. Colson, “Developing intrathoracic sentinel lymph node mapping with near-infrared fluorescent imaging in non-small cell lung cancer,” J. Thorac. Cardiovasc. Surg., 144 (3), S80 –S84 (2012). https://doi.org/http://dx.doi.org/10.1016/j.jtcvs.2012.05.072 JTCSAQ 0022-5223 Google Scholar

38. 

S. Yamashitaet al., “Sentinel node navigation surgery by thoracoscopic fluorescence imaging system and molecular examination in non-small cell lung cancer,” Ann. Surg. Oncol., 19 (3), 728 –733 (2012). https://doi.org/http://dx.doi.org/10.1245/s10434-011-2145-x 1068-9265 Google Scholar

39. 

V. Venugopalet al., “Real-time endoscopic guidance using near-infrared fluorescent light for thoracic surgery,” Proc. SPIE, 8572 85720R (2013). https://doi.org/http://dx.doi.org/10.1117/12.2004615 PSISDG 0277-786X Google Scholar

40. 

P. S. Thonget al., “Clinical application of fluorescence endoscopic imaging using hypericin for the diagnosis of human oral cavity lesions,” Br. J. Cancer, 101 (9), 1580 –1584 (2009). https://doi.org/http://dx.doi.org/10.1038/sj.bjc.6605357 BJCAAI 0007-0920 Google Scholar

41. 

R. A. Cahillet al., “Near-infrared (NIR) laparoscopy for intraoperative lymphatic road-mapping and sentinel node identification during definitive surgical resection of early-stage colorectal neoplasia,” Surg. Endosc., 26 (1), 197 –204 (2012). https://doi.org/http://dx.doi.org/10.1007/s00464-011-1854-3 SUREEX 1432-2218 Google Scholar

42. 

Y. Ashitateet al., “Real-time simultaneous near-infrared fluorescence imaging of bile duct and arterial anatomy,” J. Surg. Res., 176 (1), 7 –13 (2012). https://doi.org/http://dx.doi.org/10.1016/j.jss.2011.06.027 JSGRA2 0022-4804 Google Scholar

43. 

A. Matsuiet al., “Real-time intra-operative near-infrared fluorescence identification of the extrahepatic bile ducts using clinically available contrast agents,” Surgery, 148 (1), 87 –95 (2010). https://doi.org/http://dx.doi.org/10.1016/j.surg.2009.12.004 SURGAZ 0039-6060 Google Scholar

44. 

J. Glatzet al., “Concurrent video-rate color and near-infrared fluorescence laparoscopy,” J. Biomed. Opt., 18 (10), 101302 (2013). https://doi.org/http://dx.doi.org/10.1117/1.JBO.18.10.101302 JBOPFO 1083-3668 Google Scholar

45. 

M. J. van Detet al., “Optimal ergonomics for laparoscopic surgery in minimally invasive surgery suites: a review and guidelines,” Surg. Endosc., 23 (6), 1279 –1285 (2009). https://doi.org/http://dx.doi.org/10.1007/s00464-008-0148-x SUREEX 1432-2218 Google Scholar

46. 

F. Ayerset al., “Fabrication and characterization of silicone-based tissue phantoms with tunable optical properties in the visible and near infrared domain,” Proc. SPIE, 6870 6870E (2008). https://doi.org/http://dx.doi.org/10.1117/12.764969 PSISDG 0277-786X Google Scholar

47. 

S. Giouxet al., “First-in-human pilot study of a spatial frequency domain oxygenation imaging system,” J. Biomed. Opt., 16 (8), 086015 (2011). https://doi.org/http://dx.doi.org/10.1117/1.3614566 JBOPFO 1083-3668 Google Scholar

48. 

H. S. Choiet al., “Synthesis and in vivo fate of zwitterionic near-infrared fluorophores,” Angew. Chem., 50 (28), 6258 –6263 (2011). https://doi.org/http://dx.doi.org/10.1002/anie.201102459 ANCEAD 0044-8249 Google Scholar

49. 

M. C. JacobsonR. deVere WhiteS. G. Demos, “In vivo testing of a prototype system providing simultaneous white light and near infrared autofluorescence image acquisition for detection of bladder cancer,” J. Biomed. Opt., 17 (3), 036011 (2012). https://doi.org/http://dx.doi.org/10.1117/1.JBO.17.3.036011 JBOPFO 1083-3668 Google Scholar

50. 

M. A. Funovicset al., “Simultaneous fluorescence imaging of protease expression and vascularity during murine colonoscopy for colonic lesion characterization,” Gastrointest. Endosc., 64 (4), 589 –597 (2006). https://doi.org/http://dx.doi.org/10.1016/j.gie.2006.02.048 GAENBQ 0016-5107 Google Scholar

Biographies of the authors are not available.

© 2013 Society of Photo-Optical Instrumentation Engineers (SPIE) 0091-3286/2013/$25.00 © 2013 SPIE
Vivek Venugopal, Minho Park, Yoshitomo Ashitate, Florin Neacsu, Frank Kettenring, John V. Frangioni, Sidhu P. Gangadharan, and Sylvain Gioux "Design and characterization of an optimized simultaneous color and near-infrared fluorescence rigid endoscopic imaging system," Journal of Biomedical Optics 18(12), 126018 (20 December 2013). https://doi.org/10.1117/1.JBO.18.12.126018
Published: 20 December 2013
JOURNAL ARTICLE
10 PAGES


SHARE
Advertisement
Advertisement
Back to Top