2.1.

DCF-Based Imaging Catheter

A schematic of the system used to combine these various modalities is shown in Fig. 1. The custom-made DCF catheter [Fig. 1(a)] consists of a polyimide-coated DCF (9/105/125-20PI, FUD-3489, Nufern, East Granby, Connecticut), in which the core carries the RGB, AFI, and OCT illumination light and the inner cladding returns the AFI and RGB reflectance signals. Scattered OCT light is returned through the core. A no-core fiber (NCF, NCF125, POFC, Hsinchu, Taiwan) is spliced onto the DCF and polished at 55 deg relative to the cleave plane to reflect the diverging imaging beam sideways and slightly forward. The fiber assembly is sealed in a $152\text{-}\mu \mathrm{m}$ outer diameter polyethylene terephthalate (PET) shrink tube to trap air against the polished surface. The entire length is then fixed inside a $500\text{-}\mu \mathrm{m}$ outer diameter double-wound torque cable (Heraeus Medical Components, St. Paul, Minnesota) to transmit rotational and pullback motion from the proximal end to the distal end. A stationary plastic window tube ($900\text{-}\mu \mathrm{m}$ outer diameter and $711\text{-}\mu \mathrm{m}$ inner diameter) covers the entire rotating assembly.

Fig. 1

(a) Schematic diagram of DCF-based imaging catheter and (b) the endoscopic system incorporating narrowband RGB reflectance imaging, AFI, and OCT. The imaging catheter consists of an NCF spliced to the DCF and polished at 55 deg relative to the cleave plane. A torque cable is attached for rotary and pullback motion. The assembly is encapsulated within a $900\text{-}\mu \mathrm{m}$ window tube. WDM, wavelength division multiplexer; DCFC, double-clad fiber coupler; FORJ, fiber-optic rotary joint; RPD, rotary-pullback drive; LP, long pass; SP, short pass; BP, bandpass; PD, photodetector; SMF, single-mode fiber; DCF, double-clad fiber; MMF, multimode fiber; and BPD, balanced photodetector.

The rotational and pullback scanning operation of the DCF-based imaging catheter allows the endoscope to capture 3-D helical scans of tissues. Sampling at a rate of 100 kHz (set to match the laser sweep speed of the infrared OCT laser used as part of the OCT subsystem) allows the system to acquire up to 3072 A-line and reflectance samples per rotation (at a 33-Hz rotational frequency). The imaging catheter can cover an FOV of up to $(5.65\times 140){\mathrm{mm}}^2$, achieving a pixel count of $\sim 8200\mathrm{k}$ for RGB reflectance, AFI, and en face OCT images (using a pullback speed of $0.5\mathrm{mm}/\mathrm{s}$).

The original OCT-AFI catheters contained an MMF and graded-index (GRIN) fiber spliced to the DCF to improve OCT resolution. As illumination and detection light traveling through common optical elements can frequently introduce unwanted back reflections,²⁴ the MMF and GRIN were omitted to decrease the number of index of refraction interfaces and splices within the catheter, thereby reducing the amount of back-reflected light and improving the signal-to-noise ratio (SNR) of the system.

2.2.

Narrowband RGB Imaging

The imaging system is shown in Fig. 1(b). Fiber-pigtailed laser diodes (Thorlabs Inc., Newton, New Jersey) emitting 450 nm (LP450-SF15), 520 nm (LP520-SF15), and 635 nm (LP635-SF8) light are connected to a wavelength division multiplexer (WDM, RGB26HF, Thorlabs Inc., Newton, New Jersey) to obtain fiber-coupled “white” excitation. A second WDM (WDM6513F, Thorlabs Inc., Newton, New Jersey) combines the RGB light with the infrared light from the OCT subsystem. This light travels through the core of the DCFC (9/105/125, DC1300LEFA, Thorlabs Inc., Newton, New Jersey) and into a custom DCF FORJ (Princetel Inc., Pennington, New Jersey). Custom software controls the rotary-pullback drive (RPD), which spins the catheter and pulls it back at specified speeds from 0.5 to $10\mathrm{mm}/\mathrm{s}$.

RGB light from the core of the catheter illuminates the sample, and the reflected light is collected by the inner cladding of the catheter, coupled back through the FORJ and into the MMF output of the DCFC. Upon exiting the MMF, the beam is expanded using a $20\times$, 0.75 NA microscope objective (Fluor, Carl Zeiss Microscopy GmbH, Oberkochen, Germany).

The spatially expanded reflection light is split into red, green, and blue light using a series of filters. These include a 490-nm long-pass dichroic mirror (DMLP490, Thorlabs Inc., Newton, New Jersey), a 500-nm short-pass filter (FES0500, Thorlabs Inc., Newton, New Jersey), a 605-nm long-pass dichroic mirror (DMLP605, Thorlabs Inc., Newton, New Jersey), a 550/88-nm bandpass filter (FF01-550/88-25, Semrock Inc., Rochester, New York), and a 610-nm long-pass filter (610LP RapidEdge, Omega Optical Inc., Brattleboro, Vermont). Three silicon photodiode detectors with switchable gain (PDA100A2, Thorlabs Inc., Newton, New Jersey) are used to capture the individual signals.

2.3.

Optical Coherence Tomography and Autofluorescence Imaging

The OCT subsystem is a fiber-based Mach–Zehnder interferometer, as described in our previous publications.^13–14,21 Infrared illumination is supplied from a 100-kHz wavelength-swept laser source (SSOCT-1310, Axsun Technologies Inc., Billerica, Massachusetts) centered at 1310 nm and with a bandwidth of 100 nm.

The second WDM (WD6315F, Thorlabs Inc., Newton, New Jersey) splits the OCT reflected signal from the RGB light. OCT reflected light is transmitted through the core of the DCFC and is detected with a balanced photodetector (PDB420A, Thorlabs, Newton, New Jersey).

To perform AFI, the photodetectors in Fig. 1(b) are replaced with photomultiplier tubes (PMTs, H10723-20, Hamamatsu Photonics, Shizuoka, Japan) due to the increased sensitivity needed to detect autofluorescence. Only the blue laser is turned on, and the system captures green and red fluorescence emissions along with blue reflectance simultaneously.

2.4.

Broadband Imaging

To perform broadband RGB imaging, the laser diodes are replaced by a supercontinuum broadband white-infrared laser source with a wavelength range of $\sim 430$ to 2600 nm (Fianium WhiteLase, NKT Photonics, Birkerød, Denmark). The light is attenuated by a pair of neutral density filter wheels (FW2AND, Thorlabs, Newton, New Jersey) and connected to the illumination port of the DCFC with an SMF-28 patch cord.

The filters in the detection end of the system are replaced to better replicate conventional RGB imaging given the broadband laser source. The 500-nm short pass filter is replaced by a 455-nm SP filter. The 550/88-nm bandpass filter is replaced by a filter with a narrower bandwidth of $535/40\mathrm{nm}$ (BP 535/40, Delta Optical Thin Film, Hørsholm, Denmark). The 610-nm long-pass filter is replaced by a 630/60-nm bandpass filter (BP 630/60 TP, Delta Optical Thin Film, Hørsholm, Denmark) to reduce the amount of infrared light entering the red photodetector.

2.5.

Image Processing

The RGB channels were scaled according to the signal from a reflectance-fluorescence target used for white-balancing (uniform reflection of light with respect to wavelength), thereby allowing a normalization of the intensity relationships of the color channels with each other. This information was used for color-balancing the narrowband and broadband RGB images in postprocessing.²⁵

As the fiber is not rotationally rigid, rotating the imaging catheter can result in nonuniform rotational distribution (NURD). To reduce the amount of NURD in some of the images, a motion correction method developed by our group denoted “azimuthal en face image registration” was applied to some of the images during postprocessing.²⁶

3.1.

SNR and Resolution

To measure the backscattered signal from a resolution target, the rotating imaging catheter was placed on a flat fluorescence-reflectance ceramic target (Avian Technologies, New London, New Hampshire). The signal from the imaging catheter for a number of rotations without pullback was acquired. The SNR was determined by

Table 1 shows the SNR of the imaging system for each color channel. The highest SNR was produced by the red channel despite its lower average signal intensity since it displayed the lowest $\sigma$. The blue and green channels exhibited comparable values of average intensity, noise, and SNR.

Table 1

Signal-to-noise ratio (SNR) values for each color channel.

The resolution of the system was determined using a variable frequency reflectance resolution target (variable frequency target 5 to $120\mathrm{lp}/\mathrm{mm}$, Edmund Optics, New Jersey). The imaging catheter was oriented on top of the target in both the rotary direction (imaging catheter parallel to the target grating) and in the pullback direction (imaging catheter perpendicular to the target grating), the results of which are shown in Fig. 2. Using narrowband RGB reflectance, the imaging system was able to resolve $\sim 25\mu \mathrm{m}$ in both the rotary direction and the pullback direction. Similar target grating images were recorded for the autofluorescence and OCT imaging modes. These figures fall within the ballpark of expected resolution range of magnification endoscopes.¹⁶

Fig. 2

RGB reflectance image of a resolution target indicating fiber-optic resolution in the pullback direction. The resolving power in the rotary and pullback directions is $\sim 25\mu \mathrm{m}$ as the grating frequencies representing these line spacings are still distinguishable.

3.2.

RGB Reflectance Imaging

A white paper card printed with red, green, blue, and black (RGBK) lines was imaged to demonstrate the color recognition abilities of the system. The imaging catheter was placed onto the card and a rotary pullback was performed to obtain these images. The grayscale images of the red, green, and blue channels were combined to produce an RGB image of the card (Fig. 3). The resolution of the system is sufficient to visualize the wood fibers making up the paper card.

Fig. 3

RGB reflectance images of a color card. Printed on the card in a repeated pattern are red (R), green (G), blue (B), and black (K) lines. The fiber, while pressed down on the card, was pulled back perpendicular to the lines. (a)–(c) The false-color images obtained from each color channel and (d) the composite RGB image.

Figures 3(a)–3(c) show that the light shone on dissimilar colored lines was absorbed more and reflected less, hence the darker appearance of the lines. Even the color lines matching the color of the illumination light are slightly darker due to the partial absorption of the light by the printer ink. Overall, the RGB composite [Fig. 3(d)] compared well with the visual appearance of the color card.

Figure 4(a) shows the RGB reflectance image of a tongue acquired using a finger to press the imaging catheter against the tongue. The most prominent details are the papillae on the tongue, showing that the system can capture surface structure information from tissue. The ventral (underside) of the tongue was similarly imaged to capture blood vessels visible on the surface, which correspond to the dark and slightly blue streaks in Fig. 4(b). This demonstrates the system’s ability to capture surface details of tissue and to visualize vasculature within tissue.

Fig. 4

RGB reflectance images of (a) the top of a tongue and (b) the ventral (underside) of the tongue. Fingerprints, tongue papillae, and blood vessel features are captured.

While performing this image acquisition, an air bubble between the PET shrink tube and the window tube was dragged by the imaging catheter during part of the fiber pullback [Fig. 4(b)].

3.3.

Optical Coherence Tomography and Autofluorescence Images

Figure 5 shows simultaneously acquired co-registered RGB reflectance and OCT images. Although the same structures are visible, the details are different between the RGB intensity image [Fig. 5(a), transformed into grayscale for the comparison] and the en face OCT image [Fig. 5(b)]. The dashed lines superimposed on images (a) and (b) indicate the rotational angle for which the depth cross section is shown in the longitudinal OCT image in Fig. 5(c).

Fig. 5

RGB reflectance: (a) grayscale for feature emphasis, (b) en face mean intensity projection OCT, and (c) longitudinal OCT co-localized with dashed lines on (a) and (b).

To perform AFI, only blue excitation light was used to illuminate the sample. Autofluorescence and reflectance images of the RGBK color card were taken with PMTs (Fig. 6).

Fig. 6

Autofluorescence and reflectance images of the RGBK card taken with photomultiplier tubes: (a) red autofluorescence, (b) green autofluorescence, (c) blue reflectance, (d) co-registered red and green autofluorescence, and (e) co-registered red and green autofluorescence and blue reflectance. It appears in the image that the red ink used to mark the card exhibits strong red autofluorescence when illuminated with blue light.

The red autofluorescence image [Fig. 6(a)] shows that the red lines on the card fluoresce significantly. The green lines in the green autofluorescence image [Fig. 6(b)] of the card do not fluoresce nearly as much. The finger pressing the imaging catheter down onto the card is still visible in both images, but it does not show up in the blue reflectance image [Fig. 6(c)] due to the higher reflectivity of the card compared with that of the finger. Figure 6(d) shows co-registered red and green autofluorescence, whereas Fig. 6(e) shows co-registered autofluorescence from the red and green channels and blue reflectance.

The autofluorescence and reflectance images from a finger are shown in Fig. 7. Sweat pores along the fingers are particularly visible in the green autofluorescence image as low-fluorescence spots on the ridges of the fingerprint [Fig. 7(b)]. The fingerprint in the blue reflectance image [Fig. 7(c)] does not have as much contrast as the autofluorescence images due to the amount of surface backscatter obtained together with the signal. The co-registered red and green autofluorescence image is shown in Fig. 7(d), whereas the co-registered red and green autofluorescence and blue reflectance image is shown in Fig. 7(e).

Fig. 7

Autofluorescence and reflectance images of fingers taken with photomultiplier tubes: (a) red autofluorescence emission, (b) green autofluorescence emission, (c) blue reflectance, (d) co-registered red and green autofluorescence, and (e) co-registered red and green autofluorescence and blue reflectance. The arrows in (b) indicate sweat pores that run along the ridges of the fingerprint.

The calibration step performed for RGB reflectance imaging is not performed for the autofluorescence signals. Instead, these signals are contrast stretched such that their lower 1% and upper 99% intensities are scaled to 0 and 255, respectively.

3.4.

Broadband Images

Shown in Fig. 8(a) is the broadband RGB image of a finger captured using the supercontinuum white laser light as the illumination source. The system produced images with improved contrast evident in the broadband image of fingerprints compared with its narrowband RGB laser reflectance equivalent (Fig. 4). Striations between fingerprint ridges are now visible, as shown in Fig. 8(b) (contrast-enhanced). These are not visible in the narrowband RGB images of fingers. Vertical green bands present in Fig. 8(a) are artifacts due to wavelength-dependent output fluctuations from the supercontinuum laser.

Fig. 8

(a) Broadband RGB reflectance image of a finger using a supercontinuum white light source and (b) the contrast-enhanced close-up image further fingerprint structure in the form of striations in between ridges.

OCT can be performed at the same time as either RGB reflectance or AF, but since the red and green detectors are shared between reflectance and AFI modes, acquisition of each mode must be done separately. The endoscopic imaging system presented here can further be improved by pulsing the laser diodes to separate the acquisition, RGB, and autofluorescence modes in time, and the three modalities would be able to function simultaneously.

The endoscopic imaging system shows great potential to perform the in vivo imaging of intraepithelial lesions within small lumina in various sites, such as the cervix, fallopian tubes, and lungs. Accomplishing this would allow for the early detection and localization of cancers in these sites.