Translator Disclaimer
20 June 2019 En-face optical coherence tomography/fluorescence endomicroscopy for minimally invasive imaging using a robotic scanner
Author Affiliations +
Abstract
We report a compact rigid instrument capable of delivering en-face optical coherence tomography (OCT) images alongside (epi)-fluorescence endomicroscopy (FEM) images by means of a robotic scanning device. Two working imaging channels are included: one for a one-dimensional scanning, forward-viewing OCT probe and another for a fiber bundle used for the FEM system. The robotic scanning system provides the second axis of scanning for the OCT channel while allowing the field of view (FoV) of the FEM channel to be increased by mosaicking. The OCT channel has resolutions of 25  /  60  μm (axial/lateral) and can provide en-face images with an FoV of 1.6  ×  2.7  mm2. The FEM channel has a lateral resolution of better than 8  μm and can generate an FoV of 0.53  ×  3.25  mm2 through mosaicking. The reproducibility of the scanning was determined using phantoms to be better than the lateral resolution of the OCT channel. Combined OCT and FEM imaging were validated with ex-vivo ovine and porcine tissues, with the instrument mounted on an arm to ensure constant contact of the probe with the tissue. The OCT imaging system alone was validated for in-vivo human dermal imaging with the handheld instrument. In both cases, the instrument was capable of resolving fine features such as the sweat glands in human dermal tissue and the alveoli in porcine lung tissue.

1.

Introduction

Recent advances in surgical techniques, particularly minimally invasive keyhole oncological procedures, have seen significant improvements in patient outcomes, leading to improved recovery times and reduced complications.1 A potential avenue for further improvement is the introduction of intraoperative image guidance, which would allow identification of tumor margins in situ. This could enable a complete resection of cancerous tissue while preserving as many healthy tissues as possible (e.g., Ref. 2). Provided that high-quality rapid diagnosis can be achieved, “optical biopsy” techniques could offer a real-time alternative to current intraoperative techniques such as time-consuming and expensive frozen section biopsy.

To compete effectively with conventional excisional biopsy and histology or frozen section, optical biopsy techniques must be able to provide comparable diagnostic information, in or near real time, and should be minimally invasive and easy to integrate with the clinical workflow. Where morphological imaging is used, the technique must provide sufficient resolution to allow identification of relevant tissue features, and sufficient field of view (FoV) for an effective reading of the images. A number of optical biopsy techniques have been proposed, ranging from low-resolution wide-field fluorescence imaging2 to point-based spectroscopic measurements.3 One such promising candidate is optical coherence tomography (OCT). This is an inherently noncontact-based optical technique using low-power, near-infrared light, allowing full three-dimensional (3-D) imaging of the tissue with a high penetration depth of typically 1 to 2 mm, and having imaging rates of a few kilohertz (with volume rates of a few hertz4). To allow for in-situ imaging, a wide range of side-viewing and forward-viewing (FV) endoscopic OCT probes have been developed5 and potential surgical applications have been identified (e.g., Ref. 6). Depending on the kind of tissue being investigated, the imaging contrast provided by OCT might not be sufficient for clinical diagnosis, since it stems solely from single backscattering events and changes in the refractive index. Therefore, it is recommendable to combine OCT with other optical and nonoptical modalities,711 to enable more comprehensive characterization of the tissue properties.

Another approach to optical biopsy is the family of endoscopic fluorescence microscopy techniques, including both wide-field (epi)-fluorescence endomicroscopy (FEM)12,13 and fluorescence confocal laser endomicroscopy (CLE).14,15 FEM/CLE have demonstrated the potential to distinguish and grade cancerous tissue in situ and has been suggested as a potential intraoperative tool for rapid tissue imaging in breast surgery,16 neurosurgery,17 and laparoscopic surgery.18 Fiber imaging bundles, with or without distal micro-optics, can be used as thin, flexible, and passive FEM/CLE probes, removing the need for any distal scanning systems. However, a downside is that the FoV of fiber bundle-based FEM/CLE probes is small, typically 0.25 to 0.8 mm, depending on the selection of the distal optics.

To address this limitation, several systems for mechanical and robotic scanning of optical biopsy probes have been proposed. For example, Rosa et al.19 developed a system for surgical imaging that increased the effective FoV by scanning a CLE probe in a spiral pattern, using visual servoing to optimize the trajectory. Zhang et al.20 integrated CLE and OCT probes with the da Vinci surgical robot, demonstrating closed-loop scanning using information from both imaging channels. Zuo et al.21,22 demonstrated prototypes of several open-loop scanning systems specifically designed for intraoperative breast imaging. The operation of such systems has indicated the potential for a scanning FEM/CLE system to obtain images over significant areas of tissue.

FEM/CLE and OCT offer potentially complementary imaging modes. FEM/CLE provide high-resolution two-dimensional (2-D) surface scans but with little penetration depth and a small FoV. OCT generally has lower resolution but offers 3-D volumetric imaging, with penetration of 1 to 2 mm into tissue (although real-time OCT imaging typically comprises 2-D cross-sectional “B” scans, particularly in endoscopic implementations). Taking full advantage of the respective strengths of OCT and FEM/CLE requires a dual-modality imaging probe, combining OCT with fluorescence imaging.811

A dual-modality probe would have the benefit of enabling en-face OCT views, in addition to cross-sectional OCT images. This would provide more flexibility in imaging while also permitting coregistration with the inherently en-face fluorescence images. While OCT is capable of delivering low frame rate en-face slices extracted from 3-D volumes, this is particularly challenging for miniaturized surgical imaging systems. In particular, given that en-face images are normally constructed by raster scanning the beam over the sample, two orthogonally placed scanners are necessary. While this is common in bulk, bench-top OCT systems for a variety of applications, ranging from ophthalmic imaging systems to handheld dermal probes, introducing two scanning directions in a small form-factor, minimally invasive probe is not straightforward. Typically, it involves a cantilevered fiber being driven by either a piezoelectric device over a Lissajous or spiral pattern2326 (which however requires high voltages to be operated), or employing small AC motors and exploiting the resonance properties of the cantilevered fiber.27,28 Approaches using coherent imaging fiber bundles, with no distal scanning, have also been reported,29 but the image quality is compromised by cross talk between fiber cores, the few-mode behavior of most fiber bundles, and their high numerical apertures (NA).

Although side-viewing geometries in multimodal imaging probes are now commonplace,8,10,11 there are fewer reports on FV, multimodal probes. Ryu et al.9 have reported a double-clad fiber (DCF)-based probe, which can simultaneously deliver OCT and fluorescence imaging; however, no in-probe scanning optics were used, with the sample being translated in relation to the probe in order to construct an OCT/fluorescence en-face image.

In this paper, we attempt to address the issue of en-face OCT imaging in a small-scale, minimally invasive probe combined with a fiber bundle-based FEM system. This is achieved by the use of a rigid robotic scanning device, originally designed for CLE and laser ablation,30 into which a one-dimensional (1-D) FV OCT probe27,31 and an FEM fiber bundle are introduced. The robotic device enables en-face OCT imaging by providing a second, orthogonal scanning direction for the OCT probe. In addition, to compensate for the small FoV of the FEM probe, a mosaicking algorithm32 is used to fuse the FEM images obtained during scanning of the robotic device. Since the robotic scanning device has a relatively small physical footprint, it can be used handheld, which brings benefits in terms of cost and surgical workflow.33

The system is described in Sec. 2, which details (i) the robotic scanning device, (ii) the OCT 1-D FV probe and system, (iii) the FEM fiber bundle probe, and (iv) system integration and software. The system is fully characterized in Sec. 3, and examples of images acquired with both arm-mounted and handheld operation are presented and discussed in Sec. 4.

2.

Experimental Setup

A schematic representation of the robotic scanning OCT/FEM system is shown in Fig. 1(a). The computer workstation (PC Specialist custom build, Intel i7-5960X octo-core processor, 16 GB RAM) is used for the acquisition and control of the three subsystems: motor control for the robotic scanning device, the OCT endoscopic scanning system, and the FEM imaging system. These are detailed in Secs. 2.12.3, respectively. Custom LabVIEW™ (National Instruments, Austin, Texas) software was devised to control the three subsystems and to acquire and process the data, as described in Sec. 2.4.

Fig. 1

(a) Schematic diagram depicting the complete system with the three subsystems and the probe; (b) photograph of the individual probes laid side by side [robotic scanning device (i), OCT 1-D FV probe (ii), and FEM imaging bundle (iii)]. Scanning line (iv) on the infrared detector card in (b) shows the 1-D OCT scanning pattern at a significant (10 to 20 mm) distance from the probe end, and (v) shows the excitation light for the FEM imaging emanating from the fiber bundle tip; (c) photograph of the distal end of the probe with the two imaging probes assembled inside the robotic scanning probe tip; subfigures (d), (e), and (f) detail the motor control system, OCT imaging, and FEM imaging subsystems, respectively. BPD, balanced photodetector; DC, directional coupler; PC, polarization controller; DMD, digital micromirror device; SPF, short-pass filter; LPF, long-pass filter.

JBO_24_6_066006_f001.png

The two fiber probes (the OCT 1-D FV probe and the fiber bundle for the FEM imaging system) are placed side by side inside the tip of the robotic scanning device [inner diameter of 2.7 mm; outer diameter (OD) of 3.3 mm along the shaft; and 3.7 mm at the tip], as shown in Fig. 1(c), and are shown individually in the photograph in Fig. 1(b). In Figs. 1(b) and 1(c), (i) indicates the robotic scanning device shaft, (ii) indicates the OCT probe [projecting a linear scanning pattern (iv) on the infrared detection card], and (iii) indicates the FEM fiber bundle [projecting a divergent blue/violet beam (v)].

2.1.

Robotic Scanning Device

The robotic scanning device, which may be mounted on a robotic arm for autonomous or hands-on operation, or used in a handheld fashion, is described in Refs. 30 and 34. In summary, it consists of a 58-mm-long hollow steel tube, with 3.3 mm OD, through which imaging probes can be passed, mounted inside a 3-D printed case. The tube is fixed in place at the back of the case while the distal (tissue) end protrudes from the case and is free to move. Partway along the length of the tube, at the front of the case, the tube is fixed to a cam-roller assembly, which allows it to be deflected in two dimensions by the rotation of two brushless DC servomotors. This allows the tip of the tube to be moved over a 2-D workspace of up to 14  mm2 with an absolute positioning accuracy of better than 30  μm. The tube is offset within the assembly so that it is always under the load throughout the entire workspace, minimizing backlash and hysteresis effects.

The robotic scanner is driven via two analog input signals applied to the motor controllers, which are mounted in a separate casing to minimize the weight of the scanner. For the study reported here, only 1-D motion was required. For ease of alignment of the OCT scanning probe direction, this motion was not necessarily along any of the intrinsic axes of the scanner. The scan was, therefore, performed by sending linear ramps to both motors, scaled by cosθ and sinθ, where θ is the angle of the OCT 1-D scanning line with respect to the axis of the scanner.

At the tip of the tube, a holder with an OD of 3.7 mm is fixed, originally designed to hold a Mauna Kea Cellvizio Gastroflex UHD endomicroscopy probe (diameter of 2.6 mm) and a laser ablation fiber (diameter of approximately 0.7 mm), as seen in the photograph in Fig. 1(c). For this study, the OCT probe was passed through the endomicroscopy channel and the FEM probe was passed through the laser ablation channel of the original design. The axial positions of the probes were fixed at the rear end of the device case. The OCT probe was mounted with its tip slightly (1 to 2 mm) behind the FEM fiber bundle tip, which was placed flush against the distal end of the 3.7 mm OD holder.

2.2.

Optical Coherence Tomography Endoscopic Scanning Subsystem

The OCT endoscopic scanning subsystem shown in Fig. 1(e) is a swept-source-based system (SS-OCT) that incorporates an FV, 1-D scanning endoscopic probe in the object arm.

This probe, the principle of which has been described elsewhere,31,35 is based on the voice coil principle, employing the optical fiber as a cantilever, which is attached to the electrical coil, as pictured in Fig. 2. The electrical coil is placed around a magnetic system comprising two magnets with the same poles facing each other. The probe operates in an open-loop configuration, with no position sensing. The fiber tip is imaged onto the sample by a gradient-index (GRIN) lens with a magnification of 5×, with a working distance of 1 mm. By applying an alternating electrical current through the coil, a force is generated that laterally shifts the fiber tip, creating a raster scan pattern of up to 2 mm on the sample. The mechanical dimensions of the probe are provided in Fig. 1(b). The part labeled as (ii) has an OD of 1.81 mm at the metal tip, which has a rigid length of 13.20 mm. Elsewhere, the probe tubing has an OD of 1.71 mm.

Fig. 2

Schematic diagram of the distal end of the OCT endoscope, based on a diagram from Ref. 35, showing the different elements present in it. Scanning direction (vertical) is shown by the blue arrow.

JBO_24_6_066006_f002.png

Briefly, the SS-OCT system is driven by a 100 kHz A-scan rate, 1310 nm wavelength swept-source, with a bandwidth of 110 nm (Axsun Technologies, Billerica, Massachusetts), the output of which is directed to a fiber-based Mach–Zehnder interferometer, whose arms include the 1-D scanning endoscopic probe and a fiber-based reference arm. The optical power incident on the sample is 2.8  mW. Detection is performed by a balanced InGaAs photodetector, BPD (PDB435C-AC, 350 MHz cut-off frequency, from Thorlabs, Newton, New Jersey), whose electrical signal is digitized after a passive high-pass filter (10 MHz cut-off frequency) by a 12-bit high-speed PCIe digitizer (AlazarTech ATS9350, up to 500  MS/s sampling rate, Pointe-Claire, Quebec, Canada), configured with a voltage range of ±1  V and digitizing 2048 points per each spectral sweep at 500  MS/s.

To produce OCT images, data are processed using the Complex Master–Slave (CMS) method.36 As the CMS method does not need a clock, the maximum sampling speed of the digitizer can be used.37 No hardware-based dispersion compensation is required due to CMS being inherently tolerant to dispersion mismatches in the interferometer;38 CMS method replaces the Fourier transform conventional processing with comparison operations of the channeled spectrum from the sample against masks. These masks are channeled spectra delivered by the same interferometer for different optical path differences in the OCT interferometer using a mirror instead of the sample. As the channeled spectra involved in the comparison operation are produced by the same interferometer, such a procedure is tolerant to chirped channeled spectra, irrespective of whether such chirp is due to nonlinear sweeping or dispersion in the interferometer.39

The B-scan and C-scan images produced via CMS processing were displayed as 8-bit integers with full dynamic range conversion in the LabVIEW IMAQ display control.

2.3.

Fiber Bundle Fluorescence Endomicroscopy Imaging Subsystem

The FEM uses a flexible fiber bundle (Fujikura FIGH-30-650S) comprising 30,000 cores to deliver and collect light from a sample, as shown diagrammatically in Fig. 1(f). As no distal optics were used, the fiber bundle, therefore, needed to be in direct contact with the sample. A Texas Instruments Lightcrafter 3000 digital micromirror device (DMD) was used as an illumination source to project uniform blue light onto the sample. The blue light is produced by a light-emitting diode (LED) incorporated within the DMD (with central wavelength of 450 nm and optical power of 32 mW at 450 nm). The output beam from the DMD traverses a low-pass filter (with a 450 nm cut-off), a dichroic mirror, and a 10X microscope objective to deliver 4.3 mW incident on the proximal end of the fiber bundle. While the DMD was destined to generate structured light patterns, for this study it was simply used to deliver flood illumination. The magnification between the DMD pixels and the fiber bundle was sufficient so that the illumination was effectively uniform, as though the bundle is directly illuminated by the LED. The returning fluorescence is imaged onto a CMOS camera (PointGrey Flea 3) via the dichroic and an emission filter (cut-off >500  nm). The raw image is cropped to a circular diameter of 525  μm, and Gaussian-filtered (σ=1.2  μm) to remove pixelation due to the fiber bundle cores. Images were acquired at 20 fps.

2.4.

System Integration and Real-Time Display

As mentioned earlier, the three subsystems are controlled by a single workstation PC. The acquired data from the OCT and FEM subsystems are processed by this PC, with the option of displaying both OCT and FEM en-face frames in real-time during acquisition through a custom-made LabVIEW™ virtual instrument (VI) interface. A screenshot of the graphical user interface of this VI is presented in Fig. 3, with a video demonstration of the real-time operation in Video 1.

Fig. 3

Screenshot of the real-time multimodal display LabVIEW™ VI (screencast of the VI in operation in Video 1, MP4, 355 KB) [URL: https://doi.org/10.1117/1.JBO.24.6.066006.1].). (i) OCT B-scan display (reduced to a single line due to the fast scan mode being engaged), (ii) OCT en-face display, (iii) selection of the depth position of the en-face OCT image, (iv) FEM single-frame display, (v) FEM mosaic composition, (vi) fast scan mode toggle, (vii) lateral sine scan correction for OCT 1-D scanner toggle, (viii) scan velocity setting for the robotic scanning device, (ix) number of contiguous masks to be averaged for the OCT en-face display, and (x) toggle switch for interpolation between adjacent lines in the OCT en-face display.

JBO_24_6_066006_f003.png

A number of parameters and settings are indicated in Fig. 3. The OCT channel output can be displayed as a live B-scan image by reconstructing each depth using the CMS algorithm,36 which is equivalent to the conventional approach of performing a fast-Fourier transform (FFT) plus any additional dispersion compensation/resampling of the raw B-scan data. This is computationally expensive and unnecessary if an en-face OCT image is preferred (which would only require a few depths out of the whole depth range). To mitigate this issue, the “fast scan” mode in Fig. 3(vi) makes use of the unique property of the CMS method whereby it can reconstruct a subset of the depths with a reduced processing time. The position of this depth interval can be specified by the depth position control, as shown in Fig. 3(iii), and its range can be adjusted by the average control, as shown in Fig. 3(ix). In this way, rather than generating an entire B-scan for each position of the robotic scanner, which would be the conventional approach using FFT-based reconstruction, only a small subset of the B-scan is generated at the required depth. When the “average” parameter is set to 1, this is simply a 1-D array (a lateral T-scan). When the average value is >1 (i.e., the reconstructed interval has an axial range >1  pixel), the T-scan is created by averaging over this number of depth points. The full OCT en-face image in Fig. 3(ii) at this depth is then constructed from these T-scans. In parallel, the entire dataset can be recorded, allowing subsequent offline reconstruction of volumes, B-scans, and en-face images at any depth.

The 1-D OCT scanner is driven by a sinusoidal waveform; therefore, each raw T-scan is sampled nonlinearly along the lateral scanning function. This was corrected by a resampling algorithm, which can be toggled by control, as represented in Fig. 3(vii).

To improve the small FoV allowed by the fiber bundle [0.275  mm2, as shown in Fig. 3(iv)], a mosaicking algorithm similar to previously reported approaches32,40 was employed to stitch frames acquired during a single scan of the robotic device [as shown in Fig. 3(v)]. For real-time display, frames were registered pairwise using normalized cross correlation (NCC) (i.e., the shift of each frame was calculated relative to the previous frame). The mosaic was then formed by placing the frames dead-leaf (i.e., without blending) in their estimated shifted positions. For off-line reconstruction of saved datasets, the two-way registration described in Ref. 32 was employed and the frames were stitched using alpha blending to remove any visible seams between the images.

The robotic scanning device can potentially be run at high speeds, and the scan velocity can be specified, as shown in Fig. 3(ix). However, since both the OCT and the FEM channels have a limited acquisition rate, a compromise must be made between imaging speed and scan density. At a higher imaging speed (assuming the same scan range), the limited acquisition rate of the channels introduces undersampling in the OCT image along the scan direction (for the FEM channel, it is also necessary to ensure sufficient overlap between image frames, although in practice the OCT sampling requirement is the dominant factor). For real-time display, there was no buffering between the OCT acquisition system and the en-face imaging display system, which runs asynchronously. The sampling along the robotic scan direction was, therefore, not always linear, and so linear interpolation was used between T-scans to generate a uniformly sampled en-face image [activated by the checkbox in Fig. 3(x)].

3.

System Validation

3.1.

System Resolution and Imaging Range

The main parameters of the two imaging subsystems are shown in Table 1. The axial resolution of the OCT subsystem was measured to be 24  μm in air. By performing an analysis on the raw spectral shape of the spectra acquired, similar to what is presented in Appendix C of Rivet et al.,36 we have estimated the theoretical axial resolution to be about 16  μm. The lower axial resolution obtained experimentally can be attributed to the spectral window which has been applied during the signal processing stage. This step was necessary to increase the signal-to-noise ratio (SNR) of the system, thus improving image quality.

Table 1

Table summarizing the main properties of the two imaging subsystems.

Lateral resolution (μm)Axial resolution (μm)FoV (mm2)Optical power on the sample (mW)Imaging speed
OCT subsystem6024 (with Hamming window)2.7×1.6 (over a 1 mm working distance)2.8, at 1310 nm100 Hz per B-scan (500 A-scans)
FEM subsystem7.8n/a0.525×3.25 (with mosaicking)4.3, at 450 nm (at the proximal end of the bundle)20 fps

The OCT subsystem sensitivity was measured to be 84 dB. The sensitivity measurement procedure is a variation of that described by Bradu et al.41 and Leitgeb et al.42 Briefly, the endoscopic OCT probe was set so as to maximize the recoupling of the optical power returned from a mirror into the probe. The power Phigh was then measured at one of the detection ports of the optical interferometer (into the BPD). Then, the mirror was replaced by a low-reflectance target (a block of brushed aluminum), and the optical power measurement was repeated, obtaining Plow. With this target in place, an A-scan was obtained, and the peak value p and the noise floor value n were extracted from it, leading to a peak-to-noise ratio value of 50 dB. From the measurements taken, the sensitivity S was calculated as

Eq. (1)

S=20logpn+10logPhighPlow.

The lateral resolution of the OCT subsystem was measured using a United States Air Force (USAF) resolution target (Thorlabs R1DS1N). At the lateral range used for this study (1.6×2.7  mm), it was possible to resolve element 4.1, which is highlighted in Fig. 4(a). This puts the lateral resolution of the OCT subsystem at roughly 60  μm, which is expected due to the probe’s limited NA.

Fig. 4

(a) OCT en-face representation of a USAF target, highlighting element 4.1; (b) single FEM frame from the bundle representing the same USAF target, but covering groups 6 and 7, obtained in transmission, using a fluorescent material placed behind the USAF target; inset: close-up highlighting element 7.1, (c) OCT en-face image of a Ronchi ruling pattern (1  lp/mm), (d) OCT en-face image using the same Ronchi ruling pattern, with the pattern rotated by 90 deg, (e) composite image of (c) and (d), with (c) represented in red and two versions of (d) in blue and gold. These two versions are rotated by 90 deg and stretched in the original x-direction to match the span in the y-direction of (c). They are then shifted horizontally to match the bars of the Ronchi pattern. Images (c) and (d) differ in contrast because they correspond to slightly different depths in the imaging resolution target.

JBO_24_6_066006_f004.png

For system characterization, the robotic scanning device was mounted on an adjustable arm and fixed in place over a table. Test samples were placed horizontally on the table and, except where described otherwise, fixed in place. For all the experiments reported here, the imaging size was kept constant. With the 1-D OCT probe being driven by a 5V amplitude sine wave at 100 Hz (500 A-scans per B-scan acquired) and the robotic scanning device having a velocity setting of 0.76  mm/s for 400 buffered B-scans, the en-face image dimension was measured to be 1.6×2.7  mm2 (x×y).

The FEM resolution is limited by the sampling density of the fiber bundle cores. For the fiber bundle used, the core spacing is 3  μm, leading to a Nyquist limited resolution of approximately 6  μm (although this is shift and rotationally variant). In the image of the USAF target shown in Fig. 4(c), both orientations of element 7.1 can be clearly resolved, conservatively indicating a resolution of at least 7.8  μm. The diameter of each image is 525  μm and the nominal length of the mosaic, for a scan distance of 2.7  mm, is 3.25  mm.

Considering the separate nature of the two scanning directions of the OCT subsystem (one provided by the 1-D OCT probe and the other by the robotic scanner), it is important to understand whether there are significant differences in terms of resolution or distortion. To this end, a resolution target card (Edmund Optics Pocket USAF Optical Test Pattern) featuring a Ronchi Ruling Pattern with 1  line/mm was imaged and an en-face representation was constructed at a depth just beneath the laminated layer. The card was manually oriented in such a way that the patterned lines were perpendicular to the scan direction of the 1-D OCT probe, as shown in Fig. 4(c). The card was then rotated by 90 deg and the same scanning protocol was carried out, yielding the en-face image shown in Fig. 4(d).

To examine whether the spacing between lines was maintained, Fig. 4(e) was constructed from Figs. 4(c) and 4(d). The lines in the images were thresholded and colored; Fig. 4(c) is shown in red in its original orientation. Figure 4(d) was rotated by 90 deg and stretched to fit the vertical dimension of Fig. 4(c). Two versions were produced by translating the figure horizontally in order to match the lines with those of Fig. 4(c), shown in blue and gold.

While the spacing between lines does not seem to change appreciably between the two scanning directions, the behavior around the edges of the image in Fig. 4(d) is not linear; moreover, the bars themselves are slightly curved, as suggested by Fig. 4(d). This may be due to imperfect corrections in the sinusoidal re-sampling function along the lateral scanning direction, as mentioned in Sec. 2.4 (with some deviations at the edges of the image frame), and also due to the operating principle of the 1-D OCT probe,35 which will necessarily introduce off-axis behavior and therefore will introduce a degree of curvature into the imaging plane.

In addition, some artifacts are present in the OCT en-face images [most evidently in Figs. 4(a) and 4(d)], in the form of lines and streaks along the robotic scan direction (the y-axis). These are caused by back-reflections in the OCT endoscope introduced by small imperfections (e.g., scratches) on the GRIN optics.

For the study presented in Figs. 4(c)4(e), and to obtain any of the subsequent OCT en-face images, it is essential to ensure that the two independent scanning directions are approximately orthogonal to each other, as otherwise shear would affect the OCT images and the total imaged area would be reduced. While approximate alignment can be achieved by manually rotating the probe prior to fixing it in place, fine adjustment was achieved via an iterative procedure using a suitable resolution target. Since the robotic scanning device is able to scan in two dimensions (albeit not as fast as the 1-D OCT probe), the scanning protocol allows for a linear scan along any arbitrary angle θ. By changing this angle and observing the corresponding en-face OCT image, it is possible to calibrate the robotic scanning direction, as shown in Fig. 5. In practice, for a clinical device, the OCT probe would need to be modified so that it could only be inserted in the correct orientation.

Fig. 5

Montage of OCT en-face frames (30-frame average over a depth of 150  μm) acquired for several angle settings, θ, on the robotic scanning device. The imaged sample is a paper USAF resolution target (elements 2.4 to 2.6). (a) θ=2.7  rad, empirically aligned with the OCT scanning probe direction, (b) θ=2.4  rad, (c) montage of four θ settings after the USAF pattern was thresholded out—red corresponds to θ=2.7  rad, green to θ=2.6  rad, blue to θ=2.5  rad, and gold to θ=2.4  rad, (d) same as (a) but with π added to the initial angle setting (totaling 5.86 rad), (e) montage with thresholded version of (a) (in red) and thresholded version of (d) flipped along the horizontal direction (blue). Red arrows in (a), (b), and (d) depict the direction of scanning of the robotic scanning probe.

JBO_24_6_066006_f005.png

In Figs. 5(a) and 5(b), two OCT en-face images are shown, acquired with different θ settings on the robotic scanning device (2.7 and 2.4 radians, respectively). For this particular case, it was found that the optimum setting for θ is 2.7 radians; it is clear from Fig. 5(b) that the USAF pattern is deformed, presenting some shearing.

Figure 5(c) presents this effect more clearly, with four OCT en-face images superposed together (after the USAF patterns were thresholded out), with their corresponding initial angle θ setting ranging from 2.7 rad (red) to 2.4 rad (gold). It becomes clear that the larger elements present a larger shift (up to 280  μm) than the smaller elements in the bottom of the image, indicating a shear-based distortion.

The robotic scanning instrument also presents good repeatability in terms of the scanning direction; in Fig. 5(d), an OCT en-face image for a setting of 2.7+π radians is shown. Since the initial angle is varied by π, the B-scans making up the OCT volume are now buffered in reverse order, which explains why the image is flipped along the horizontal axis in relation to that in Fig. 5(a). To show that the angle adjustment does not appreciably affect the image, Fig. 5(e) shows a thresholded version of Fig. 5(a) (in red) overlaid on a thresholded and flipped version of Fig. 5(d) (in blue). It is clear that, apart from a slight vertical shift, amounting to 40  μm on average (which could be attributed to triggering and timing issues in the acquisition itself), the two images are well matched, which shows that the scanner is invariant to the direction of travel.

3.2.

Repeatability Studies

To assess the scanning repeatability of the system, a procedure was devised involving an imaging phantom, which is schematically represented in Fig. 6(a). This phantom comprises a piece of fluorescent-stained lens tissue paper [Fig. 6(ai)], placed over an imaging resolution card [Fig. 6(aii)] (Edmund Optics Pocket USAF Optical Test Pattern). Since the probe [Fig. 6(aiii)] (and, therefore, the fiber bundle used in the FEM system) is in contact with the lens tissue paper [Fig. 6(ai)], the FEM images should only show the lens tissue paper, whereas the OCT images, having a longer imaging depth range, should show both layers [Figs. 6(ai) and 6(aii)].

Fig. 6

Comparison between en-face OCT and FEM frames acquired from an imaging phantom is schematically represented in (a). This imaging phantom comprises a top layer (i) made of lens tissue paper stained with a fluorescent marker and a bottom layer (ii) incorporating the Edmund Optics Pocket USAF Optical Test Pattern card, which has been immobilized in the sample holder. The probe (iii) is in contact with layer (i) when FEM imaging occurs. Images (b1)-(d2) present comparisons between two datasets acquired at the same position (the first and the last of a set of five volumes acquired consecutively), where points belonging to only one of the datasets are represented in either red or green, and those that overlap are represented in yellow. In (b1)–(b3), layer (i) is not present and the probe (iii) is scanned over the sample without contact, hence only OCT images are generated. Images (b1)–(b2) are B-scans taken at opposite ends of the volume, as represented in dotted lines in (b3), covering the printed pattern of the USAF resolution target, showing element 2.1 on the bottom right corner, and elements 3.3 to 3.6 on the top right corner. In (c1)–(c2), FEM mosaics of layer (i) are represented; (c3) shows the comparison between these two images. Image (d1) shows a comparison between OCT en-face frames at the depth of layer (i), and (d2) shows a similar comparison, but at the depth of the printed pattern of the USAF card.

JBO_24_6_066006_f006.png

Two different datasets were acquired, each having five volumes (five runs of the robotic scanning probe). In the first set, layer (i) was removed and the probe was allowed to run in a contactless manner (therefore, no FEM images were produced, only en-face OCT images). In the second set, both layers were present, and the probe was in contact with layer (i) throughout the whole span of the robotic scanning probe movement, and both FEM and en-face OCT images were then obtained.

To visually compare the different volumes acquired within each dataset, the colocalization plug-in from ImageJ Colocalization Colormap43 was used. This plug-in colors each of the two volumes with a separate color (either red or green), with any overlapped points shown in yellow. The results from the first dataset (comparing the first with the last volume) are shown in Figs. 6(b1)6(b2) (OCT B-scans) and Fig. 6(b3) (en-face OCT), and the results from the second dataset (also comparing the first with the last volume) are shown in Figs. 6(c1)6(c3) (FEM) and Figs. 6(d1)6(d2) (en-face OCT).

In Fig. 6(b3), the composite en-face OCT image appears to have a gradient of color, ranging from red at the top to green at the bottom. We believe that this is caused by an axial shift, rather than lateral shift, as shown by the two B-scans in Figs. 6(b1) and 6(b2). Across the five volumes, this axial shift ranges from 50 to 100  μm, and it seems to be more pronounced in the case when the probe is run without contact, i.e., when layer (i) is removed.

To quantify the lateral pixel shifts, the NCC was computed across the en-face OCT images [Figs. 6(b3) and 6(d1)6(d2)] extracted from the five volumes for each of the two datasets. In addition, in the dataset where the two layers are present, the NCC is computed across the FEM data [Figs. 6(c1)6(c2)].

Figure 7 depicts a 2-D map of NCC values calculated for each image in the set considered as reference with the other four template images in the set. Obviously, comparing each frame to itself yields a NCC value of 1, as shown by the values along the diagonals in all plots. It is also worth noting that the NCC values in all plots are symmetric in relation to the diagonals since all five images are considered either as references or templates.

Fig. 7

Intensity plots of the NCC values computed for the en-face OCT and FEM images presented in Fig. 6 (values along the diagonal are all equal to 1). (a) Comparison between the five en-face OCT frames acquired without layer (i), with representative frames shown in Fig. 6(b3). (b) Comparison between the five FEM mosaic images acquired [only layer (i) covered], with representative frames shown in Figs. 6(c1)–(c3). (c) Comparison between the five en-face OCT frames whose depth is set to cover layer (i), as shown in Fig. 6(d1). (d) Comparison between the five en-face OCT frames whose depth is set to cover layer (ii), as shown in Fig. 6(d2).

JBO_24_6_066006_f007.png

It was found that, for the OCT en-face frames, the average lateral pixel shift is less than the lateral resolution (60  μm). It was also found that the average of the NCC values (without including the values on the diagonal) is lower (0.76) for the layer depicted in Fig. 6(d1) than the average for the layers of Figs. 6(b3) and 6(d2) (0.83 and 0.88, respectively), possibly due to the probe dragging and deforming layer (i) during each scan. The higher average NCC value in Fig. 6(d2) may be due to the relative stability conferred by the probe contact in terms of axial movements. Moreover, the NCC analysis for the FEM data [Fig. 7(b)] yielded the highest NCC values (0.93), with average lateral shifts of around 10 to 11  μm. This seems to be consistent with the standard deviation of the measured mosaic lengths (13  μm).

3.3.

Coregistration between Channels

As shown in Fig. 8(a), the two fiber probes are laterally separated by 1.6  mm inside the robotic scanning device tube. Therefore, at any given moment, the OCT and the FEM subsystems image different regions of the sample. However, given the considerable size of the lateral scanning, especially in the y-direction (2.7  mm), it is possible to identify a region of the sample covered by both imaging subsystems. This overlapping region depends on the initial setup and calibration of the scanning procedure, particularly the rotation of the OCT probe and hence the scan angle required to generate an orthogonal scan with the robotic scanner. When placing the OCT and FEM probes inside the robotic scanner, we optimized the relative position of the probes so that the separation was roughly along the diagonal of the OCT image, in order to maximize the overlap between the two channels.

Fig. 8

(a) Photograph of the tube showing the two OCT and FEM probes and the distance between their centers. Images (b) to (e) depict the coregistration study for two separate FEM/OCT acquisitions [(b)/(c) and (d)/(e) respectively]. White arrow in the FEM images (b) and (d) depicts scanning direction, and the red shaded area in all frames corresponds to the area that has been covered by the two imaging subsystems. All frames share the same pixel size.

JBO_24_6_066006_f008.png

Figures 8(b)8(e) depict a study carried out with the Edmund Optics USAF test pattern. In this case, the surface of the card was stained with a fluorescent marker and the probe is allowed to run in near contact to the surface of the card. In this way, images of the USAF patterns are obtained in both channels.

The images were scaled so that the pixel sizes were the same, and then the FEM image was translated so that it would match the pattern observed on the OCT en-face image. The overlapped region is shaded in red in all images in Fig. 8. Approximately 7% to 8% of the OCT images were also included in the FEM mosaic. The same probe configuration was used for all other datasets presented in this paper, and so the overlap region remains the same.

There is one important difference between the two acquired datasets shown in Figs. 8(b)/8(c) and 8(d)/8(e). In the latter, it seems that the mosaicked FEM image (d) has a shorter vertical span than the former (b). In fact, when the FEM image is translated and placed above the OCT en-face image (e), the top end of the shaded region does not agree with the features shown in (e), namely element 3.6. This is due to the mosaicking algorithm requiring features in the image to detect the movement. Since in (b) the probe was slightly more shifted to the right, it was able to cover features throughout the whole scanning range. In (d), on the other hand, there is a sizeable gap with no features between elements 3.6 and 2.1, which impacted the mosaicking algorithm by not registering motion where there should have been some. In principle, this problem could be mitigated by using the known velocity of the robotic scanner when there are insufficient image details for registration, although this procedure would not then deal with any tissue deformation.

4.

Imaging Results

4.1.

Animal Tissue Experiments

To test the system in a biomedical imaging environment, ex-vivo animal tissue imaging was performed. Ovine kidney and porcine lung/esophagus were stained using the topical staining agent acriflavine hydrochloride (0.01% in water) for 60 s. The samples were then rinsed with water, as described in Ref. 16. During these imaging trials, the robotic scanning was mounted on an arm attached to the table, and the probe was in contact with the sample, yielding OCT volumes and FEM mosaics, as shown in Fig. 9.

Fig. 9

(a1) A 3-D render of a full OCT volume (2.7×1.6×3.4  mm3) of ovine kidney tissue. (a2) OCT color-coded depth projection of the same section of kidney tissue. (a3) FEM mosaic of the same section of kidney tissue as shown in images (a1) and (a2). (a4) FEM mosaic of a different region of kidney tissue, with corresponding OCT en-face image (three-frame average, covering 30  μm at a depth of 270  μm below the surface of the tissue) in image (a5). (b1) A 3-D render of a full OCT volume (2.7×1.6×3.4  mm3) of a section of porcine lung tissue. (b2) OCT B-scan along the y-direction (performed by the robotic scanner), showing individual alveoli (yellow arrows). (b3) OCT en-face visualization of the same section of lung tissue (depth average covering 340  μm in the axial direction), showing individual alveoli (yellow arrows). (b4) FEM mosaic of the same region (shortened due to probe not being in complete contact with the tissue). (c1) A 3-D render of a full OCT volume (2.7×1.6×4.3  mm3) of a section of porcine esophageal tissue. (c2) OCT B-scan along the y-direction (performed by the robotic scanner). (c3) OCT en-face visualization taken from a region 1  mm below the surface (depth average covering roughly 1 mm in the axial direction). (c4) FEM mosaic of the same region, depicting the structures present in the most superficial layer.

JBO_24_6_066006_f009.png

Figures 9(a1)9(a5) all correspond to the same section of kidney tissue, whereas Figs. 9(b1)9(b4) and 9(c1)9(c4) correspond to the same section of porcine lung and esophageal tissues, respectively.

In Fig. 9(a1), a 3-D render of the full OCT volume is shown, with a good depiction of the tissue texture being present. Figure 9(a2) employs the same data but in a color-coded depth projection.

As discussed in Sec. 3.3, there is a region that is common to both the OCT images and the FEM mosaic. A correspondence in features can be observed in the boxed region in both Figs. 9(a2) and 9(a3), with a crevice in the surface of the tissue being visible on both images.

If the probe is prevented from running smoothly across the surface of the sample being imaged (due to snagging or extreme tissue deformation), as shown in some of the repeatability studies in Sec. 3.2, then artifacts will arise in both the OCT en-face image and the FEM mosaic. This can be observed in Figs. 9(a3)9(a4), showing two image datasets collected from the same area of tissue; Figure 9(a4) is evidently much shorter than Fig. 9(a3), despite the fact that the probe was run across the exact same distance and at the same speed. This is because the probe has dragged the tissue during scanning along the robotic device scanning direction. When the tissue moves with the probe, the length of FEM mosaics, which are generated purely based on image registration, is reduced. Artifacts are also observed in the OCT en-face image in Fig. 9(a5) but of a different form. Because the OCT image is assembled based on the expected scanning motion, tissue deformation is exhibited as elongation of tissue features along the y-scanning direction, as shown in the regions delimited by the yellow dashed boxes.

Figures 9(b1)9(b4) all correspond to the same section of porcine lung tissue acquired with both OCT and FEM subsystems. It is possible to identify individual alveoli (yellow arrows) in this particular piece of lung tissue on both B-scan along the robotic scan direction (b2) and en-face (b3) visualizations, with the individual widths ranging between 100 and 130  μm, which is consistent with the values presented in the literature.44 Similar to (a4), the FEM mosaic does not have the full length consistent with the actual probe scan, but unlike the previous case, the probe does not appear to be caught in the tissue, dragging it with it; instead, it appears that, due to the topography of the tissue, the fiber bundle of the FEM subsystem lost contact with the tissue in parts of the scan, consequently yielding a shorter mosaic.

Finally, in Figs. 9(c1)9(c4), a section of porcine esophageal tissue has been analyzed, with the OCT volume allowing for clear distinction between the different layers in both the 3-D volume render (c1) and the B-scan along the robotic scan direction (c2). Unlike the previous two cases, the FEM mosaic (c4) corresponding to this tissue section is complete [with no scanning artifacts present in the en-face visualization in (c3)], although some of the structures appear to have moved with the probe during scanning, which when combined with the mosaic reconstruction yield some artifacts in the image, such as the spiral structure present in the middle of the FEM mosaic (arrowed).

4.2.

Handheld Operation

The robotic scanning device has a relatively small footprint, making handheld operation possible. In Fig. 10(a), a photograph depicts the robotic scanning device in handheld operation, imaging the skin of a volunteer’s index finger. However, in practice we were only able to generate good quality handheld scans when the device was not in direct contact with the tissue. Handheld operation is, therefore, shown only for OCT imaging modality, where the probe could be held a small distance above the tissue.

Fig. 10

(a) Photograph of the robotic scanning device in handheld operation. (b) A 3-D render of a full OCT volume (2.7×1.6×2.9  mm3) of a volunteer’s thumb. (c) Cropped OCT B-scan (1.48×2.9  mm2) taken from the same volume shown in (b), with the sweat glands highlighted by yellow arrows (i). (d) En-face OCT visualization (three-frame average spanning 30  μm at a depth of 700  μm below the surface) of the same volume, highlighting the interface between stratum corneum and stratum germinativum (ii) but also showing the skin/air interface in (i). Video 2, MP4, 140 KB) [URL: https://doi.org/10.1117/1.JBO.24.6.066006.2]. shows a screencast of the acquisition UI in real time, handheld operation, with the OCT en-face frames refreshing at 1 Hz (sped up by a factor of 2). Owing to the limited en-face imaging rate and the fact that a single en-face image (albeit averaged over 200  μm) is being displayed, there are some unavoidable motion artifacts both axially and laterally.

JBO_24_6_066006_f010.png

Owing to the limited speed and the increased possibility of motion artifacts affecting the images, the 1-D OCT scanning frequency was increased to 250 Hz and the number of lateral points in the OCT volumes was reduced to 200×200 (x×y). A full volume was then acquired in 0.8 s.

Despite some motion artifacts, it is possible to resolve some fine features from in-vivo skin samples, as shown in Figs. 10(b) and 10(c), where an OCT volume was obtained from a volunteer’s thumb. It is possible to distinguish the stratum corneum from the stratum germinativum, and as shown in Fig. 10(c) [which is a B-scan taken from the full OCT volume in (b)], it is possible to visualize sweat glands [arrowed, labeled (i)]. In an en-face visualization of the same region [Fig. 10(d)], it is possible to recognize the dendrite-like structure of the stratum germinativum [arrowed, labeled (ii)].

As mentioned earlier in Sec. 3.1, the OCT images present a slight curvature along the fast (x) scanning axis, due to the construction and operating principle of the 1-D OCT probe and the fact that we had to employ a sine wave function to perform the scanning. This can be seen in the 3-D render shown in Fig. 10(b), with the curvature being more evident along the x-axis than along the y-axis, for a lateral scanning range of the same order of magnitude in each direction.

5.

Discussion

In this paper, we have presented a combined OCT/FEM system that is capable of bi-dimensional lateral scans by means of a robotic scanning device. The 1-D, FV OCT scanning probe allows a lateral FoV of 1.6  mm with a resolution of 60  μm; the OCT subsystem is capable of resolving 24  μm in depth (in air). The fiber bundle-based FEM subsystem has a lateral resolution of better than 8  μm, limited by the core spacing of the imaging bundle. The scanning device is capable of scanning 2.7  mm, generating images in as short as 0.8 s.

Scanning-wise, the lateral deviations between successive scans (average pixel shifts) are lower than the lateral resolution of the OCT subsystem, quantified to about 10 to 11  μm in the FEM subsystem, which is in line with the deviation measured across mosaic length, 13  μm. The most significant deviations seem to happen in the axial direction (OCT only), particularly when the probe is not in contact with the sample (and therefore no FEM imaging can take place). Other reports19,20 have also reported this dependency of repeatability on the sample and imaging procedure (contact versus noncontact), therefore this is not specific to our implementation.

Our implementation does have some shortcomings, which are summarized below.

  • The robotic scanning device is rigid, which limits the scope of applications.

  • Owing to the separate nature of the two channels and probes, there is only a very small overlap between the areas covered by the two subsystems, as shown in Sec. 3.3.

  • While still able to deliver images of biological tissue (as shown in Fig. 9 in Sec. 4), the lateral resolution of the OCT subsystem is relatively poor (especially in comparison with its axial resolution). Lateral resolution was also affected by imperfections on the GRIN optics, whose effects could not be easily compensated for.

  • Comparing the system sensitivity with that of the other equivalent systems we assembled in the past with different interface optics, we can assume that the OCT FV probe employed in this study contributed to losses in the system that led to lower sensitivity.

  • Owing to the fact that we are employing a fiber bundle for imaging with no distal imaging optics, when imaging with the FEM subsystem, the probe must remain in contact with the tissue/sample being investigated, with the undesirable effect of some tissue dragging.

Because it was not possible to decouple the effect of each scanning subsystem on the overall instrument performance, the repeatability study could only be conducted on the combined effect of the robotic scanner and that of the raster scanner in the OCT scanning probe. Given that the OCT scanning probe runs in open loop, the system repeatability is sensitive to external factors.

Some of these shortcomings could be overcome by improved mechanical design of the robotic scanning device or by merging the two subsystems into a single optical probe, by means of a DCF9 inside a 1-D, FV scanning probe, such as the one employed in the OCT subsystem. However, such an approach would introduce a penalty in the imaging resolution for the FEM subsystem, as a cantilevered optical fiber introduces off-axis scanning aberrations, and the low numerical aperture of the miniature lens in the probe prevents matching of the imaging resolution to that of a fiber bundle.

Handheld operation was not possible using both OCT and FEM channels, as this requires contact between the probe tip and the tissue. It is extremely difficult to maintain sufficient pressure to obtain an image while avoiding pressing too hard and generating severe tissue deformations. One possibility to address this, which has been explored to some extent by Zhang et al.,20 would be to use an FEM probe with a finite working distance and employ the OCT system to estimate the distance of the probe from the tissue, and actuate the fiber bundle axially using a high-speed motor to maintain the correct working distance. Alternatively, a force sensor34 could be used to ensure optimal contact between the probe and the sample. Handheld operation was shown to be possible for the OCT imaging channel alone, including generating real-time en-face slices using a 1-D scanning probe, as shown in Sec. 4.2. In this mode of operation it would still be possible to then bring the probe into contact with the tissue to obtain a single FEM image or a small manually formed FEM mosaic.

6.

Conclusions and Future Outlook

In this report, we have presented a combined OCT/FEM system that is capable of bidimensional lateral scans by means of a robotic scanning device. The whole probe is housed in a compact, lightweight package with a minimal footprint, with a scanning end of 3.7 mm OD. This makes it suitable for in-situ tissue investigations, presenting some advantages over conventional robotic approaches in terms of footprint, cost, and surgical workflow.33 The device can be operated handheld or supported by an articulated arm. It is possible to obtain higher speed (1  fps), direct en-face OCT images in conjunction with the FEM mosaics by sacrificing some lateral resolution in the OCT channel along the robotic scanning direction. While the device only offers two degrees of freedom, it is the simplicity of the mechanism that enables a level of reproducibility suitable for assembling both FEM mosaics and OCT volumes. More complex and expensive devices, such as robotic arms, provide more flexibility and have been used for FEM mosaicking (e.g., Ref. 34), but the assembly of OCT volumes from B-scans is more sensitive to small errors than FEM mosaicking, where the errors are automatically corrected by the mosaicking registration algorithm.

As a rigid instrument, the scanner is unsuitable for general endoscopic applications. However, there are a number of suitable interventional procedures, such as in breast surgery and neurosurgery, where a direct line of sight is available, and there is need to determine tissue characteristics—and in particular to identify tumor margins—in real time. Probe based imaging has been proposed for these applications (e.g., Refs. 16 and 45), but without a scanning mechanism the field of view (FoV) is inevitably small compared to the region of interest.21 The type of device presented here could, therefore, provide a good compromise between ease of deployment and imaged area. In particular, even though the scanner is rigid, both of the probes are flexible along the remainder of their length, allowing the bulk of the optical systems to be sited a convenient distance away from the patient. A modified device with a longer scanning tube could also be considered for more general laparoscopic surgical applications, although further work would be required to confirm that the scanning performance is not degraded.

Despite the limitations described above, the results presented here demonstrate the feasibility of using a compact robotic scanner to provide one direction of scanning for en-face and volumetric OCT imaging and of combining high-resolution FEM and OCT imaging in a minimally invasive probe. These promising results will support further development of laparoscopic and endoscopic multimodal, en-face imaging systems.

Disclosures

The authors declare that there are no conflicts of interest related to this article.

Acknowledgments

The authors wish to acknowledge the following funding sources: Engineering and Physical Sciences Research Council (EPSRC) (“REBOT” EP/N019229/1, “Translational Alliance: SMART-Endomicroscopy” EP/R019274/1, and “Ultrathin fluorescence microscope in a needle” EP/N022521/1), H2020 European Research Council (ERC) (“ADASMART,” 754695), National Institute for Health Research Biomedical Research Centre at Moorfields Eye Hospital NHS Foundation Trust (NIHR), the UCL Institute of Ophthalmology, University College London, and the Royal Society Wolfson research merit award.

References

1. 

K. Fuchs, “Minimally invasive surgery,” Endoscopy, 34 (02), 154 –159 (2002). https://doi.org/10.1055/s-2002-19857 ENDCAM Google Scholar

2. 

S. Zhao et al., “Intraoperative fluorescence-guided resection of high-grade malignant gliomas using 5-aminolevulinic acid-induced porphyrins: a systematic review and meta-analysis of prospective studies,” PLoS One, 8 (5), e63682 (2013). https://doi.org/10.1371/journal.pone.0063682 POLNCL 1932-6203 Google Scholar

3. 

K. Kong et al., “Raman spectroscopy for medical diagnostics—From in-vitro biofluid assays to in-vivo cancer detection,” Adv. Drug Deliv. Rev., 89 121 –134 (2015). https://doi.org/10.1016/j.addr.2015.03.009 ADDREP 0169-409X Google Scholar

4. 

Z. Zhi et al., “4D optical coherence tomography-based micro-angiography achieved by 1.6-MHz FDML swept source,” Opt. Lett., 40 (8), 1779 –1782 (2015). https://doi.org/10.1364/OL.40.001779 OPLEDP 0146-9592 Google Scholar

5. 

M. J. Gora et al., “Endoscopic optical coherence tomography: technologies and clinical applications,” Biomed. Opt. Express, 8 (5), 2405 –2444 (2017). https://doi.org/10.1364/BOE.8.002405 BOEICL 2156-7085 Google Scholar

6. 

T.-H. Tsai, J. G. Fujimoto and H. Mashimo, “Endoscopic optical coherence tomography for clinical gastroenterology,” Diagnostics, 4 (2), 57 –93 (2014). https://doi.org/10.3390/diagnostics4020057 Google Scholar

7. 

W. Drexler et al., “Optical coherence tomography today: speed, contrast, and multimodality,” J. Biomed. Opt., 19 (7), 071412 (2014). https://doi.org/10.1117/1.JBO.19.7.071412 JBOPFO 1083-3668 Google Scholar

8. 

A. R. Tumlinson et al., “Miniature endoscope for simultaneous optical coherence tomography and laser-induced fluorescence measurement,” Appl. Opt., 43 (1), 113 –121 (2004). https://doi.org/10.1364/AO.43.000113 APOPAI 0003-6935 Google Scholar

9. 

S. Y. Ryu et al., “Combined system of optical coherence tomography and fluorescence spectroscopy based on double-cladding fiber,” Opt. Lett., 33 (20), 2347 –2349 (2008). https://doi.org/10.1364/OL.33.002347 OPLEDP 0146-9592 Google Scholar

10. 

H. Pahlevaninezhad et al., “A high-efficiency fiber-based imaging system for co-registered autofluorescence and optical coherence tomography,” Biomed. Opt. Express, 5 (9), 2978 –2987 (2014). https://doi.org/10.1364/BOE.5.002978 BOEICL 2156-7085 Google Scholar

11. 

L. Scolaro et al., “Molecular imaging needles: dual-modality optical coherence tomography and fluorescence imaging of labeled antibodies deep in tissue,” Biomed. Opt. Express, 6 (5), 1767 –1781 (2015). https://doi.org/10.1364/BOE.6.001767 BOEICL 2156-7085 Google Scholar

12. 

K. Schmeler et al., “High-resolution microendoscopy (HRME): a low-cost, point-of-care alternative to colposcopy and biopsies?,” Gynecol. Oncol., 137 17 (2015). https://doi.org/10.1016/j.ygyno.2015.01.040 GYNOA3 Google Scholar

13. 

B. D. Grant et al., “High-resolution microendoscopy: a point-of-care diagnostic for cervical dysplasia in low-resource settings,” Eur. J. Cancer Prev., 26 (1), 63 –70 (2017). https://doi.org/10.1097/CEJ.0000000000000219 EJUPEK 0959-8278 Google Scholar

14. 

P. Sharma et al., “Real-time increased detection of neoplastic tissue in Barrett’s esophagus with probe-based confocal laser endomicroscopy: final results of an international multicenter, prospective, randomized, controlled trial,” Gastrointest. Endosc., 74 (3), 465 –472 (2011). https://doi.org/10.1016/j.gie.2011.04.004 Google Scholar

15. 

P. Su et al., “Efficacy of confocal laser endomicroscopy for discriminating colorectal neoplasms from non-neoplasms: a systematic review and meta-analysis,” Colorectal Dis., 15 (1), e1 –e12 (2013). https://doi.org/10.1111/codi.2012.15.issue-1 Google Scholar

16. 

T. P. Chang et al., “Imaging breast cancer morphology using probe-based confocal laser endomicroscopy: towards a real-time intraoperative imaging tool for cavity scanning,” Breast Cancer Res. Treat., 153 (2), 299 –310 (2015). https://doi.org/10.1007/s10549-015-3543-8 BCTRD6 Google Scholar

17. 

V. Pavlov et al., “Intraoperative probe-based confocal laser endomicroscopy in surgery and stereotactic biopsy of low-grade and high-grade gliomas: a feasibility study in humans,” Neurosurgery, 79 (4), 604 –612 (2016). https://doi.org/10.1227/NEU.0000000000001365 NEQUEB Google Scholar

18. 

A. Pierangelo et al., “Diagnostic accuracy of confocal laser endomicroscopy for the ex vivo characterization of peritoneal nodules during laparoscopic surgery,” Surg. Endosc., 31 (4), 1974 –1981 (2017). https://doi.org/10.1007/s00464-016-5172-7 Google Scholar

19. 

B. Rosa et al., “Building large mosaics of confocal edomicroscopic images using visual servoing,” IEEE Trans. Bio-Med. Eng., 60 (4), 1041 –1049 (2013). https://doi.org/10.1109/TBME.2012.2228859 Google Scholar

20. 

L. Zhang et al., “From macro to micro: autonomous multiscale image fusion for robotic surgery,” IEEE Rob. Autom. Mag., 24 63 –72 (2017). https://doi.org/10.1109/MRA.100 IRAMEB 1070-9932 Google Scholar

21. 

S. Zuo et al., “Toward intraoperative breast endomicroscopy with a novel surface-scanning device,” IEEE Trans. Bio-Med. Eng., 62 (12), 2941 –2952 (2015). https://doi.org/10.1109/TBME.2015.2455597 Google Scholar

22. 

S. Zuo, M. Hughes and G.-Z. Yang, “Novel balloon surface scanning device for intraoperative breast endomicroscopy,” Ann. Biomed. Eng., 44 (7), 2313 –2326 (2016). https://doi.org/10.1007/s10439-015-1493-2 ABMECF 0090-6964 Google Scholar

23. 

L. Huo et al., “Forward-viewing resonant fiber-optic scanning endoscope of appropriate scanning speed for 3D OCT imaging,” Opt. Express, 18 (14), 14375 –14384 (2010). https://doi.org/10.1364/OE.18.014375 OPEXFF 1094-4087 Google Scholar

24. 

S. Moon et al., “Semi-resonant operation of a fiber-cantilever piezotube scanner for stable optical coherence tomography endoscope imaging,” Opt. Express, 18 (20), 21183 –21197 (2010). https://doi.org/10.1364/OE.18.021183 OPEXFF 1094-4087 Google Scholar

25. 

N. Zhang et al., “Compact piezoelectric transducer fiber scanning probe for optical coherence tomography,” Opt. Lett., 39 (2), 186 –188 (2014). https://doi.org/10.1364/OL.39.000186 OPLEDP 0146-9592 Google Scholar

26. 

H. Schulz-Hildebrandt et al., “Novel endoscope with increased depth of field for imaging human nasal tissue by microscopic optical coherence tomography,” Biomed. Opt. Express, 9 (2), 636 –647 (2018). https://doi.org/10.1364/BOE.9.000636 BOEICL 2156-7085 Google Scholar

27. 

A. M. Sergeev et al., “In vivo endoscopic OCT imaging of precancer and cancer states of human mucosa,” Opt. Express, 1 (13), 432 –440 (1997). https://doi.org/10.1364/OE.1.000432 OPEXFF 1094-4087 Google Scholar

28. 

K. M. Joos and J.-H. Shen, “Miniature real-time intraoperative forward-imaging optical coherence tomography probe,” Biomed. Opt. Express, 4 (8), 1342 –1350 (2013). https://doi.org/10.1364/BOE.4.001342 BOEICL 2156-7085 Google Scholar

29. 

T. Xie et al., “Fiber-optic-bundle-based optical coherence tomography,” Opt. Lett., 30 (14), 1803 –1805 (2005). https://doi.org/10.1364/OL.30.001803 OPLEDP 0146-9592 Google Scholar

30. 

P. Giataganas et al., “Intraoperative robotic-assisted large-area high-speed microscopic imaging and intervention,” IEEE Trans. Bio-Med. Eng., 66 (1), 208 –216 (2018). https://doi.org/10.1109/TBME.2018.2837058 Google Scholar

31. 

R. Cernat et al., “Dual instrument for in vivo and ex vivo OCT imaging in an ENT department,” Biomed. Opt. Express, 3 (12), 3346 –3356 (2012). https://doi.org/10.1364/BOE.3.003346 BOEICL 2156-7085 Google Scholar

32. 

T. Vercauteren et al., “Real time autonomous video image registration for endomicroscopy: fighting the compromises,” Proc. SPIE, 6861 68610C (2008). https://doi.org/10.1117/12.763089 PSISDG 0277-786X Google Scholar

33. 

C. J. Payne and G.-Z. Yang, “Hand-held medical robots,” Ann. Biomed. Eng., 42 1594 –1605 (2014). https://doi.org/10.1007/s10439-014-1042-4 ABMECF 0090-6964 Google Scholar

34. 

P. Wisanuvej et al., “Three-dimensional robotic-assisted endomicroscopy with a force adaptive robotic arm,” in IEEE Int. Conf. Rob. and Autom. (ICRA), 2379 –2384 (2017). https://doi.org/10.1109/ICRA.2017.7989276 Google Scholar

35. 

F. I. Feldchtein, V. M. Gelikonov, G. V. Gelikonov, Handbook of Optical Coherence Tomography, 125 –142 Marcel Dekker, New York (2001). Google Scholar

36. 

S. Rivet et al., “Complex master slave interferometry,” Opt. Express, 24 (3), 2885 –2904 (2016). https://doi.org/10.1364/OE.24.002885 OPEXFF 1094-4087 Google Scholar

37. 

M. J. Marques et al., “Complex master-slave for long axial range swept-source optical coherence tomography,” OSA Continuum, 1 (4), 1251 –1259 (2018). https://doi.org/10.1364/OSAC.1.001251 Google Scholar

38. 

A. Bradu, M. Maria and A. G. Podoleanu, “Demonstration of tolerance to dispersion of master/slave interferometry,” Opt. Express, 23 (11), 14148 –14161 (2015). https://doi.org/10.1364/OE.23.014148 OPEXFF 1094-4087 Google Scholar

39. 

A. G. Podoleanu and A. Bradu, “Master–slave interferometry for parallel spectral domain interferometry sensing and versatile 3D optical coherence tomography,” Opt. Express, 21 (16), 19324 –19338 (2013). https://doi.org/10.1364/OE.21.019324 OPEXFF 1094-4087 Google Scholar

40. 

N. Bedard et al., “Real-time video mosaicing with a high-resolution microendoscope,” Biomed. Opt. Express, 3 (10), 2428 –2435 (2012). https://doi.org/10.1364/BOE.3.002428 BOEICL 2156-7085 Google Scholar

41. 

A. Bradu and A. G. Podoleanu, “Attenuation of mirror image and enhancement of the signal-to-noise ratio in a Talbot bands optical coherence tomography system,” J. Biomed. Opt., 16 (7), 076010 (2011). https://doi.org/10.1117/1.3598446 JBOPFO 1083-3668 Google Scholar

42. 

R. Leitgeb, C. Hitzenberger and A. F. Fercher, “Performance of Fourier domain vs. time domain optical coherence tomography,” Opt. Express, 11 (8), 889 –894 (2003). https://doi.org/10.1364/OE.11.000889 OPEXFF 1094-4087 Google Scholar

43. 

A. Gorlewicz, “Colocalization Colormap,” (2018) https://sites.google.com/site/colocalizationcolormap/ September ). 2018). Google Scholar

44. 

H. Lum and W. Mitzner, “A species comparison of alveolar size and surface forces,” J. Appl. Physiol., 62 (5), 1865 –1871 (1987). https://doi.org/10.1152/jappl.1987.62.5.1865 Google Scholar

45. 

A. H. Zehri et al., “Neurosurgical confocal endomicroscopy: a review of contrast agents, confocal systems, and future imaging modalities,” Surg. Neurol. Int., 5 60 (2014). https://doi.org/10.4103/2152-7806.131638 Google Scholar

Biography

Manuel J. Marques is a postdoctoral researcher in the School of Physical Sciences at the University of Kent. He earned his MSc degree in engineering physics from the University of Porto, Portugal, in 2012, and his PhD in physics from the University of Kent, United Kingdom, in 2016. He has specialized in novel optical coherence tomography (OCT) applications and methods, in particular functional extensions, such as polarization-sensitive OCT and, more recently, endoscopic devices for OCT.

Michael R. Hughes is a lecturer in applied optics in the School of Physical Sciences at the University of Kent. He holds his MSc degree in physics from Durham University and his PhD in physics from the University of Kent. He spent two years training in medical physics at the Oxford Radcliffe Hospitals NHS Trust and five years as a postdoc at the Hamlyn Centre for Robotic Surgery at Imperial College London. He moved to Kent in 2017 to develop a research programme in endoscopic and point of care microscopy.

Andrew Thrapp is a PhD researcher in the Applied Optics Group at the University of Kent, United Kingdom. He graduated with a bachelor of science in physics from the University of Colorado, USA in 2017. During his bachelor’s degree, he was an intern in the Superconducting Quantum Interference Device Lab and his senior project was atomic force microscopy characterizations of single-wall carbon-nanotube thin-films for use as thermoelectrics. His current research is in developing optical-sectioning methods in fiber-bundle endomicroscopy.

Adrian Bradu is a lecturer in applied optics in the School of Physical Sciences at University of Kent, Canterbury, United Kingdom. He obtained his PhD at the University Joseph Fourier, Grenoble France on a project focused on optical techniques used to investigate biological tissues. After his PhD he worked as a research associate, at the University of Kent on subjects mainly related to optical coherence tomography, adaptive optics, confocal microscopy, fiber optics, photo-acoustics, elastography, etc. According to Google Scholar, his papers have been cited 1800 times while his h-index has reached 22.

Christopher J. Payne is a postdoctoral research fellow in the Department of Cardiac Surgery, Boston Children’s Hospital. Previously, he was a postdoctoral research fellow at the Wyss Institute for Biologically-Inspired Engineering, Harvard University. He obtained his PhD at the Hamlyn Centre for Robotic Surgery, Imperial College London. He specializes in medical device design and medical robotic technologies.

Adrian Podoleanu is a professor of biomedical optics at the University of Kent, United Kingdom, and he is a fellow of OSA, SPIE, and IOP. He leads the research of the Applied Optics Group, oriented on high resolution noninvasive optical imaging. He received his Royal Society Wolfson Research Merit Award 2015, Ambassador’s Diploma, Embassy of Romania in United Kingdom, 2009, and the Romanian Academy Constantin Miculescu Prize, 1984. He was supported by an ERC Advanced Research Fellowship, 2010–2015, and by a Leverhulme Research Fellowship, 2004–2006.

Biographies of the other authors are not available.

© The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.
Manuel J. Marques, Michael R. Hughes, Khushi Vyas, Andrew Thrapp, Haojie Zhang, Adrian Bradu, Grigory Gelikonov, Petros Giataganas, Christopher J. Payne, Guang-Zhong Yang, and Adrian Podoleanu "En-face optical coherence tomography/fluorescence endomicroscopy for minimally invasive imaging using a robotic scanner," Journal of Biomedical Optics 24(6), 066006 (20 June 2019). https://doi.org/10.1117/1.JBO.24.6.066006
Received: 11 February 2019; Accepted: 30 May 2019; Published: 20 June 2019
JOURNAL ARTICLE
15 PAGES


SHARE
Advertisement
Advertisement
Back to Top