2.1.

Robotic Scanning Device

The robotic scanning device, which may be mounted on a robotic arm for autonomous or hands-on operation, or used in a handheld fashion, is described in Refs. 30 and 34. In summary, it consists of a 58-mm-long hollow steel tube, with 3.3 mm OD, through which imaging probes can be passed, mounted inside a 3-D printed case. The tube is fixed in place at the back of the case while the distal (tissue) end protrudes from the case and is free to move. Partway along the length of the tube, at the front of the case, the tube is fixed to a cam-roller assembly, which allows it to be deflected in two dimensions by the rotation of two brushless DC servomotors. This allows the tip of the tube to be moved over a 2-D workspace of up to $14{\mathrm{mm}}^2$ with an absolute positioning accuracy of better than $30\mu \mathrm{m}$. The tube is offset within the assembly so that it is always under the load throughout the entire workspace, minimizing backlash and hysteresis effects.

The robotic scanner is driven via two analog input signals applied to the motor controllers, which are mounted in a separate casing to minimize the weight of the scanner. For the study reported here, only 1-D motion was required. For ease of alignment of the OCT scanning probe direction, this motion was not necessarily along any of the intrinsic axes of the scanner. The scan was, therefore, performed by sending linear ramps to both motors, scaled by $\cos \theta$ and $\sin \theta$, where $\theta$ is the angle of the OCT 1-D scanning line with respect to the axis of the scanner.

At the tip of the tube, a holder with an OD of 3.7 mm is fixed, originally designed to hold a Mauna Kea Cellvizio Gastroflex UHD endomicroscopy probe (diameter of 2.6 mm) and a laser ablation fiber (diameter of approximately 0.7 mm), as seen in the photograph in Fig. 1(c). For this study, the OCT probe was passed through the endomicroscopy channel and the FEM probe was passed through the laser ablation channel of the original design. The axial positions of the probes were fixed at the rear end of the device case. The OCT probe was mounted with its tip slightly ($\sim 1$ to 2 mm) behind the FEM fiber bundle tip, which was placed flush against the distal end of the 3.7 mm OD holder.

2.2.

Optical Coherence Tomography Endoscopic Scanning Subsystem

The OCT endoscopic scanning subsystem shown in Fig. 1(e) is a swept-source-based system (SS-OCT) that incorporates an FV, 1-D scanning endoscopic probe in the object arm.

This probe, the principle of which has been described elsewhere,^31,35 is based on the voice coil principle, employing the optical fiber as a cantilever, which is attached to the electrical coil, as pictured in Fig. 2. The electrical coil is placed around a magnetic system comprising two magnets with the same poles facing each other. The probe operates in an open-loop configuration, with no position sensing. The fiber tip is imaged onto the sample by a gradient-index (GRIN) lens with a magnification of $5\times$, with a working distance of 1 mm. By applying an alternating electrical current through the coil, a force is generated that laterally shifts the fiber tip, creating a raster scan pattern of up to 2 mm on the sample. The mechanical dimensions of the probe are provided in Fig. 1(b). The part labeled as (ii) has an OD of 1.81 mm at the metal tip, which has a rigid length of 13.20 mm. Elsewhere, the probe tubing has an OD of 1.71 mm.

Fig. 2

Schematic diagram of the distal end of the OCT endoscope, based on a diagram from Ref. 35, showing the different elements present in it. Scanning direction (vertical) is shown by the blue arrow.

Briefly, the SS-OCT system is driven by a 100 kHz A-scan rate, 1310 nm wavelength swept-source, with a bandwidth of 110 nm (Axsun Technologies, Billerica, Massachusetts), the output of which is directed to a fiber-based Mach–Zehnder interferometer, whose arms include the 1-D scanning endoscopic probe and a fiber-based reference arm. The optical power incident on the sample is $\sim 2.8\mathrm{mW}$. Detection is performed by a balanced InGaAs photodetector, BPD (PDB435C-AC, 350 MHz cut-off frequency, from Thorlabs, Newton, New Jersey), whose electrical signal is digitized after a passive high-pass filter (10 MHz cut-off frequency) by a 12-bit high-speed PCIe digitizer (AlazarTech ATS9350, up to $500\mathrm{MS}/\mathrm{s}$ sampling rate, Pointe-Claire, Quebec, Canada), configured with a voltage range of $\pm 1\mathrm{V}$ and digitizing 2048 points per each spectral sweep at $500\mathrm{MS}/\mathrm{s}$.

To produce OCT images, data are processed using the Complex Master–Slave (CMS) method.³⁶ As the CMS method does not need a clock, the maximum sampling speed of the digitizer can be used.³⁷ No hardware-based dispersion compensation is required due to CMS being inherently tolerant to dispersion mismatches in the interferometer;³⁸ CMS method replaces the Fourier transform conventional processing with comparison operations of the channeled spectrum from the sample against masks. These masks are channeled spectra delivered by the same interferometer for different optical path differences in the OCT interferometer using a mirror instead of the sample. As the channeled spectra involved in the comparison operation are produced by the same interferometer, such a procedure is tolerant to chirped channeled spectra, irrespective of whether such chirp is due to nonlinear sweeping or dispersion in the interferometer.³⁹

The B-scan and C-scan images produced via CMS processing were displayed as 8-bit integers with full dynamic range conversion in the LabVIEW IMAQ display control.

2.3.

Fiber Bundle Fluorescence Endomicroscopy Imaging Subsystem

The FEM uses a flexible fiber bundle (Fujikura FIGH-30-650S) comprising $\sim \mathrm{30,000}$ cores to deliver and collect light from a sample, as shown diagrammatically in Fig. 1(f). As no distal optics were used, the fiber bundle, therefore, needed to be in direct contact with the sample. A Texas Instruments Lightcrafter 3000 digital micromirror device (DMD) was used as an illumination source to project uniform blue light onto the sample. The blue light is produced by a light-emitting diode (LED) incorporated within the DMD (with central wavelength of 450 nm and optical power of 32 mW at 450 nm). The output beam from the DMD traverses a low-pass filter (with a 450 nm cut-off), a dichroic mirror, and a 10X microscope objective to deliver 4.3 mW incident on the proximal end of the fiber bundle. While the DMD was destined to generate structured light patterns, for this study it was simply used to deliver flood illumination. The magnification between the DMD pixels and the fiber bundle was sufficient so that the illumination was effectively uniform, as though the bundle is directly illuminated by the LED. The returning fluorescence is imaged onto a CMOS camera (PointGrey Flea 3) via the dichroic and an emission filter (cut-off $>500\mathrm{nm}$). The raw image is cropped to a circular diameter of $525\mu \mathrm{m}$, and Gaussian-filtered ($\sigma =1.2\mu \mathrm{m}$) to remove pixelation due to the fiber bundle cores. Images were acquired at 20 fps.

2.4.

System Integration and Real-Time Display

As mentioned earlier, the three subsystems are controlled by a single workstation PC. The acquired data from the OCT and FEM subsystems are processed by this PC, with the option of displaying both OCT and FEM en-face frames in real-time during acquisition through a custom-made LabVIEW™ virtual instrument (VI) interface. A screenshot of the graphical user interface of this VI is presented in Fig. 3, with a video demonstration of the real-time operation in Video 1.

Fig. 3

Screenshot of the real-time multimodal display LabVIEW™ VI (screencast of the VI in operation in Video 1, MP4, 355 KB) [URL: https://doi.org/10.1117/1.JBO.24.6.066006.1].). (i) OCT B-scan display (reduced to a single line due to the fast scan mode being engaged), (ii) OCT en-face display, (iii) selection of the depth position of the en-face OCT image, (iv) FEM single-frame display, (v) FEM mosaic composition, (vi) fast scan mode toggle, (vii) lateral sine scan correction for OCT 1-D scanner toggle, (viii) scan velocity setting for the robotic scanning device, (ix) number of contiguous masks to be averaged for the OCT en-face display, and (x) toggle switch for interpolation between adjacent lines in the OCT en-face display.

A number of parameters and settings are indicated in Fig. 3. The OCT channel output can be displayed as a live B-scan image by reconstructing each depth using the CMS algorithm,³⁶ which is equivalent to the conventional approach of performing a fast-Fourier transform (FFT) plus any additional dispersion compensation/resampling of the raw B-scan data. This is computationally expensive and unnecessary if an en-face OCT image is preferred (which would only require a few depths out of the whole depth range). To mitigate this issue, the “fast scan” mode in Fig. 3(vi) makes use of the unique property of the CMS method whereby it can reconstruct a subset of the depths with a reduced processing time. The position of this depth interval can be specified by the depth position control, as shown in Fig. 3(iii), and its range can be adjusted by the average control, as shown in Fig. 3(ix). In this way, rather than generating an entire B-scan for each position of the robotic scanner, which would be the conventional approach using FFT-based reconstruction, only a small subset of the B-scan is generated at the required depth. When the “average” parameter is set to 1, this is simply a 1-D array (a lateral T-scan). When the average value is $>1$ (i.e., the reconstructed interval has an axial range $>1\text{pixel}$), the T-scan is created by averaging over this number of depth points. The full OCT en-face image in Fig. 3(ii) at this depth is then constructed from these T-scans. In parallel, the entire dataset can be recorded, allowing subsequent offline reconstruction of volumes, B-scans, and en-face images at any depth.

The 1-D OCT scanner is driven by a sinusoidal waveform; therefore, each raw T-scan is sampled nonlinearly along the lateral scanning function. This was corrected by a resampling algorithm, which can be toggled by control, as represented in Fig. 3(vii).

To improve the small FoV allowed by the fiber bundle [$0.275{\mathrm{mm}}^2$, as shown in Fig. 3(iv)], a mosaicking algorithm similar to previously reported approaches^32,40 was employed to stitch frames acquired during a single scan of the robotic device [as shown in Fig. 3(v)]. For real-time display, frames were registered pairwise using normalized cross correlation (NCC) (i.e., the shift of each frame was calculated relative to the previous frame). The mosaic was then formed by placing the frames dead-leaf (i.e., without blending) in their estimated shifted positions. For off-line reconstruction of saved datasets, the two-way registration described in Ref. 32 was employed and the frames were stitched using alpha blending to remove any visible seams between the images.

The robotic scanning device can potentially be run at high speeds, and the scan velocity can be specified, as shown in Fig. 3(ix). However, since both the OCT and the FEM channels have a limited acquisition rate, a compromise must be made between imaging speed and scan density. At a higher imaging speed (assuming the same scan range), the limited acquisition rate of the channels introduces undersampling in the OCT image along the scan direction (for the FEM channel, it is also necessary to ensure sufficient overlap between image frames, although in practice the OCT sampling requirement is the dominant factor). For real-time display, there was no buffering between the OCT acquisition system and the en-face imaging display system, which runs asynchronously. The sampling along the robotic scan direction was, therefore, not always linear, and so linear interpolation was used between T-scans to generate a uniformly sampled en-face image [activated by the checkbox in Fig. 3(x)].

3.1.

System Resolution and Imaging Range

The main parameters of the two imaging subsystems are shown in Table 1. The axial resolution of the OCT subsystem was measured to be $\sim 24\mu \mathrm{m}$ in air. By performing an analysis on the raw spectral shape of the spectra acquired, similar to what is presented in Appendix C of Rivet et al.,³⁶ we have estimated the theoretical axial resolution to be about $16\mu \mathrm{m}$. The lower axial resolution obtained experimentally can be attributed to the spectral window which has been applied during the signal processing stage. This step was necessary to increase the signal-to-noise ratio (SNR) of the system, thus improving image quality.

Table 1

Table summarizing the main properties of the two imaging subsystems.

The OCT subsystem sensitivity was measured to be 84 dB. The sensitivity measurement procedure is a variation of that described by Bradu et al.⁴¹ and Leitgeb et al.⁴² Briefly, the endoscopic OCT probe was set so as to maximize the recoupling of the optical power returned from a mirror into the probe. The power $P_{\text{high}}$ was then measured at one of the detection ports of the optical interferometer (into the BPD). Then, the mirror was replaced by a low-reflectance target (a block of brushed aluminum), and the optical power measurement was repeated, obtaining $P_{\text{low}}$. With this target in place, an A-scan was obtained, and the peak value $p$ and the noise floor value $n$ were extracted from it, leading to a peak-to-noise ratio value of 50 dB. From the measurements taken, the sensitivity $S$ was calculated as

The lateral resolution of the OCT subsystem was measured using a United States Air Force (USAF) resolution target (Thorlabs R1DS1N). At the lateral range used for this study ($1.6\times 2.7\mathrm{mm}$), it was possible to resolve element 4.1, which is highlighted in Fig. 4(a). This puts the lateral resolution of the OCT subsystem at roughly $60\mu \mathrm{m}$, which is expected due to the probe’s limited NA.

Fig. 4

(a) OCT en-face representation of a USAF target, highlighting element 4.1; (b) single FEM frame from the bundle representing the same USAF target, but covering groups 6 and 7, obtained in transmission, using a fluorescent material placed behind the USAF target; inset: close-up highlighting element 7.1, (c) OCT en-face image of a Ronchi ruling pattern ($1\mathrm{lp}/\mathrm{mm}$), (d) OCT en-face image using the same Ronchi ruling pattern, with the pattern rotated by 90 deg, (e) composite image of (c) and (d), with (c) represented in red and two versions of (d) in blue and gold. These two versions are rotated by 90 deg and stretched in the original $x$-direction to match the span in the $y$-direction of (c). They are then shifted horizontally to match the bars of the Ronchi pattern. Images (c) and (d) differ in contrast because they correspond to slightly different depths in the imaging resolution target.

For system characterization, the robotic scanning device was mounted on an adjustable arm and fixed in place over a table. Test samples were placed horizontally on the table and, except where described otherwise, fixed in place. For all the experiments reported here, the imaging size was kept constant. With the 1-D OCT probe being driven by a 5V amplitude sine wave at 100 Hz (500 A-scans per B-scan acquired) and the robotic scanning device having a velocity setting of $\sim 0.76\mathrm{mm}/\mathrm{s}$ for 400 buffered B-scans, the en-face image dimension was measured to be $1.6\times 2.7{\mathrm{mm}}^2$ ($x\times y$).

The FEM resolution is limited by the sampling density of the fiber bundle cores. For the fiber bundle used, the core spacing is $\sim 3\mu \mathrm{m}$, leading to a Nyquist limited resolution of approximately $6\mu \mathrm{m}$ (although this is shift and rotationally variant). In the image of the USAF target shown in Fig. 4(c), both orientations of element 7.1 can be clearly resolved, conservatively indicating a resolution of at least $7.8\mu \mathrm{m}$. The diameter of each image is $525\mu \mathrm{m}$ and the nominal length of the mosaic, for a scan distance of 2.7  mm, is $\sim 3.25\mathrm{mm}$.

Considering the separate nature of the two scanning directions of the OCT subsystem (one provided by the 1-D OCT probe and the other by the robotic scanner), it is important to understand whether there are significant differences in terms of resolution or distortion. To this end, a resolution target card (Edmund Optics Pocket USAF Optical Test Pattern) featuring a Ronchi Ruling Pattern with $1\text{line}/\mathrm{mm}$ was imaged and an en-face representation was constructed at a depth just beneath the laminated layer. The card was manually oriented in such a way that the patterned lines were perpendicular to the scan direction of the 1-D OCT probe, as shown in Fig. 4(c). The card was then rotated by 90 deg and the same scanning protocol was carried out, yielding the en-face image shown in Fig. 4(d).

To examine whether the spacing between lines was maintained, Fig. 4(e) was constructed from Figs. 4(c) and 4(d). The lines in the images were thresholded and colored; Fig. 4(c) is shown in red in its original orientation. Figure 4(d) was rotated by 90 deg and stretched to fit the vertical dimension of Fig. 4(c). Two versions were produced by translating the figure horizontally in order to match the lines with those of Fig. 4(c), shown in blue and gold.

While the spacing between lines does not seem to change appreciably between the two scanning directions, the behavior around the edges of the image in Fig. 4(d) is not linear; moreover, the bars themselves are slightly curved, as suggested by Fig. 4(d). This may be due to imperfect corrections in the sinusoidal re-sampling function along the lateral scanning direction, as mentioned in Sec. 2.4 (with some deviations at the edges of the image frame), and also due to the operating principle of the 1-D OCT probe,³⁵ which will necessarily introduce off-axis behavior and therefore will introduce a degree of curvature into the imaging plane.

In addition, some artifacts are present in the OCT en-face images [most evidently in Figs. 4(a) and 4(d)], in the form of lines and streaks along the robotic scan direction (the $y$-axis). These are caused by back-reflections in the OCT endoscope introduced by small imperfections (e.g., scratches) on the GRIN optics.

For the study presented in Figs. 4(c)–4(e), and to obtain any of the subsequent OCT en-face images, it is essential to ensure that the two independent scanning directions are approximately orthogonal to each other, as otherwise shear would affect the OCT images and the total imaged area would be reduced. While approximate alignment can be achieved by manually rotating the probe prior to fixing it in place, fine adjustment was achieved via an iterative procedure using a suitable resolution target. Since the robotic scanning device is able to scan in two dimensions (albeit not as fast as the 1-D OCT probe), the scanning protocol allows for a linear scan along any arbitrary angle $\theta$. By changing this angle and observing the corresponding en-face OCT image, it is possible to calibrate the robotic scanning direction, as shown in Fig. 5. In practice, for a clinical device, the OCT probe would need to be modified so that it could only be inserted in the correct orientation.

Fig. 5

Montage of OCT en-face frames (30-frame average over a depth of $150\mu \mathrm{m}$) acquired for several angle settings, $\theta$, on the robotic scanning device. The imaged sample is a paper USAF resolution target (elements 2.4 to 2.6). (a) $\theta =2.7\mathrm{rad}$, empirically aligned with the OCT scanning probe direction, (b) $\theta =2.4\mathrm{rad}$, (c) montage of four $\theta$ settings after the USAF pattern was thresholded out—red corresponds to $\theta =2.7\mathrm{rad}$, green to $\theta =2.6\mathrm{rad}$, blue to $\theta =2.5\mathrm{rad}$, and gold to $\theta =2.4\mathrm{rad}$, (d) same as (a) but with $\pi$ added to the initial angle setting (totaling 5.86 rad), (e) montage with thresholded version of (a) (in red) and thresholded version of (d) flipped along the horizontal direction (blue). Red arrows in (a), (b), and (d) depict the direction of scanning of the robotic scanning probe.

In Figs. 5(a) and 5(b), two OCT en-face images are shown, acquired with different $\theta$ settings on the robotic scanning device (2.7 and 2.4 radians, respectively). For this particular case, it was found that the optimum setting for $\theta$ is 2.7 radians; it is clear from Fig. 5(b) that the USAF pattern is deformed, presenting some shearing.

Figure 5(c) presents this effect more clearly, with four OCT en-face images superposed together (after the USAF patterns were thresholded out), with their corresponding initial angle $\theta$ setting ranging from 2.7 rad (red) to 2.4 rad (gold). It becomes clear that the larger elements present a larger shift (up to $\sim 280\mu \mathrm{m}$) than the smaller elements in the bottom of the image, indicating a shear-based distortion.

The robotic scanning instrument also presents good repeatability in terms of the scanning direction; in Fig. 5(d), an OCT en-face image for a setting of $2.7+\pi$ radians is shown. Since the initial angle is varied by $\pi$, the B-scans making up the OCT volume are now buffered in reverse order, which explains why the image is flipped along the horizontal axis in relation to that in Fig. 5(a). To show that the angle adjustment does not appreciably affect the image, Fig. 5(e) shows a thresholded version of Fig. 5(a) (in red) overlaid on a thresholded and flipped version of Fig. 5(d) (in blue). It is clear that, apart from a slight vertical shift, amounting to $\sim 40\mu \mathrm{m}$ on average (which could be attributed to triggering and timing issues in the acquisition itself), the two images are well matched, which shows that the scanner is invariant to the direction of travel.

3.2.

Repeatability Studies

To assess the scanning repeatability of the system, a procedure was devised involving an imaging phantom, which is schematically represented in Fig. 6(a). This phantom comprises a piece of fluorescent-stained lens tissue paper [Fig. 6(ai)], placed over an imaging resolution card [Fig. 6(aii)] (Edmund Optics Pocket USAF Optical Test Pattern). Since the probe [Fig. 6(aiii)] (and, therefore, the fiber bundle used in the FEM system) is in contact with the lens tissue paper [Fig. 6(ai)], the FEM images should only show the lens tissue paper, whereas the OCT images, having a longer imaging depth range, should show both layers [Figs. 6(ai) and 6(aii)].

Fig. 6

Comparison between en-face OCT and FEM frames acquired from an imaging phantom is schematically represented in (a). This imaging phantom comprises a top layer (i) made of lens tissue paper stained with a fluorescent marker and a bottom layer (ii) incorporating the Edmund Optics Pocket USAF Optical Test Pattern card, which has been immobilized in the sample holder. The probe (iii) is in contact with layer (i) when FEM imaging occurs. Images (b1)-(d2) present comparisons between two datasets acquired at the same position (the first and the last of a set of five volumes acquired consecutively), where points belonging to only one of the datasets are represented in either red or green, and those that overlap are represented in yellow. In (b1)–(b3), layer (i) is not present and the probe (iii) is scanned over the sample without contact, hence only OCT images are generated. Images (b1)–(b2) are B-scans taken at opposite ends of the volume, as represented in dotted lines in (b3), covering the printed pattern of the USAF resolution target, showing element 2.1 on the bottom right corner, and elements 3.3 to 3.6 on the top right corner. In (c1)–(c2), FEM mosaics of layer (i) are represented; (c3) shows the comparison between these two images. Image (d1) shows a comparison between OCT en-face frames at the depth of layer (i), and (d2) shows a similar comparison, but at the depth of the printed pattern of the USAF card.

Two different datasets were acquired, each having five volumes (five runs of the robotic scanning probe). In the first set, layer (i) was removed and the probe was allowed to run in a contactless manner (therefore, no FEM images were produced, only en-face OCT images). In the second set, both layers were present, and the probe was in contact with layer (i) throughout the whole span of the robotic scanning probe movement, and both FEM and en-face OCT images were then obtained.

To visually compare the different volumes acquired within each dataset, the colocalization plug-in from ImageJ Colocalization Colormap⁴³ was used. This plug-in colors each of the two volumes with a separate color (either red or green), with any overlapped points shown in yellow. The results from the first dataset (comparing the first with the last volume) are shown in Figs. 6(b1)–6(b2) (OCT B-scans) and Fig. 6(b3) (en-face OCT), and the results from the second dataset (also comparing the first with the last volume) are shown in Figs. 6(c1)–6(c3) (FEM) and Figs. 6(d1)–6(d2) (en-face OCT).

In Fig. 6(b3), the composite en-face OCT image appears to have a gradient of color, ranging from red at the top to green at the bottom. We believe that this is caused by an axial shift, rather than lateral shift, as shown by the two B-scans in Figs. 6(b1) and 6(b2). Across the five volumes, this axial shift ranges from 50 to $100\mu \mathrm{m}$, and it seems to be more pronounced in the case when the probe is run without contact, i.e., when layer (i) is removed.

To quantify the lateral pixel shifts, the NCC was computed across the en-face OCT images [Figs. 6(b3) and 6(d1)–6(d2)] extracted from the five volumes for each of the two datasets. In addition, in the dataset where the two layers are present, the NCC is computed across the FEM data [Figs. 6(c1)–6(c2)].

Figure 7 depicts a 2-D map of NCC values calculated for each image in the set considered as reference with the other four template images in the set. Obviously, comparing each frame to itself yields a NCC value of 1, as shown by the values along the diagonals in all plots. It is also worth noting that the NCC values in all plots are symmetric in relation to the diagonals since all five images are considered either as references or templates.

Fig. 7

Intensity plots of the NCC values computed for the en-face OCT and FEM images presented in Fig. 6 (values along the diagonal are all equal to 1). (a) Comparison between the five en-face OCT frames acquired without layer (i), with representative frames shown in Fig. 6(b3). (b) Comparison between the five FEM mosaic images acquired [only layer (i) covered], with representative frames shown in Figs. 6(c1)–(c3). (c) Comparison between the five en-face OCT frames whose depth is set to cover layer (i), as shown in Fig. 6(d1). (d) Comparison between the five en-face OCT frames whose depth is set to cover layer (ii), as shown in Fig. 6(d2).

It was found that, for the OCT en-face frames, the average lateral pixel shift is less than the lateral resolution ($60\mu \mathrm{m}$). It was also found that the average of the NCC values (without including the values on the diagonal) is lower ($\sim 0.76$) for the layer depicted in Fig. 6(d1) than the average for the layers of Figs. 6(b3) and 6(d2) ($\sim 0.83$ and $\sim 0.88$, respectively), possibly due to the probe dragging and deforming layer (i) during each scan. The higher average NCC value in Fig. 6(d2) may be due to the relative stability conferred by the probe contact in terms of axial movements. Moreover, the NCC analysis for the FEM data [Fig. 7(b)] yielded the highest NCC values ($\sim 0.93$), with average lateral shifts of around 10 to $11\mu \mathrm{m}$. This seems to be consistent with the standard deviation of the measured mosaic lengths ($\sim 13\mu \mathrm{m}$).

3.3.

Coregistration between Channels

As shown in Fig. 8(a), the two fiber probes are laterally separated by $\sim 1.6\mathrm{mm}$ inside the robotic scanning device tube. Therefore, at any given moment, the OCT and the FEM subsystems image different regions of the sample. However, given the considerable size of the lateral scanning, especially in the $y$-direction ($\sim 2.7\mathrm{mm}$), it is possible to identify a region of the sample covered by both imaging subsystems. This overlapping region depends on the initial setup and calibration of the scanning procedure, particularly the rotation of the OCT probe and hence the scan angle required to generate an orthogonal scan with the robotic scanner. When placing the OCT and FEM probes inside the robotic scanner, we optimized the relative position of the probes so that the separation was roughly along the diagonal of the OCT image, in order to maximize the overlap between the two channels.

Fig. 8

(a) Photograph of the tube showing the two OCT and FEM probes and the distance between their centers. Images (b) to (e) depict the coregistration study for two separate FEM/OCT acquisitions [(b)/(c) and (d)/(e) respectively]. White arrow in the FEM images (b) and (d) depicts scanning direction, and the red shaded area in all frames corresponds to the area that has been covered by the two imaging subsystems. All frames share the same pixel size.

Figures 8(b)–8(e) depict a study carried out with the Edmund Optics USAF test pattern. In this case, the surface of the card was stained with a fluorescent marker and the probe is allowed to run in near contact to the surface of the card. In this way, images of the USAF patterns are obtained in both channels.

The images were scaled so that the pixel sizes were the same, and then the FEM image was translated so that it would match the pattern observed on the OCT en-face image. The overlapped region is shaded in red in all images in Fig. 8. Approximately 7% to 8% of the OCT images were also included in the FEM mosaic. The same probe configuration was used for all other datasets presented in this paper, and so the overlap region remains the same.

There is one important difference between the two acquired datasets shown in Figs. 8(b)/8(c) and 8(d)/8(e). In the latter, it seems that the mosaicked FEM image (d) has a shorter vertical span than the former (b). In fact, when the FEM image is translated and placed above the OCT en-face image (e), the top end of the shaded region does not agree with the features shown in (e), namely element 3.6. This is due to the mosaicking algorithm requiring features in the image to detect the movement. Since in (b) the probe was slightly more shifted to the right, it was able to cover features throughout the whole scanning range. In (d), on the other hand, there is a sizeable gap with no features between elements 3.6 and 2.1, which impacted the mosaicking algorithm by not registering motion where there should have been some. In principle, this problem could be mitigated by using the known velocity of the robotic scanner when there are insufficient image details for registration, although this procedure would not then deal with any tissue deformation.

4.1.

Animal Tissue Experiments

To test the system in a biomedical imaging environment, ex-vivo animal tissue imaging was performed. Ovine kidney and porcine lung/esophagus were stained using the topical staining agent acriflavine hydrochloride (0.01% in water) for 60 s. The samples were then rinsed with water, as described in Ref. 16. During these imaging trials, the robotic scanning was mounted on an arm attached to the table, and the probe was in contact with the sample, yielding OCT volumes and FEM mosaics, as shown in Fig. 9.

Fig. 9

(a1) A 3-D render of a full OCT volume ($2.7\times 1.6\times 3.4{\mathrm{mm}}^3$) of ovine kidney tissue. (a2) OCT color-coded depth projection of the same section of kidney tissue. (a3) FEM mosaic of the same section of kidney tissue as shown in images (a1) and (a2). (a4) FEM mosaic of a different region of kidney tissue, with corresponding OCT en-face image (three-frame average, covering $\sim 30\mu \mathrm{m}$ at a depth of $270\mu \mathrm{m}$ below the surface of the tissue) in image (a5). (b1) A 3-D render of a full OCT volume ($2.7\times 1.6\times 3.4{\mathrm{mm}}^3$) of a section of porcine lung tissue. (b2) OCT B-scan along the $y$-direction (performed by the robotic scanner), showing individual alveoli (yellow arrows). (b3) OCT en-face visualization of the same section of lung tissue (depth average covering $\sim 340\mu \mathrm{m}$ in the axial direction), showing individual alveoli (yellow arrows). (b4) FEM mosaic of the same region (shortened due to probe not being in complete contact with the tissue). (c1) A 3-D render of a full OCT volume ($2.7\times 1.6\times 4.3{\mathrm{mm}}^3$) of a section of porcine esophageal tissue. (c2) OCT B-scan along the $y$-direction (performed by the robotic scanner). (c3) OCT en-face visualization taken from a region $\sim 1\mathrm{mm}$ below the surface (depth average covering roughly 1 mm in the axial direction). (c4) FEM mosaic of the same region, depicting the structures present in the most superficial layer.

Figures 9(a1)–9(a5) all correspond to the same section of kidney tissue, whereas Figs. 9(b1)–9(b4) and 9(c1)–9(c4) correspond to the same section of porcine lung and esophageal tissues, respectively.

In Fig. 9(a1), a 3-D render of the full OCT volume is shown, with a good depiction of the tissue texture being present. Figure 9(a2) employs the same data but in a color-coded depth projection.

As discussed in Sec. 3.3, there is a region that is common to both the OCT images and the FEM mosaic. A correspondence in features can be observed in the boxed region in both Figs. 9(a2) and 9(a3), with a crevice in the surface of the tissue being visible on both images.

If the probe is prevented from running smoothly across the surface of the sample being imaged (due to snagging or extreme tissue deformation), as shown in some of the repeatability studies in Sec. 3.2, then artifacts will arise in both the OCT en-face image and the FEM mosaic. This can be observed in Figs. 9(a3)–9(a4), showing two image datasets collected from the same area of tissue; Figure 9(a4) is evidently much shorter than Fig. 9(a3), despite the fact that the probe was run across the exact same distance and at the same speed. This is because the probe has dragged the tissue during scanning along the robotic device scanning direction. When the tissue moves with the probe, the length of FEM mosaics, which are generated purely based on image registration, is reduced. Artifacts are also observed in the OCT en-face image in Fig. 9(a5) but of a different form. Because the OCT image is assembled based on the expected scanning motion, tissue deformation is exhibited as elongation of tissue features along the $y$-scanning direction, as shown in the regions delimited by the yellow dashed boxes.

Figures 9(b1)–9(b4) all correspond to the same section of porcine lung tissue acquired with both OCT and FEM subsystems. It is possible to identify individual alveoli (yellow arrows) in this particular piece of lung tissue on both B-scan along the robotic scan direction (b2) and en-face (b3) visualizations, with the individual widths ranging between 100 and $130\mu \mathrm{m}$, which is consistent with the values presented in the literature.⁴⁴ Similar to (a4), the FEM mosaic does not have the full length consistent with the actual probe scan, but unlike the previous case, the probe does not appear to be caught in the tissue, dragging it with it; instead, it appears that, due to the topography of the tissue, the fiber bundle of the FEM subsystem lost contact with the tissue in parts of the scan, consequently yielding a shorter mosaic.

Finally, in Figs. 9(c1)–9(c4), a section of porcine esophageal tissue has been analyzed, with the OCT volume allowing for clear distinction between the different layers in both the 3-D volume render (c1) and the B-scan along the robotic scan direction (c2). Unlike the previous two cases, the FEM mosaic (c4) corresponding to this tissue section is complete [with no scanning artifacts present in the en-face visualization in (c3)], although some of the structures appear to have moved with the probe during scanning, which when combined with the mosaic reconstruction yield some artifacts in the image, such as the spiral structure present in the middle of the FEM mosaic (arrowed).

4.2.

Handheld Operation

The robotic scanning device has a relatively small footprint, making handheld operation possible. In Fig. 10(a), a photograph depicts the robotic scanning device in handheld operation, imaging the skin of a volunteer’s index finger. However, in practice we were only able to generate good quality handheld scans when the device was not in direct contact with the tissue. Handheld operation is, therefore, shown only for OCT imaging modality, where the probe could be held a small distance above the tissue.

Fig. 10

(a) Photograph of the robotic scanning device in handheld operation. (b) A 3-D render of a full OCT volume ($2.7\times 1.6\times 2.9{\mathrm{mm}}^3$) of a volunteer’s thumb. (c) Cropped OCT B-scan ($1.48\times 2.9{\mathrm{mm}}^2$) taken from the same volume shown in (b), with the sweat glands highlighted by yellow arrows (i). (d) En-face OCT visualization (three-frame average spanning $\sim 30\mu \mathrm{m}$ at a depth of $700\mu \mathrm{m}$ below the surface) of the same volume, highlighting the interface between stratum corneum and stratum germinativum (ii) but also showing the skin/air interface in (i). Video 2, MP4, 140 KB) [URL: https://doi.org/10.1117/1.JBO.24.6.066006.2]. shows a screencast of the acquisition UI in real time, handheld operation, with the OCT en-face frames refreshing at 1 Hz (sped up by a factor of 2). Owing to the limited en-face imaging rate and the fact that a single en-face image (albeit averaged over $\sim 200\mu \mathrm{m}$) is being displayed, there are some unavoidable motion artifacts both axially and laterally.

Owing to the limited speed and the increased possibility of motion artifacts affecting the images, the 1-D OCT scanning frequency was increased to 250 Hz and the number of lateral points in the OCT volumes was reduced to $200\times 200$ ($x\times y$). A full volume was then acquired in 0.8 s.

Despite some motion artifacts, it is possible to resolve some fine features from in-vivo skin samples, as shown in Figs. 10(b) and 10(c), where an OCT volume was obtained from a volunteer’s thumb. It is possible to distinguish the stratum corneum from the stratum germinativum, and as shown in Fig. 10(c) [which is a B-scan taken from the full OCT volume in (b)], it is possible to visualize sweat glands [arrowed, labeled (i)]. In an en-face visualization of the same region [Fig. 10(d)], it is possible to recognize the dendrite-like structure of the stratum germinativum [arrowed, labeled (ii)].

As mentioned earlier in Sec. 3.1, the OCT images present a slight curvature along the fast ($x$) scanning axis, due to the construction and operating principle of the 1-D OCT probe and the fact that we had to employ a sine wave function to perform the scanning. This can be seen in the 3-D render shown in Fig. 10(b), with the curvature being more evident along the $x$-axis than along the $y$-axis, for a lateral scanning range of the same order of magnitude in each direction.