2.4.1.

Field of view

To enable wide-field surveillance of the whole esophagus it is important that the endoscope has sufficient imaging performance across a wide field of view (FOV). In order to measure the FOV we captured 3 images of 1 mm graph paper at four different working distances (WD): 5.7, 8.7, 11.7, and 16.7 mm ($\text{error}\pm 0.3\mathrm{mm}$). The resulting images will show a barrel distortion defined by

2.4.2.

Power

To establish a maximum power for our system, we measured the maximum broadband power of a clinical endoscope (Olympus Evis Lucera CVL-260SL Xenon light source with an Olympus Gastroscope GIF-FQ260Z) and of our endoscope in normal operating conditions, at the same working distance, using a thermal power meter (A-02-D12-BBF-USB, LaserPoint).

2.4.3.

Resolution

The resolution of the system is fundamentally limited by the fiber bundle structure. To determine the limiting resolution, we took images of a 1951 USAF resolution test target (#53-714, Edmund Optics) at 4 working distances using external illumination from a broadband halogen light source (OSL2B2, Thorlabs). Background subtraction was performed using an averaged dark image of 10 frames acquired at the end of the experiment. White light images were analyzed since the test target is printed on white reflective photo paper intended for visible imaging. Since both imaging channels are able to resolve the individual fibers of the fiber bundle, the limiting resolution is determined by the fiber bundle properties rather than the properties of the cameras, and so the resolution measured using the white light images is applicable to both imaging channels.

2.4.4.

Sensitivity

For in vivo application it is important that we are able to detect WGA-IR800CW at low concentrations. The reasons for this are threefold. First, since binding of WGA-IR800CW decreases with the progression of the disease, we must be capable of detecting it at low concentrations to avoid false positives. Second, although WGA occurs naturally in food³⁶ and IR800CW can be produced under cGMP conditions, minimizing the amount of any exogenous agent sprayed inside the esophagus is desirable for clinical translation. Third, we would like to minimize nonspecific binding effects that may become most problematic at high concentrations where they reduce our contrast. Therefore, determining the sensitivity of the system is a crucial aspect of the characterization process.

In order to characterize sensitivity, we prepared a two-fold dilution series of WGA-IR800CW in phosphate buffered saline (PBS) and pipetted $30\mu \mathrm{L}$ of each solution into a well plate ($\mu$-Slide 18 Well—Flat, ibidi GmbH), which had been spray-painted matte black to avoid specular reflections. We then captured images of the dye at 5 different working distances (5, 10, 12, 16, and 20 mm) representative of the range that would be used in vivo. The NIR images were captured using an EM gain of 20 and an exposure time of 200 ms. The images were corrected and coregistered as described in Sec. 2.3. $50\times 50\text{pixel}$ regions of interest (ROIs) were drawn on the white light images inside (signal) and outside (background) of the well. These ROIs were then applied to the coregistered NIR images for analysis in MATLAB^®. The signal-to-noise ratio (SNR) was calculated using

We also characterized the relationship between the SNR and electron multiplying (EM) gain of the NIR sensor by capturing images of $30\mu \mathrm{L}$ of 3100 nM WGA-IR800CW with different levels of gain at working distances of 5, 9.5, and 12 mm. For an EMCCD we would expect

2.5.1.

Imaging of mouse stomachs

To demonstrate the feasibility of imaging WGA-IR800CW on a tissue background we stained excised mouse stomachs with the fluorescently labeled lectin and acquired imaging data using our endoscope. All animal procedures were conducted in accordance with project and personal licenses issued under the United Kingdom Animals (Scientific Procedures) Act, 1986. Surplus mice from breeding were obtained post mortem ($n=6$).

Mouse stomachs were prepared by opening and pinning the excised stomachs on parafilm covered cork. The stomachs were washed with PBS to remove contents before the first round of imaging. This washing consisted of tilting the stomachs and pouring the PBS such that it ran from left to right over the stomachs to avoid cross contamination between the tissue types. Data acquisition was then performed using EM $\text{gain}=20$ and $\mathrm{WD}=7\pm 1\mathrm{mm}$. After the baseline imaging, $100\mu \mathrm{L}$ of $29\mu \mathrm{g}/\mathrm{mL}$ (780 nM) WGA-IR800CW was evenly pipetted onto the face of the stomachs and left to incubate for 10 min. The stomachs were then imaged four more times: immediately after incubation; after a wash with 5 mL of PBS; after a further wash of 15 mL of PBS; and after a final wash of 45 mL PBS. Each round of imaging captured three images: squamous tissue; gastric tissue; and parafilm covered cork to serve as a background control. For each image, an ROI was drawn in the center of the image and the mean signal calculated by fitting a Gaussian to the pixel intensity distribution.

In order to confirm the differential binding of WGA to squamous and gastric tissue types rather than tissue geometry, we repeated the experiment using an additional wash with 33 ml of 1 mM $N$-acetyl-D-glucosamine (Sigma-Aldrich) in PBS, which has been shown previously to compete with WGA for sialic acid binding.³⁶ The washing was performed in the same manner as described above. Excess glucosamine that did not run off was left to incubate for 5 min before a final wash of 50 mL of PBS.

2.5.2.

Imaging of human resection specimens

In order to validate our system in human tissue we performed a pilot study assessing five endoscopic mucosal resection (EMR) specimens from a patient undergoing endoscopic therapy for Barrett’s-related intramucosal EAC. The study was approved by the Cambridgeshire 2 Research Ethics Committee (09/H0308/118).

Resections were first washed with PBS to remove superficial debris and excess mucus then stained with $10\mu \mathrm{g}/\mathrm{mL}$ (268 nM) WGA-IR800CW using a small spray bottle to mimic the spray catheter application that would be used in an endoscopic practice. Tissues were left to incubate for 10 min at room temperature then washed with 15 mL of cold PBS to remove unbound fluorescent probe. The EMRs were imaged with a Fluobeam^® 800 intraoperative wide-field fluorescence imaging system (Fluooptics, France), taken as the gold standard for imaging of NIR fluorescence, and then with our system.

To fit the entire EMRs ($\sim 2.5\mathrm{cm}$ diameter) into a single field of view, we captured images at a working distance of $\sim 2\mathrm{cm}$ using our endoscope, resulting in a decrease in illumination power and resulting SNR. To counter this, we increased our exposure time to 2 s, allowing us to achieve adequate SNR.

In order to compare our endoscope images with the gold standard Fluobeam^® images, we performed a simple coregistration of the images. We first manually drew ROIs around the EMR in the Fluobeam^® image and the white light endoscope image (which is already coregistered with the NIR endoscope image as described in Sec. 2.3). Using these ROIs, we generated binary masks, which were then coregistered using the “imregtform” function in MATLAB^®. This generated a similarity transformation that was applied to the raw Fluobeam^® image to coregister it with the endoscope images. Finally, using both the endoscope mask and the coregistered Fluobeam^® mask, the coregistered images were masked so that only image points inside both of the initial ROIs would be compared in the final coregistered images. This process is summarized in Fig. 3.

Fig. 3

Coregistration of Fluobeam and endoscope images for validation of ex vivo specimen imaging. (a). ROIs drawn around the EMRs in the Fluobeam image and white light endoscope image are turned into binary masks. Initially these are unregistered. (b) We similarity transform the Fluobeam mask in order to maximize overlap with the endoscope mask. (c) The resulting Fluobeam mask and the endoscope mask are multiplied together to find the common mask. (d) The similarity transform found by coregistering the binary masks is now applied to the Fluobeam image to coregister it with the endoscope image. (e) The coregistered images can be compared in the region defined by the common mask, which represents image points that were found inside both of the original unregistered ROIs.

After NIR imaging, EMR specimens were fixed in formalin and embedded in paraffin, according to standard histopathological procedures at Cambridge University Hospitals Human Research Tissue Bank. The EMR paraffin block was cut at intervals of 2 mm and sections mounted onto glass slides. Slides were H&E stained and scanned using an Axio Scan.Z1 and imported into Zen 2 lite software (both Carl Zeiss Microscopy, Jena). The EMR sections were scored by the study pathologist every 1 mm for pathological grade according to the Vienna classification, with the help of the ZEN software graphical grid. This allowed construction of a histology grid, which was superimposed manually onto FluoBeam^® fluorescence images using methodology established previously.⁴⁵ To facilitate histopathological correlation, the normal oesophageal squamous epithelium and oesophageal gastric/intestinal metaplasia (ND Barrett’s) were grouped together as “ND” and any grade of neoplasia, including indefinite for dysplasia (ID), LGD and HGD, and intramucosal cancer, were grouped together as “dysplastic.” To illustrate the relationship between the NIR fluorescence data and underlying pathology, we used EMR B (see Sec. 3.2.2) as there were similarly sized areas of ND and dysplastic tissue within this sample.

3.1.1.

Field of view

We first assessed the FOV of the system. An example image displaying barrel distortion can be seen in Fig. 4(a). The distortion constant $k$ and the constant $A$ were determined by fitting Eq. 4 to the data shown in Fig. 4(b) ($R^2=0.9949-0.9974$). The values of $k$ and $A$ were then used to determine the FOV ($=2r_u$) based on the diameter of the images in pixels ($=2r_d$). Combining these data for the four working distances, we determined the angle of the FOV to be $63\pm 1\mathrm{deg}$ [Fig. 4(c)], which compares favorably to the manufacturer specified angle of 70 deg.

Fig. 4

Optical characterization of the endoscopy field of view (FOV) and barrel distortion. (a) An endoscopic image of 1 mm square graph paper shows barrel distortion. From such an image we can measure $r_u$ and $r_d$ (as defined in Sec. 2.4.1) for several vertices on the paper. (b) An example of the fit to Eq. 4 for images taken at a working distance (WD) of 5.7 mm ($R^2=0.9954$). The straight line shows the case for no barrel distortion. The fit was used to extract the constant $A$ and the distortion parameter $k$. The values of $A$ and $k$ can be used with Eq. 4 to determine the FOV ($=2r_u$) based on the diameter of the images in pixels ($=2r_d$). (c) Determined FOV for four WDs ($R^2=0.9996$). Error bars represent the standard error of the FOV radius derived from the standard deviations of the fit parameters $A$ and $k$. From the fitted line we calculate the angular FOV to be $63\pm 1\mathrm{deg}$.

3.1.2.

Power

To establish a safe maximum power for our endoscope, we compared its maximum broadband power to that of a clinical endoscope. The maximum power from the clinical endoscope in white light reflectance mode was measured to be $19\pm 1\mathrm{mW}$ at a working distance of $1.0\pm 0.1\mathrm{cm}$ while the maximum power from our system was measured to be $3.9\pm 0.2\mathrm{mW}$ at a working distance of $1.0\pm 0.1\mathrm{cm}$.

3.1.3.

Resolution

Resolution was determined by taking images of a USAF test target. At least four replicate data points for each line pair element were captured (centered within the FOV) for each of four working distances, 5, 10, 15, and 20 mm. The Michelson contrast was calculated for each element (before and after honeycomb artifact correction) and the results plotted against half the spatial frequency of the element, which is equal to the reciprocal of the width of a single line element (Fig. 5).

Fig. 5

Extraction of imaging resolution using the Michelson contrast. Resolution was characterized by imaging a USAF chart at four different working distances. The resolution was determined as the point where a linear fit applied in the low-frequency region drops below 1% Michelson contrast, which was determined to be the minimum contrast required for detection of the pattern by an observer viewing the images in normal operating conditions. $R^2=0.9995$, 0.9567, 0.8818, and 0.9138 for working distances of 5, 10, 15, and 20 mm, respectively.

Using a model of the modulation threshold,⁴⁶ which has been shown to reproduce the results of several studies, we determined 1% to be the minimum contrast required for detection of the pattern by an observer viewing the images in normal operating conditions. By finding the intersect of this threshold with linear fits applied in the low frequency region, we determined the resolution of our system to be $182\pm 2\mu \mathrm{m}$, $210\pm 30\mu \mathrm{m}$, $230\pm 50\mu \mathrm{m}$, and $290\pm 60\mu \mathrm{m}$ at working distances of 5, 10, 15, and 20 mm, respectively. These values are not significantly altered if we do not perform honeycomb correction, being $181\pm 3\mu \mathrm{m}$, $200\pm 20\mu \mathrm{m}$, $220\pm 30\mu \mathrm{m}$, and $280\pm 80\mu \mathrm{m}$, respectively.

3.1.4.

Sensitivity

Finally, we established the system sensitivity by investigating the dependence of SNR on concentration; a SNR of 3 was used to determine the minimum detectable concentration at each working distance. Example images from the white light and NIR channels are shown in Fig. 6(a) while Fig. 6(b) shows the relationship between SNR and concentration at working distances of 5, 10, and 12 mm. At working distances beyond 16 mm, the dye was not detectable at any concentration $\le 3100\mathrm{nM}$. A linear fit is made to these curves for $\mathrm{SNR}>1$ since below this we expect only noise in our data. For a working distance of 5 mm, the nonlinear region at high concentration, which is possibly due to saturation, was excluded from the linear fit. The minimum detectable concentrations assessed from Fig. 6(b) are $110\pm 60\mathrm{nM}$ and $430\pm 170\mathrm{nM}$ for working distances of 5 and 10 mm, respectively. Without the honeycomb correction outlined in Sec. 2.3 the minimum detectable concentrations are $170\pm 40\mathrm{nM}$ and $810\pm 520\mathrm{nM}$ for working distances of 5 and 10 mm, respectively, suggesting removal of the honeycomb artifact grants us an improvement in SNR.

Fig. 6

Endoscope sensitivity for detection of WGA-IR800. (a) Coregistered images of white light (left) and NIR fluorescence (right) with example ROIs illustrated. (b) The signal to noise ratio (SNR) increases with the concentration of WGA-IR800. Linear fits were applied giving $R^2=0.9987$, 0.9936, and 0.9974 for working distances of 5, 10, and 12 mm, respectively. The horizontal line shows the detection limit defined as $\mathrm{SNR}=3$. Inset shows the region at low concentration. Error bars have been removed for clarity. (c) SNR of 3100 nM WGA-IR800 follows an inverse power law with the working distance: $R^2=0.9952$. (d) SNR eventually reaches a plateau with increasing EM gain applied to the EMCCD for images taken of 3100 nM WGA-IR800. The lines show the fit according to Eq. 6. with $R^2=0.9358$, 0.9819, and 0.9727 for working distances of 5, 9.5, and 18 mm, respectively. ($\text{Exposure time}=200\mathrm{ms}$ and EM $\text{Gain}=20$ unless otherwise stated. Error bars represent the standard error of the SNR derived from the standard deviation of pixel values in the signal ROI).

We also assessed the influence of working distance and EMCCD gain on these data. The relationship between sensitivity and working distance arises due to the dependence on the irradiance of the excitation light. This was investigated by capturing images of $30\mu \mathrm{L}$ of 3100 nM WGA-IR800CW at eight working distances and plotting SNR against working distance [Fig. 6(c)]. The fit suggests that the SNR falls off as working distance to the power of $-1.91\pm 0.08$, consistent with the inverse square law expected for an illumination cone.

Figure 6(d) shows the fit of our data to Eq. (6) to assess the relationship between SNR and EMCCD gain. According to our data, using a gain of $\sim 20$ provides optimal SNR while preserving the lifetime of our EMCCD.

3.2.1.

Imaging of mouse stomachs

To demonstrate the feasibility of imaging WGA-IR800CW on a tissue background we stained excised mouse stomachs with the dye and acquired data using our endoscope. The upper nonglandular fore stomach, which has squamous tissue at the exposed surface, and the lower glandular stomach, which has simple columnar epithelium (gastric type) tissue at the exposed surface, provide us with a model of the corresponding tissue types found in the human esophagus in healthy (squamous) and diseased columnar-lined esophagus or Barrett’s, respectively. The different regions of tissue within the stomach are indicated in Figs. 7(a) and 7(b).

Fig. 7

Fluorescent WGA-IR800 binding to excised mouse stomach. (a), (b) Photographs of the tissue specimens. The arrow in (a) shows the location of the esophagus. The dotted line shows the approximate location of the cut that was made to open the stomach and expose the inner wall as shown in (b). The arrow in (b) shows the direction of washing. In both images we can see the limiting ridge separating the upper nonglandular fore stomach, which has squamous tissue at the exposed surface, and the lower glandular stomach, which has simple columnar epithelium (gastric type) tissue at the exposed surface, which provide us with a model of the corresponding tissue types found in the human esophagus in healthy (squamous) and diseased columnar-lined esophagus or Barrett’s, respectively. For each stomach three images were taken: one of the center of the squamous tissue (s); one of the center of the gastric region (g); and one of cork as a background (k). (c) The stomachs were stained with WGA-IR800 and washed with PBS. At each time point, images of each tissue type and cork were taken and the mean intensity in a central ROI of each was calculated. The intensity was normalized to the average background cork level for each mouse. The mean of $n=6$ mice is plotted with error bars representing the standard error in the mean. Statistical testing was carried out using 2 way ANOVA. (ns = no significant difference versus cork; * $p<0.05$ versus cork; ***$p<0.001$ versus cork; ****$p<0.0001$ versus cork; †† $p<0.01$ versus squamous; ††† $p<0.001$ versus squamous; †††† $p<0.001$ versus squamous). (d) Following a further wash with glucosamine the bound WGA is removed and the gastric tissue fluorescence intensity returns to the level of the background. Statistical testing was carried out using 2 way ANOVA. (ns = no significant difference; * $p<0.05$).

Gastric tissue is clearly distinguishable (2 way ANOVA, $p=0.0005$) from the squamous tissue using our system [Fig. 7(c)], even following extensive washing with PBS, suggesting that the WGA-IR800CW binds preferentially to this mouse tissue. The results shown in Fig. 7(d) show a significant reduction of the fluorescence to background level (2 way ANOVA, $p=0.0472$) following the glucosamine wash, confirming our results are due to the differential binding of WGA to different tissue types.

3.2.2.

Imaging of human resection specimens

In a first step towards clinical translation, we performed a pilot study on endoscopic mucosal resections (EMRs) collected from a patient with Barrett’s esophagus. The coregistered Fluobeam^® and endoscope EMR images for all collected specimens are shown in Figs. 8(a) and 8(b), respectively. The high signal observed at the edge of the specimens is due to pooling of the dye between the edge of the tissue specimen and the underlying parafilm, which remains even after washing. The intensity recorded in each coregistered pixel in the Fluobeam^® and endoscope images was compared to determine if there was correlation between the images and hence whether the data acquired with our endoscope faithfully recapitulates that acquired with the gold standard Fluobeam^® system.

Fig. 8

Fluorescent WGA-IR800 binding to human endoscopic mucosal resection and correlation to gold standard. (a) Wide-field high resolution NIR images acquired using the Fluobeam intraoperative imaging system. (b) NIR images acquired using the bimodal endoscope. (c) Correlation between pixel intensities in the Fluobeam images and the average pixel intensities of coregistered pixels in the endoscope image. The gray areas correspond to the standard deviation of pixels in the endoscope images. A threshold was placed on the Fluobeam images in order to remove low intensity pixels (due to the pins holding tissue in place rather than signal from the tissue surface itself) and their corresponding endoscope image pixels. The threshold was determined by removing low intensity Fluobeam pixels until these pixels began to correspond to tissue as well as the pins. The same pixels were then removed from the endoscope images. The thresholds were determined to be 65, 65, 80, 75, 75 for EMRs A to E, respectively. The red line shows a robust locally weighted regression smoothing. Spearman correlation coefficients are given within the graphs. High signal intensity observed at the periphery of the specimens is due to pooling of dye between the edge of the tissue specimen and the underlying parafilm.

Intensity scatter plots of these data clearly show a direct relationship between fluorescence intensity in the Fluobeam^® and endoscope images for 4 of the 5 EMRs [Fig. 8(c)]. Spearman’s rank correlation coefficient reveals a moderate but significant correlation between the Fluobeam^® and endoscope images in these 4 of 5 EMRs ($r_s=0.90-0.97$). The fluorescence signal in central 80% of the EMRs within Fluobeam^® images, given as $\text{mean}\pm \text{S.D.}$, was found to be $110\pm 50$, $130\pm 40$, $160\pm 40$, $100\pm 9$, $140\pm 30$ AU for EMRs A to E, respectively. The lack of significant correlation in EMR D is therefore likely due to the relatively uniform fluorescence observed in both Fluobeam^® and endoscope images.

Using EMR B, which had two large regions of ND and dysplastic tissue (see Sec. 2.5.2), we then coregistered the NIR endoscopy data to the histology grid [Fig. 9(a)]. The expected negative binding relationship between dysplastic and ND tissue was confirmed [Fig. 9(b)]. These results provide a preliminary indication that WGA-IR800CW fluorescence imaging with our endoscope would be capable of distinguishing between disease pathologies in esophageal tissue.

Fig. 9

Example coregistration with pathology from EMR B. (a) Histology grid manually coregistered to the Fluobeam image and then transferred to NIR endoscope image (since the Fluobeam and endoscope images were previously coregistered). The outside region was excluded to remove edge effects due to pooling of the dye between the edges of the tissue and the parafilm (yellow). An ulcer identified by the pathologist was excluded (blue). Areas labeled “artifact” in the histology grid may: contain pin holes; have no tissue present for analysis (due to the rectangular cuts); or be blurry in the scanned image. (b) To facilitate histopathological correlation, the normal oesophageal squamous epithelium and oesophageal gastric/intestinal metaplasia (nondysplastic Barrett’s) were grouped together as “nondysplastic” (ND) and any grade of neoplasia, including indefinite for dysplasia (ID), low- (LGD) and high-grade (HGD) dysplasia and intramucosal cancer (IMC), were grouped together as “dysplastic”. Pixel values for the endoscope image are plotted as a histogram for the largest continuous “nondysplastic” (green) and “dysplastic” (red) regions.