1.

Introduction

The incidence of esophageal adenocarcinoma (EAC) has increased at an alarming rate over the last four decades. Since 1975, there has been a 463% increase among men and 335% increase among women.¹ This dramatic increase is particularly concerning due to the very low 5-year survival rate (12%) associated with the disease, which is often diagnosed at a late stage.² Conversely, if detected early, the 5-year survival rate can be as high as 81%; however, only a small fraction of esophageal cancers is detected at an early stage.³

Those with an early form of esophageal metaplasia known as Barrett’s esophagus (BE), caused by chronic acid damage over time, are at a greater risk for developing EAC.^4–6 Patients presenting with BE are recommended to undergo routine surveillance at regular intervals.⁷ The standard surveillance procedure uses white-light endoscopic imaging to scan the entire BE segment for visible nodules or epithelial abnormalities. Since there are lesions that can be flat and indistinguishable from nondysplastic mucosa in white-light imaging, the procedure also consists of random four-quadrant biopsies every 1 to 2 cm of the BE segment.⁷ Unfortunately, this standard surveillance procedure lacks the resolution and accuracy that are required to identify all neoplastic lesions in BE; as a result, this procedure has been shown to miss over 50% of dysplastic or cancerous lesions.⁸ Thus, there is an important need to develop and incorporate new techniques that may improve the detection of EAC and its precursors.

A variety of wide-field modalities have been proposed to improve current surveillance techniques including endoscopic ultrasound^9–11 and optical coherence tomography.^12–14 Endoscopic ultrasound has been particularly beneficial for disease staging due to its ability to resolve structures millimeters below the surface.⁹ Optical coherence tomography has shown promise in detecting glands underneath re-epithelialized squamous mucosa,¹² which is of particular interest given the increased use of ablation therapy for BE. However, these technologies are still being evaluated to determine their effectiveness during routine surveillance. Additional wide-field imaging techniques, such as narrowband imaging (NBI)¹⁵ and autofluorescence imaging (AFI),¹⁶ have undergone more extensive clinical evaluation. However, recent studies suggest that NBI and AFI have insufficient clinical accuracy¹⁷ and that AFI has a limited role in diagnostic and therapeutic decision making in routine surveillance and management of patients with BE,¹⁸ in part since inflammatory changes can mimic neoplasia during AFI.¹⁹

Exogenous fluorescent agents can enhance the contrast between neoplastic and non-neoplastic tissue and can potentially improve the accuracy of fluorescence-based endoscopic surveillance protocols. Ingestible agents such as 5-aminolevulinic acid have been shown to identify neoplastic lesions in vivo;^20,21 however, current research prioritizes its use with photodynamic therapy.^22,23 A variety of biomarkers have been identified to assess the presence of disease ex vivo through histologic staining; the cytosponge technique has been shown to aid in identifying those biomarkers postprocedure.^24,25 However, being able to identify and localize these biomarkers at the point of surveillance may help identify lesions in vivo. Indeed, intravenously administered contrast agents that target such biomarkers have been shown to identify neoplastic lesions during in vivo imaging of BE;²⁶ however, additional studies are needed to determine effectiveness during surveillance. Alternatively, topically administered vital-dye such as proflavine hemisulfate has been shown to improve visualization of mucosal architecture in the gastrointestinal (GI) tract when coupled with wide-field and high-resolution fluorescence imaging.^27,28

A modular video endoscope (MVE) capable of in vivo vital-dye fluorescence imaging (VFI) has recently been developed.²⁸ In vivo images acquired with the system suggest that VFI enhances the ability to identify regions of glandular effacement,²⁸ a hallmark of neoplastic progression, which can then be interrogated with higher spatial resolution imaging modalities, potentially enabling a wide-field and confirmatory high-resolution surveillance protocol.²⁸ Here, we describe results from a 14 patient pilot study using the MVE to identify early-neoplastic lesions in BE. Images of 65 sites were analyzed to identify relevant quantitative VFI image features that could be used to classify BE as neoplastic or non-neoplastic. Results show that features extracted from VFI images can be used to objectively classify BE tissues with high sensitivity and specificity.

2.

Methods

2.1.

Instrumentation

The MVE, which has been described previously,²⁸ consists of a modified high definition ($1280\times 1024\text{pixels}$) video processor (Pentax EPK-i) and a standard upper GI endoscope (Pentax EG-29901), modified to enable both white light imaging (WLI) and VFI²⁸ using proflavine hemisulfate, a topically applied fluorescent dye, which stains cell nuclei.²⁹ The MVE is also capable of high-resolution imaging via the instrument channel using the high-resolution microendoscope (HRME). All three modes of imaging are shown in Fig. 1.

Fig. 1

Endoscopic schematics for (a) white-light imaging, (b) vital-dye fluorescence imaging, and (c) high-resolution microendoscopy.

For tissue illumination in the VFI mode, a 455-nm laser diode (Nichia Corporation, Tokyo, Japan) was coupled to the bifurcating endoscope light guide used for white-light illumination. The diode is controlled using a laser diode driver (Wavelength Electronics, Bozeman, Montana), which regulates the input current to adjust the illumination intensity. A mechanical control on the processor is used to switch between white light and laser illumination. A stainless steel VFI filter module containing the necessary optical filters for VFI is attached to the distal tip of the endoscope using a commercially available endoscope cap (Barrx Medical Inc., Sunnyvale, California). The VFI filter module holds a custom-designed long pass filter in front of the endoscope CCD. The filter ($4.6\times 4.8\times 0.8{\mathrm{mm}}^3$) transmits wavelengths longer than 500 nm, enabling collection of fluorescent light; the design and implementation has been described previously.²⁸ With the filter module and blue laser diode in place at the center of the field of view (FOV), the surface irradiance is $\sim 14.9\mathrm{mW}/{\mathrm{cm}}^2$ at 5 mm and $\sim 7.5\mathrm{mW}/{\mathrm{cm}}^2$ at 10 mm, both of which are typical working distances for imaging.²⁸ The full-width half maximum of the laser diode is 12 nm.

The MVE was designed for the standard endoscopic working distance, ranging from 5 to 20 mm. The FOV depends on both the working distance and the digital magnification; at a typical working distance of 10 mm, the FOV ranges from 14 to 45 mm in diameter, with a resolution of $50\mu \mathrm{m}$.

The HRME has been described in detail.^30–32 Briefly, the HRME is a fiber-optic fluorescence microscope in which fluorescence is excited using a 455-nm LED and collected through a 550-nm bandpass filter ($\text{bandwidth}=88\mathrm{nm}$).^30,31 The distal tip of the fiber optic probe is placed in contact with the mucosa for imaging, delivering a total illumination power of 1 mW.^30,33 The fiber optic probe of the HRME can be inserted through the instrument channel of the MVE to acquire fluorescence images with a FOV of $720\mu \mathrm{m}$ in diameter and a lateral spatial resolution of $4.5\mu \mathrm{m}$.

2.4.

Quantitative Image Analysis

Images were extracted from videos recorded from each site. To ensure the images corresponded to the biopsied region, clinical landmarks indicated by the endoscopist (such as the epithelial borders or ulceration), endoscope depth and quadrant, and time stamps were recorded. Videos were evaluated frame by frame to ensure images from the correct site were extracted; all image frames were reviewed and agreed upon by researchers (N.T., M.L., S.A., and R.R.K.) to ensure spatial concordance between WLI, VFI, HRME, and biopsy location.

The extracted images from each clinically normal and abnormal site were reviewed for quality control and a single $250\times 250\text{pixel}$ region of interest (ROI) corresponding to the biopsy site was selected for quantitative analysis. ROIs included in the final evaluation were clear of debris, in focus, and did not contain evidence of movement artifact.

The diagnostic potential of various VFI image features was explored (Table 1). In total, 49 features were computed for each ROI in the VFI images (MATLAB R2011b, Mathworks, Natick, Massachusetts).^38–44 Versions of these features have been utilized in the past to quantify image features of non-neoplastic and neoplastic tissue from various anatomical sites; in this study, they are used to quantify proflavine labeling in BE and associated neoplasia. To explore the relative intensity of proflavine uptake, first-order statistical features (variance, standard deviation, etc.) were computed based on individual pixel values.^37,45 To explore textural image features which help identify glandular epithelium, a gray-level co-occurrence matrix (GLCM) with pixel offsets (1 to 6) was first computed for each ROI. Then features such as correlation, contrast, energy, and homogeneity were computed from each GLCM. This method has been described in detail;⁴⁶ and variations have been used previously.^37,39,42 To explore spatial frequency features which also help identify glandular features, a two-dimensional Fourier transform was used to calculate the power spectrum of each ROI. The resulting power spectrum was divided into 10 individual frequency ranges, where the frequency content in each of the 10 components corresponded to a particular fixed spatial frequency range. This method has been described in detail;⁴⁶ and variations have been used previously.^37,42,47 To characterize epithelial thickness which changes with the progression of neoplasia, granulometry metrics, which assess the size distribution of disk-shaped elements in an image, were computed for each ROI.^41,44 The resulting plot of the total disk surface area as a function of disk size characterizes the relative distribution of various sized disks within each ROI; the disk size range was chosen to be 1 to 100 pixels to accommodate the various sized image features that are seen in the ROIs. Statistical features (skewness, kurtosis, etc.) of this distribution were computed.

Table 1

Description of features calculated for each vital dye fluorescence imaging (VFI) region of interest.

Metric (# of features)	Description
First-order statistical values (6)	Mean, standard deviation, variance, entropy, skewness, kurtosis.
Gray-level co-occurrence matrix (GLCM)—correlation, energy, homogeneity, contrast (24 total; offsets 1 to 6 for each)	Pixel neighborhood correlation in the GLCM, Sum of squared elements in the GLCM, Closeness of the distribution of GLCM elements to the GLCM diagonal, Intensity contrast between pixel and neighboring pixel over entire region of interest (ROI).
Frequency content (10)	Frequency distribution of pixel values in each of the 10 partitions of the power spectrum.
Granulometry (6)	Skewness and kurtosis of relative disk distribution, Most prominent disk size (1 to 100), most prominent large disk size ($>50$), peak value (for disks 1 to 100), peak value (for disks $>50$)
Edge-based morphometry (3)	Sum of long segments, perimeter of long segments, standard deviation of long segments

Since changes in glandular structure are crucial for identifying neoplastic progression, the objective quantification of glandular features in fluorescence images was a priority. During VFI, glands are identifiable by a bright glandular pattern with a dark lumen in the center;^27,28 to characterize these gland edges, a multistep, automated edge detection algorithm was applied to each ROI (Fig. 2). The image processing steps for this feature use bilateral filtering,³⁸ Canny-edge detection,⁴⁰ and global thresholding⁴³ to identify gland edges; parameters for each step were chosen to optimize the detection of glands in all ROIs. Figure 2 details the image processing steps taken to automate the identification of gland edges to calculate relevant features. First, to reduce noise, a bilateral edge-preserving filter was applied to each ROI.³⁸ Second, to identify edges, a Canny-edge detection algorithm was applied to the filtered ROI.⁴⁰ Next, to reduce the over approximation of glands in an image, a binary glandular-epithelium mask was computed. This glandular-epithelium mask was produced by applying a standard deviation filter to the original ROI, where the neighborhood of each image pixel was used to compute the local standard deviation of that neighborhood. Pixel neighborhoods, where the local standard deviation was greater than a global threshold,⁴³ were used to indicate areas containing glandular epithelium. Last, the glandular-epithelium mask was combined with the Canny-edge image using a logical AND operation. The result of the logical AND operation was used to compute the number of gland segments, the average gland perimeter, and the standard deviation of the gland perimeter for each ROI.

Fig. 2

Image processing steps to identify and calculate metrics associated with gland perimeters illustrated for a sample diagnosed as metaplasia (top row) and neoplasia (bottom row). First, a bilateral edge preserving filter is applied to the original image to reduce noise in the image, resulting in the image shown in the second column. Next, a Canny edge detection algorithm is applied to the filtered image, resulting in a binary edge mask shown in the third column. Additionally, to reduce the over-approximation of gland edges by the Canny edge detection algorithm, a glandular-epithelium mask was applied to each image, where the $11\times 11\text{pixel}$ neighborhood of each image pixel was used to compute the local standard deviation. Resulting output pixels, where the standard deviation exceeded a global threshold, were used to approximate areas containing glands. Results are shown in the fourth column. Gland perimeters are then calculated by performing a logical AND operation of the Canny edge detection and the glandular epithelium mask; results are shown in the fifth column.

A Student’s t-test was used to determine whether there was a statistically significant difference ($p<0.05$) in the mean value of each of these 49 parameters for non-neoplastic metaplasia (BE/IND/LGD) and neoplastic sites (HGD/EAC). Stepwise linear discriminant analysis was used to classify images as non-neoplastic or neoplastic using only the image features where the differences in the means were statistically significant. The diagnostic algorithm was developed using leave-one-patient-out cross validation; for each fold, the imaged sites from all but one patient were used as a training set to develop the algorithm; this algorithm was then applied to all the image sites from the withheld patient. This cycle was repeated for each individual patient. Histologic diagnosis was used as the gold standard. In each fold, a sequential forward selection algorithm was used to identify the best performing subset of up to three metrics to classify the image data. As the number of features increased, performance was monitored by noting whether the area under the receiver operator characteristic (ROC) curve increased.

3.

Results

A total of 20 patients underwent the study procedure; four patients were excluded due to lack of metaplasia or neoplasia in the postprocedure pathology read and in two patients, no VFI data were collected due to software error. When the software error occurred in the first two cases, the endoscopist could visualize the real-time endoscopic footage on the high-definition monitor, but video could not be recorded. This error was remedied in the following 18 patients. In vivo images were obtained from 72 sites in 14 patients with biopsy confirmed metaplasia or neoplasia. After quality control, images from 65 sites in 14 patients were analyzed. Of these, 16 sites (eight patients) were diagnosed as neoplasia, and 49 (nine patients) were diagnosed as metaplasia. For these samples, the interobserver agreement for metaplasia (BE/IND/LGD) versus neoplasia (HGD/EAC) was more than substantial ($\kappa =0.836$). The interobserver agreement for BE versus IND/LGD versus HGD/EAC was moderate ($\kappa =0.547$).

Figure 3 shows representative images from metaplasia. In Fig. 3(a), the WLI shows a distinction between squamous epithelium and metaplasia (labeled), however, glandular details are not easily discernible. In VFI, as shown in Fig. 3(b), gland edges are distinct and the glandular pattern appears uniform throughout the region (box). The corresponding HRME image in Fig 3(c) shows thick glandular borders (white arrows), consistent with the VFI features. Additionally, the HRME image shows small, evenly spaced nuclei (yellow arrows). These features are visible in the corresponding histology cross-section shown in Fig. 3(d), showing BE and LGD.

Fig. 3

(a) White-light endoscopic image, (b) vital-dye fluorescence endoscopic image, and (c) high-resolution microendoscope image which were all read endoscopically as non-neoplasia. Shown in (d) is the histology section of the same site. Biopsy was diagnosed as Barrett’s esophagus (BE) with low grade dysplasia. White box in (a) and in (b) indicates area from where the biopsy was taken. White arrows in (c) indicate thick epithelial border. Yellow arrows in (c) show examples of small nuclei.

Figure 4 shows representative images from HGD. In the WLI shown in Fig. 4(a), glandular architecture appears uniform. There is an area (box) that appears raised. In the corresponding VFI (box) shown in Fig. 4(b), distorted glandular architecture is apparent. Glands appear thin and irregular (left arrow); glandular effacement is also present (right arrow). The corresponding HRME image in Fig. 4(c) shows irregularly sized glands (white arrow) and nuclear crowding (yellow circle) throughout. These features are visible in the corresponding histologic cross-section in Fig. 4(d) showing HGD.

Fig. 4

(a) White-light endoscopic image, (b) vital-dye fluorescence endoscopic image, and (c) high-resolution microendoscope image, which were all considered endoscopically suspicious for neoplasia. Shown in (d) is a histology section of the same site. Biopsy was diagnosed as high grade dysplasia. White box in (a) and (b) indicates area from where the biopsy was taken. White arrows in (b) indicate irregular and effaced glands. White arrows in (c) indicate examples of glands with irregular borders. Yellow circle in (c) indicates an example of area with nuclear crowding.

Figure 5 shows representative images from an area of HGD and EAC. In the WLI shown in Fig. 5(a), a nodule is visible (box); areas of hypervascularization are also apparent (star). In the corresponding VFI shown in Fig. 5(b), glandular effacement is observed throughout (box). In the HRME image obtained from the nodule shown in Fig. 5(c), thick brush borders associated with metaplasia are not present and crowded pleomorphic nuclei are prominent (circle), which are visible in the corresponding histology cross-section in Fig. 5(d) showing HGD and EAC.

Fig. 5

(a) White-light endoscopic image, (b) vital-dye fluorescence endoscopic image, and (c) high-resolution microendoscope image were all endoscopically suspicious for neoplasia. Shown in (d) is the histology section of the same site. Biopsy was diagnosed as esophageal adenocarcinoma. White box in (a) and (b) indicates the nodule from which the biopsy was taken. Star in (a) indicates hypervascularization. White arrow in (b) indicates glandular effacement. Yellow circle in (c) indicates nuclear enlargement and crowding.

Figure 6 demonstrates an instance where the WLI image in Fig. 6(a) shows areas of hypervascularization (arrows), prompting a false-positive read by standard endoscopy. The VFI image in Fig. 6(b) from the same area shows characteristic metaplastic patterns where gland edges are easily discernible. The corresponding HRME image in Fig. 6(c) also shows a discernable glandular pattern. The histology cross-section from this image site shown in Fig. 6(d) was read as BE.

Fig. 6

(a) White-light endoscopic image of an area endoscopically suspicious for neoplasia. (b) Vital-dye fluorescence endoscopic image of the same area and (c) corresponding high resolution microendoscope image were both read endoscopically as non-neoplasia. Shown in (d) is histology section of the same site. Biopsy was diagnosed as BE. White box in (a) and (b) indicates the area from which the biopsy was taken. Black arrows in (a) indicate areas of apparent hypervascularization. White arrows in (c) indicate examples of thick epithelial borders, which are characteristic of high-resolution microendoscope images of metaplasia.

Differences in the mean values of five of the 49 image features for neoplastic and non-neoplastic tissue were found to be statistically significant ($p<0.05$) (Table 2). Features which were always selected in the feed-forward step-wise linear discriminant analysis developed include: standard deviation of gland perimeter in an ROI, skewness of size distribution of disks in an ROI, and frequency content in the first partition of the power spectrum.

Table 2

Image analysis features calculated from VFI images with statistically significant differences in mean values for non-neoplastic and neoplastic tissue sites (p-value <0.05).

Figure 7(a) shows the resulting ROC curve for the validation fold; at the Q-point, the sensitivity is 87.8% and the specificity is 77.6% with an area under the ROC curve (AUC) of 0.878. Figure 7(b) shows a scatter plot of the posterior probability values for each site based on a developed two class linear classifier. Table 3 shows the percentage of data points in each diagnostic category that were categorized correctly.

Fig. 7

Receiver operator characteristic curve resulting from leave-one-patient-out cross validation (a). At the Q-point, the sensitivity and specificity are 87.5% and 77.6%, respectively; the area under the curve is 0.878. Corresponding scatter plot (b) shows posterior probability of neoplasia organized by diagnostic category.

Table 3

Percentage of sites in each diagnostic category, which were classified correctly using quantitative algorithm compared to histopathology.

4.

Discussion

We report results from an in vivo study to assess the classification potential of VFI using quantitative image features. Three quantifiable image features were found to consistently aid in neoplasia identification. The first is the standard deviation of gland perimeter, which was typically greater in metaplastic lesions than in neoplastic lesions. The second is that the skew of the distribution of disk sizes, which we hypothesize, represents the extent of glandular structures present. Results indicate that as glandular structures are lost in more advanced neoplastic lesions, this distribution is less skewed. The third is the frequency content in the first partition of the power spectrum; the low frequency content is higher in neoplastic images, likely corresponding to the loss of glandular structure. Linear discriminant analysis using these three features resulted in sensitivity and specificity of 87.8% and 77.6%, respectively, with an AUC of 0.878.

Of the 49 images classified as neoplastic, 12 were falsely positive when compared to the histology gold standard. VFI images from the 12 false positive sites showed neoplastic characteristics such as loss of glandular structure. Potential contributing factors include recent biopsy and resulting re-epithelialization, which has been shown to contribute to false positives in another study evaluating a different fluorescent contrast agent.⁴⁸ While it is important to understand how such factors influence the VFI technique, a larger study is required to make such an assessment. Moreover, additional studies are needed to understand whether high-resolution microscopy or another secondary imaging technique would aid in reducing the number of false positives.

Two of the 16 sites with a histologic diagnosis of neoplasia were falsely negative by VFI. Both were histologically diagnosed as HGD without EAC; one corresponded to a focal HGD. Though some glandular structure was present in VFI, the glands appear fragmented when compared to VFI images of metaplasia. Additional studies are needed to determine whether image characteristics are consistent in a significant number of HGDs without EAC; these larger studies could also incorporate an assessment of how adjusting the cut-off value would impact the overall sensitivity and specificity for different disease categories.

This study uses proflavine hemisulfate, a component of acriflavine, which has been used in vivo in previous GI studies coupled with confocal imaging.^49,50 It is a major component of triple dye, which is used as a part of an antiseptic regimen for the care of newborn umbilical cords;⁵¹ our study concentration is 10 times less than the concentration in triple dye. The long, safe history of clinical use of proflavine, coupled with promising imaging results, support future use of the agent for GI imaging.

This in vivo pilot study marks an important step in clinical translation of vital-dye fluorescence endoscopy. There are many advantages to VFI; video endoscopes can be easily modified to achieve VFI, contrast is provided by a topically applied dye and VFI image results can be quantified. However, our study has a number of limitations which need to be addressed with additional studies. First, though it was not a common occurrence in this study, fluorescence intensity may vary within a FOV potentially impacting the automated edge detection algorithm; in those cases, even glands with weak fluorescence will need to be detected and segmented. Future studies will need to incorporate additional steps in the segmentation algorithm to reduce the potential impact of intensity variation. Next, this pilot study was conducted retrospectively in a small population with a high prevalence of disease to begin to understand the types of features that could contribute to objective classification of disease; in the future, larger studies with separate training and test sets are required to assess overall accuracy and to further characterize feature parameters for neoplasia identification. Additionally, larger studies are needed to determine whether VFI improves the detection rate for neoplasia when compared to white-light endoscopic surveillance and standard four-quadrant biopsies in both high and low prevalence populations. At the same time, it is also important to understand VFI in the context of other current wide-field techniques being evaluated in conjunction with WFI, such as NBI and AFI.^16,19,52 Future studies should address the benefits and limitations of each imaging modality in the same patient subset. Moreover, the role of high-resolution imaging in improving overall diagnostic accuracy should be further investigated; VFI image analysis algorithms that combine features from high-resolution imaging modalities such as confocal microendscopy and high-resolution microendoscopy should be tested to determine how the addition of higher resolution imaging might improve the overall detection of neoplastic lesions. Finally, given the known variability in differentiating BE, IND, LGD, HGD, and EAC,^34,53,54 in a larger clinical study, consensus pathology review with multiple independent pathologists is required to guard against the variability in the gold standard.

This study demonstrates the potential for quantitative features to aid in the interpretation of wide-field images. Subjective interpretation of endoscopic imaging is variable and highly dependent on clinician experience.^55,56 Quantifying image features provides a means for objective interpretation and may also be helpful in guiding the endoscopist’s index of suspicion regarding the presence or absence of neoplasia within a lesion. In order to optimize the potential benefit of this technique, quantitative results must be automated and presented in real-time during endoscopy, allowing endoscopists to use quantitative features to make informed, real-time decisions regarding patient care.