Colorectal cancer (CRC) is currently the third leading cause of cancer death in the United States, despite increased screening rates and a concomitant decline in incidence and mortality.1,2 CRC commonly arises from adenomatous polyps, some of which may progress into invasive adenocarcinomas over time.3 This progression is thought to be slow and is dependent on the size and histology of the polyp.4 Screening for colorectal polyps via conventional white light colonoscopy has had a dramatic effect on reducing the overall morbidity and mortality due to CRC, as visible polyps can be safely removed during this procedure.5 However, small lesions, sessile (i.e., flat) lesions, or regions of occult dysplasia can be occasionally missed. A broad range of alternative strategies have been recently employed to improve endoscopic screening methods, including autofluorescence endoscopy with real-time image processing, narrow band reflectance imaging, and combined multimodal methods.67.–8 While these approaches have demonstrated improved sensitivity for early detection of neoplasia, false positive rates remain elevated, limiting widespread adoption of these methods for screening purposes.9
High-resolution endoscopic imaging modalities have been widely explored to improve detection of these focal regions of early dysplasia in gastrointestinal epithelium.1011.12.–13 Laser scanning confocal imaging methods employing novel miniaturized distal scanning mechanisms based on microelectromechanical systems have been demonstrated to yield excellent lateral and axial spatial resolution.1415.–16 Clinical trials of a hybrid endoscope/miniaturized laser scanning confocal microscope system have demonstrated utility in diagnosing focal regions of dysplasia in the setting of Barrett’s esophagus and during colonoscopy.17 These systems offer real-time, histology-level information to be displayed to the clinician at the time of endoscopy, potentially greatly improving biopsy targeting to locations most likely to show evidence of dysplasia.18,19 Nonscanning, wide-field fiber bundle microendoscopy methods have also been demonstrated to yield high-resolution image data from the superficial epithelium with promising diagnostic accuracy despite the limited field of view,20 restricted by the biopsy port () through which these devices are deployed in vivo. While certain technical challenges that remain before microendoscopy methods may be widely disseminated in clinical applications, the combination of improved wide-field endoscopic methods and high-resolution microendoscopy methods may greatly improve upon currently achieved accuracy.20,21
Furthermore, although several clinical trials of a high-resolution fiber bundle microendoscopy device have demonstrated excellent interobserver agreement in interpretation and classification of fiber bundle microendoscopy images of colorectal polyps,20 there is currently no widely accepted, quantitative image analysis criteria that can be used to objectively classify fluorescence image data of colorectal polyps without human intervention. Previously demonstrated interobserver agreement of microendoscopic image interpretation is dependent on highly trained staff and physicians. This need for high levels of training can be limiting to the dissemination of microendoscopic screening devices, and quantifiable image analysis criteria is an option to grant unspecialized practitioners the tools for consistent diagnosis at a reduced burden of training.
Quantitative image analysis applied in other microendoscopy studies include texture features of ovarian cancer confocal microendoscopy, cell density in mouse model sarcomas using wide-field structured illumination, and wavelet decomposition in color colonoscopic video frames.2223.–24 These algorithms quantify a range of image features, such as texture (including spatial-frequency content), cell nuclei image segmentation and counting, and discrete wavelet transform and covariance.2223.–24 There are numerous image features that could be extracted for quantification, depending on the data acquired by the imaging modality, but for the quantification of image features in fluorescence microendoscopy of nonspecific staining of bulk tissue, we chose image features that trained clinicians would typically consider in their analysis, namely crypt shape, size, and homogeneity. Pathologists who study en face histopathology of colorectal tissue for signs of dysplasia take into account image features such as elongated or branched crypts, clusters of crypts or crypts of abnormal size, irregular contours, and stratification of the nuclei surrounding the crypt. While pathologists commonly use nucleic patterns as a marker for abnormality, due to the length of the colorectal tract, it is of greater relevance during microendoscopic screening to use a fiber bundle with a larger field of view than to acquire images that resolve individual nuclei. Therefore, the algorithm we developed focuses on quantifying the larger structures of epithelial morphology such as crypt area and circularity. Additionally, the extent to which image morphology varies in clinically normal colorectal tissue in patients with different clinical histories, such as diagnosis, chemotherapy, radiation, prior surgeries, or other therapy, has not been described or quantified. There is a significant need for a quantitative benchmark for future detection of occult dysplasia in patients who have a range of diagnoses and/or prior treatment, and a need for generalized image analysis tools for advanced microendoscopy imaging modalities within the gastrointestinal tract.
Any quantitative image analysis algorithm must be able to exclude motion-blurred or other low-quality images; this is of particular concern in the demanding environment of an in vivo endoscopy procedure. Approaches such as microarray image processors have demonstrated that the inherent variability in signal-to-background ratios is correlated with image quality.25 Low signal-to-background intensity ratios can affect the ability of an algorithm to accurately detect image features, and qualitative image exclusion via manual inspection may introduce human bias and additional error. While some microendoscopy images may be significantly distorted, the parameters for excluding low-quality images for machine detection by observation alone may vary in accuracy when compared to a quantitative threshold based on a standardized metric.
The objective of the study presented in this paper is to explore the quantification of microendoscopy images in the clinically normal appearing tissue surrounding grossly visible lesions in recently resected colorectal specimens, in order to extract key differences that may be used to classify benign epithelial tissue from dysplastic tissue in future computer-aided diagnostic algorithms. We present both a quantitative image quality control algorithm to exclude low-quality images, as well as an intensity-adaptive quantitative image feature extraction algorithm (QFEA) to segment and quantify crypt morphology in images of superficial colorectal epithelium acquired via fiber bundle microendoscopy. Both algorithms were first validated using an extensive training library constructed from image data acquired from fresh porcine colorectal epithelium, a tissue histologically similar to human colorectal epithelium. In addition to individual image acquisition, consecutive images were acquired for mosaicking, to demonstrate the ability to improve the inherently small field of view in fiber bundle microendoscopy images, one prominent disadvantage of microendoscopic systems, using image registration of consecutive fields of view.26 All individual images, and several mosaics, collected from human specimens were then analyzed for quality and subsequently quantified using QFEA. Images were analyzed to segment crypts within the superficial epithelium, and a centroid map was created for identified crypts. From each of these maps, quantitative features were calculated, focused on crypt-specific properties such as crypt area and circularity. While these metrics were calculated for proflavine fluorescence fiber bundle microendoscopy, this method could be broadly applied over a range of high-resolution microendoscopy imaging modalities.
Materials and Methods
Fiber Bundle Microendoscopy System
The microscopy system used has been described previously27,28 and comprises a blue light-emitting diode light source (455 nm, Philips) producing 0.5 mW at the sample, a filter set (Chroma Tech) with a emission bandpass filter, a 460 nm shortpass filter, and a dichroic mirror with a cutoff wavelength of 475 nm, a camera [Flea 3 (USB 3.0), Point Grey Research Inc., California], a objective (NA 0.25, Olympus, Japan), and a customized fiber bundle image guide with a 1 mm active area diameter (FIGH-50-1100N fiber, Myriad Fiber Imaging Tech. Inc.). The fiber bundle image guide has custom SMA connectors on both ends; on the distal end, a modified SMA connector’s rounded edges reduce the friction between the fiber bundle and the epithelial tissue for smoother continuous image acquisition, as well as protecting the glass surface from chipping.
Microendoscopic images of freshly resected colonic porcine tissue were acquired, in collaboration with Siloam Springs Processing (Siloam Springs, Arkansas), a local slaughterhouse. One hundred images were acquired from four healthy animals for training validation of both the quality control and quantitative feature extraction algorithms. These images were acquired from regions in length, with an average of 30 images acquired per region. Porcine colorectal epithelium, histologically similar in structure and scale to human colorectal epithelium, has been previously used as a human analog in myriad imaging studies.29,30 Figure 1 shows a comparison of normal porcine and normal human colorectal tissue using microendoscopic and histopathology images.
Microendoscopic images of freshly resected human colorectal tissue were acquired in collaboration with the University of Arkansas for Medical Sciences (UAMS) in Little Rock, Arkansas, following informed consent of each subject per Institutional Review Board standards at both institutions (IRB #13-06-759 University of Arkansas, Fayetteville, IRB #202224 UAMS). A total of 10 subjects were recruited for this study, with two subjects excluded due to lack of images of clinically normal regions. Thirty-nine microendoscopic images were used to quantify the image segmentation algorithm’s crypt detection sensitivity. Twenty-eight of these images were used for quantification (after exclusion due to image quality, as described in Sec. 2.5). For each human specimen, all microendoscopy images were acquired from a region in length, with an average of 15 images acquired per region. Participation was limited to both male and female subjects over the age of 18 with known or suspected colorectal dysplasia or cancer, and were scheduled for either endoscopic or surgical resection of abnormal tissue. Informed consent was performed on each subject prior to scheduled surgery.
Microendoscopy Image Acquisition
Upon receipt of the freshly resected porcine tissue, the colon specimen was longitudinally sectioned and cleaned using cold saline. Microendoscopy images were acquired immediately following topical application of proflavine (0.01% w/v in phosphate buffered saline) with a cotton swab to epithelial surface, with occasional reapplication of dye after several minutes as needed. Individual images were acquired by manually placing the distal end of the fiber bundle in contact with the epithelial surface of the tissue; consecutive images were acquired by slowly dragging the distal end over the epithelial surface for subsequent processing into mosaics. Sequential image stacks were acquired over 10 to 20 s (covering to 2 cm of epithelium), and all images were acquired in the range of 100 to 150 ms exposure and 0 to 5 dB gain. At the conclusion of the imaging study, porcine tissue was placed in 10% formalin for fixation overnight, prior to paraffin embedding, sectioning, and hematoxylin and eosin staining.
For human subject imaging, the resected colorectal tissue was first longitudinally sectioned and then pinned onto a paraffin block. Additional pins were placed 2 cm apart at regions of interest, and images were acquired along a linear path between the pins. Microendoscopy images were acquired immediately following topical application of proflavine (0.01% w/v in phosphate buffered saline) with a cotton swab to epithelial surface, with occasional reapplication of dye after several minutes as needed. Individual images were acquired by manually placing the distal end of the fiber bundle in contact with the epithelial surface of the tissue; consecutive images were acquired by slowly dragging the distal end over the epithelial surface for subsequent processing into mosaics. Sequential image stacks were acquired over 10 to 20 s (covering to 2 cm of epithelium), and all images were acquired in the range of 100 to 150 ms exposure and 0 to 5 dB gain. For each specimen, typically one to two regions were imaged and sectioned for histopathology confirmation. After imaging, the study pathologist (K.L.) sectioned the tissue along the path marked by the pins, which facilitated the correlation of histopathology to microendoscopy (Fig. 2).
Two-centimeter sections of transversely sliced tissue were taken from imaging sites, and diagnoses provided by a clinically trained gastrointestinal pathologist (K.L.). Tissue was classified as clinically normal, normal with underlying invasive adenocarcinoma, invasive adenocarcinoma, or tubular adenoma at each of the imaged sites.
Quantitative Image Quality Control
We developed a quantitative image quality control algorithm to exclude low-quality images prior to image quantification, based on crypt-specific signal-to-background ratio. This signal-to-background is based on the contrast between the signal intensity within each crypt and the surrounding region; the crypt should be darker than the DNA-containing cell nuclei around it. If the ratio between the intensity values in the surrounding region and the intensity values in the crypt was too low, the algorithm lost sensitivity in image segmentation and quantification (Fig. 3). The image segmentation and quantification algorithm was applied to images of both low and high quality (contrast); specifically, the sensitivity and positive predictive value (PPV) of crypt identification was calculated. Images with low contrast were excluded from pilot study quantification of crypt image features.
Prior to calculating the contrast (signal-to-background) ratio, which we have termed the QR, all images were preprocessed to reduce common intrascene variation, which can result from, e.g., varying amounts of mucosa or variations in staining due to the topography of the tissue. The algorithm first masked the original images to restrict analysis and image segmentation to only the active area of the fiber bundle, and then contrast the enhanced images, whereby intensity values in the image are mapped to new values so that 1% of the image pixels are saturated at the lower and upper contrast limits (in this preprocessing step, 1% was saturated at zero intensity and 1% was saturated at 255), serving to standardize the images by utilizing the full range of the intensity values, before calculating the QR. Images were masked, excluding any values outside a circle 1000 pixels () in diameter, ensuring only the uniformly illuminated region of the active area is analyzed. A microendoscopy image collected from a uniformly fluorescent agarose phantom (2% w/v agarose, mixed with 0.01% w/v proflavine, in distilled water) displayed in the full range of colors in the colormap demonstrates this process (Fig. 4).
To calculate the QR, the image quality control algorithm extracted vertical and horizontal line profiles (90 total) from the image, at intervals of 40 pixels (Fig. 5). This was done to ensure that line profiles would cross the majority of crypts at least once, accounting for the physical scale of crypts (ranging 75 to in diameter) and the scale of the pixels at the image plane (Fig. 5). Each line profile was smoothed to eliminate small intensity variations due to the fiber bundle pattern itself. This fiber pattern exhibits a spatial frequency of ; smoothing eliminates this periodic variation [Figs. 6(a)–6(d)]. Then, local regional minima and maxima were determined for each line profile, and the ratio between each minimum and its adjacent maximum was calculated [Fig. 6(e)]. The regional minima and maxima were determined by comparing regions of eight values (eight-connected neighborhoods) to surrounding values and outputting a binary map of the values of the local minima or maxima. All ratios within each image were then averaged to output a single global average QR metric for the image. Figure 7 shows the flow diagram of steps for preprocessing (image masking and contrast enhancement) and QR calculation.
In order to demonstrate the effect of QR on image quality and overall automated crypt segmentation, we performed the automated QR calculation on the entire porcine image data set (100 images) and human data set (39 images). For each image, in both data sets, the number of crypts was determined via manual inspection to determine a ground truth value. The average QR was tabulated against crypt detection sensitivity (see Secs. 2.6.1 and 3.1) to compute the optimum image exclusion threshold value. The aggregate average sensitivity was calculated by averaging the sensitivity of all images with a QR greater than, or equal to, the respective QR value on the -axis, further described in Sec. 3.2.
Automated Segmentation of Crypt Features
QFEA (Fig. 8) was developed to automatically identify and segment each crypt present within microendoscopic images, and use this segmented map of crypt location to compute morphologic parameters such as area and circularity (Sec. 2.6.2). The accuracy of the QFEA’s image segmentation and crypt identification was greatly influenced by the contrast enhancement applied prior to conversion into a binary image (further explained in Sec. 2.6.1). To increase sensitivity of image segmentation, fivefold cross-validation was performed using the porcine (training) data set in order to compute the average detection sensitivity for different contrast enhancement parameters (Sec. 2.6.2). The selected contrast enhancement contrast limits were then applied to both porcine and human images for calculation of sensitivity and PPV of images with various QR values (results in Sec. 3.2).
Selection of optimal contrast limits for image segmentation, using porcine image test data set
As previously stated, all images were preprocessed to reduce common intrascene variation, by masking and contrast enhancement. Section 2.5 describes the preprocessing contrast enhancement, which used a lower contrast limit value of 0 and upper contrast limit value of 1, hereafter referred as the [0 1] contrast limits. Due to contrast intensity variations in images, further contrast enhancement is necessary to create a binary image of crypts with acceptable crypt detection sensitivity. As seen in Fig. 9, intensity-adaptive contrast enhancement was used to improve overall crypt detection accuracy. Contrast enhancement, which requires selection of lower and upper contrast limits, scales image intensity values (0 to 255, for our images) between 0 and 1, whereby intensity values in the image are mapped to new values so that 1% of the image pixels are saturated at the lower and upper contrast limits, as described in Sec. 2.5. Figure 9 shows the histograms of an original image and images after contrast enhancement with different contrast limit pairs. Crypt detection is not shown for the original image, and the image with [0 1] contrast limits, because the image segmentation algorithm (QFEA) did not detect any crypts (neither true positive nor false positive) for either of these images. Note that the QFEA detected crypts in the image that was contrast enhanced using the [0.4 0.9] contrast limits, which were missed by the [0.5 0.9] contrast limits. Contrast limits affect the ability of the QFEA to recognize and segment crypts in different regions of the image; therefore, the overall crypt detection sensitivity of the algorithm was increased with the combination of multiple contrast limit pairs.
To determine the optimal combination of contrast limit pairs for image segmentation, we iteratively increased the values of the lower and upper contrast limits by 10% (Fig. 10). There were 36 contrast limit pairs in the range from [0.1 0.2] to [0.8 0.9]. Due to extremely low crypt detection sensitivity in images with [0 1] contrast limits, the lower contrast limit 0 and the upper contrast limit 1 were excluded from testing.
A training data set comprising 100 images, acquired from porcine colorectal mucosa, was used to assess the ideal intensity contrast limit pairs to optimize crypt segmentation sensitivity. All 36 upper / lower contrast limit combinations were applied to each image, and the sensitivity and PPV of each contrast limit pair was calculated using the number of true positive, false positive, and false negative crypts detected, as compared to manual inspection of the image. Fivefold cross-validation was used to define test and training sets; one fifth of the data was set aside as the test set, and the remaining four fifths were used to train the algorithm. The algorithm selected the best contrast limits using the training set, and then those contrast limits were applied to the test set to calculate crypt detection sensitivity. This was repeated four additional times, so that each of the five groups served as a test set, and resulting output displayed the selected contrast limits and the average sensitivity of the five test sets. The fivefold cross-validation algorithm was repeated 10 times, with the random assignment of images yielding slightly varying sensitivity averages. Sensitivities of several combinations of contrast limit pairs were also calculated using the fivefold cross-validation algorithm; selection of these combinations is described in Sec. 3.1. The average sensitivity, and standard deviation, of each selected contrast limit pair and combination was tabulated.
Iterative contrast limits and automated crypt identification, segmentation, and morphology quantification
Using the optimal contrast limit pair combinations (determined using the previously described process), porcine and human preprocessed (masked and contrast-enhanced as previously described) individual images and human image mosaics were quantified with QFEA. The quality control step was not used for the initial analysis, of all porcine and human images, in order to determine the optimal QR exclusion value. The QFEA contained the list of selected contrast limit pairs, and passed an image and first set of contrast limits to a quantification subroutine that returned the coordinates of centroids located, and calculated crypt area and circularity. These steps were repeated iteratively (see loop in Fig. 8) with five different pairs of contrast limits, to increase the sensitivity of QFEA. The last step was to save the quantified data and display the centroids (last step in Fig. 8).
Using the optimal contrast limit pair combinations selected after fivefold cross-validation of the crypt segmentation algorithm, as described in Sec. 2.6.1, each of the porcine (100) and human (39) preprocessed images were converted into a binary image, using a variable threshold determined by Otsu’s method,31 choosing a threshold that minimizes the variance of black (value 0) and white (value 1) pixels. The binary image was then eroded, with a disk structuring element of 5 pixels radius. Erosion, in short, changes binary values to 0 unless the structuring element, in this case the disk, overlaps only with binary values of 1 when translated to different positions.32 Since we are quantifying circularity of relatively circular crypts, our structuring element is necessarily round and of a radius of 5 pixels since our individual fibers are in diameter.
The resulting binary image was then inverted, with all 1 values converted to 0, and all 0 values converted to 1, and remasked using the field-of-view (FOV) binary image, to ensure that the crypts are the value-1 binary shapes. Holes were then closed, where if a region of value 0 is completely enclosed by value-1 pixel, the region is converted into value-1s as well. The resulting shapes were once more eroded with a disk (5 pixels radius), and the coordinates of centroids of detected shapes were extracted, hereafter referred to as crypts, as well as each crypt’s area and circularity. The centroid for each crypt is the coordinates for the center of mass of each shape. In the eroded binary image, each crypt is defined as a group of continuous value-1 pixel, and the centroid is determined as the average of those pixel positions based on the equations33. Crypt area was quantified as the actual number of pixels in the shape (binary value of 1, in this case). The circularity was calculated using
Image stack preparation and mosaicking
As described in Sec. 2.4, consecutive images were acquired by slowly dragging the distal end over the epithelial surface for subsequent processing into mosaics. To develop a mosaic image, images from each location (image stack) were cleared of duplicates and images with obvious motion blur. The remaining images in the stack were then preprocessed by masking and contrast enhancement, as described in Sec. 2.5. The image stacks were then stitched together; our mosaics consisted of four to six consecutive images stitched together for a final FOV ranging 3 to 5 mm in length and in height. Mosaics were created from individual images using a modified version of an existing algorithm,31 which requires manual input to select pairs of matching points on two adjacent images, and then uses bilinear interpolation to assign appropriate intensity values in spaces created by stretching or shrinking the second image to match the selected coordinates. Once image stacks were converted into a mosaic, they were treated the same way as a preprocessed individual image and subject to the image segmentation algorithm (QFEA).
Individual mages for this manuscript were acquired from histologically normal regions of human and porcine colorectal tissue, as described in Secs. 2.2 and 2.3. One hundred individual images were acquired from porcine colorectal mucosa as an algorithm training data set and were used in the selection of the best five contrast limit pairs for image segmentation. Thirty-nine images of human epithelial tissue used for automated sensitivity quantification were acquired from clinically normal regions surrounding grossly visible tumors, in excised tissue of eight patients, five diagnosed with invasive colonic adenocarcinoma, two with invasive rectal adenocarcinoma, and one with tubular adenoma. Images of human epithelial tissue used for image mosaicking included both normal and abnormal regions in the same excised tissue of eight patients. All porcine (100) and human (39) images were used for calculating the relationship between average image sensitivity and the QR ratio metric, and determining the optimal QR exclusion threshold value. The images with passing QR were used for quantification of human crypt features in a pilot study, and each individual crypt, as identified by QFEA, was considered a separate data point for quantitative comparison. This comparison is not comprehensive, considering our small sample size, but it shows the ability of QFEA to distinguish variations in crypt size and circularity. A few mosaics were also qualitatively described, as well as subject to the image segmentation and quantification algorithm as proof-of-concept.
Selection of Optimal Contrast Limits for Image Segmentation, Using Porcine Image Test Data Set
For improved crypt detection sensitivity during image segmentation, optimal contrast limit pair combinations were selected using fivefold cross-validation, as described in Sec. 2.6.1. The average sensitivity at each of the 36 contrast limit pairs (Fig. 10) is shown in Fig. 11(a) as a heatmap, with color corresponding to the average percent sensitivity at each contrast limit pair. As described previously, multiple contrast limit pairs increase the sensitivity of QFEA. Figure 11(b) shows the average of the contrast limits with the highest test sensitivity (labeled in Fig. 11 as “1”), as calculated via fivefold cross validation, as well as the average and standard deviation of various combinations of contrast limit pairs. Figure 11(a) shows “1” as the contrast limit pair with the highest stand-alone sensitivity and “2” as the contrast limit pair with the second highest sensitivity; combining contrast limit pairs with the same lower contrast limit (contrast limits labeled “1” and “2” both have a lower contrast limit of 0.5) does not yield as large of an increase in overall sensitivity of the QFEA as combining the best contrast limit pair from each row/lower contrast limit (contrast limits labeled “1” and “3”). As seen in Fig. 11(b), combining the two contrast limit pairs with the highest sensitivity increases the overall test sensitivity by , but combining two contrast limit pairs with different lower contrast limits [from different rows in Fig. 11(a)] yields an increase in overall sensitivity of over 35%. A combination of five contrast limit pairs, the highest sensitivity from each row (lower contrast limit) from 0.2 to 0.7, yielded the highest overall test sensitivity [Fig. 11(b)] before a plateau in performance. While adding more than five contrast limit pairs might slightly increase the average reported sensitivity of QFEA, there is the risk of overtraining the algorithm to be too specific to the training set.
Quantitative Quality Ratio and Crypt Detection Sensitivity of the Image Segmentation Algorithm
All porcine (100) and human (39) microendoscopy images were quantified (QFEA) using the five contrast limit pairs selected in Sec. 3.1. The quantitative image quality control algorithm, described previously, yielded a global QR metric (peak-to-trough ratio) per image, and this average ratio was tabulated along with the crypt detection sensitivity of QFEA, for each image. In Fig. 12(a), the sensitivities and PPVs of images with a QR value equal to, or higher than, the specified QR value (-axis) were averaged for an aggregate average sensitivity at that QR. The peak aggregate sensitivity, before the decline, was at a QR value of 1.07 [Fig. 12(a)]. Therefore, a QR value of 1.07 was selected as the image quality control threshold, excluding for quantification any images with a QR of . QR values used images for average sensitivity calculation, and are not reliable indicators.
Pilot Study of Quantified Crypt Morphology
Quantified crypt data were compared between images of surrounding grossly appearing normal regions in patients grouped as case 1 or case 2; case 1 were patients diagnosed with invasive adenocarcinoma and case 2 were patients diagnosed with tubular adenoma. For individual images, crypt area and circularity were significantly greater for the case 2 group than the case 1 group. The tabulated area and circularity, displayed in Fig. 13 in arbitrary units, were divided by the maximum data value in area and circularity, respectively. A total number of 10 images with 219 cumulative identified crypts were used for case 1, and nine images with 219 cumulative identified crypts for case 2. A two-tailed Student -test yielded values of for the area, and for the circularity.
Qualitative Image Analysis and Automated Crypt Identification of Mosaics
Mosaics were created from consecutive fields of view to demonstrate the ability to improve the inherently small field of view in fiber bundle microendoscopy images, and the image segmentation and quantification algorithm was applied to them to determine if quantification of the mosaics using our algorithm was viable. Figure 14 shows a qualitative comparison of image mosaics acquired from both normal and dysplastic regions of colonic epithelium. The original images were acquired over 10 s, and the new FOV was long and 0.7 mm wide for both mosaics. The mosaics comprise five individual consecutive images. Normal tissue [Fig. 14(a)] has regularly spaced crypts, surrounded by a pericryptal sheath, demonstrated here as a bright ring enclosing the luminal compartment (see yellow arrow). Neoplastic tissue [Fig. 14(b)] oftentimes demonstrates irregularly shaped crypts that are haphazardly arranged, and the yellow arrows point to crypts that have uneven ring structures enclosing them. The crypt detection sensitivity, when applied to the mosaics in Fig. 14, was 70% [Fig. 14(b)] or higher [Fig. 14(a)].
Figure 15 shows an occult hyperplastic polyp in otherwise normal-appearing tissue, in diameter, whose relative size and morphology was highlighted by mosaicking. A stack of eight images was mosaicked using the method stated previously, with the yellow arrows [Fig. 15(a)] pointing to the edges of the hyperplastic polyp in diameter. The imaged region was in length and 0.7 mm in width—the yellow arrows on gross tissue [Fig. 15(b)] point to the edges of the imaged region. The white arrow [Fig. 15(b)] points to the grossly visible tumor, which was to the right of the hyperplastic polyp [yellow arrows, Fig. 15(b)].
Fiber bundle microendoscopy images of ex vivo colorectal epithelium contain useful quantitative data that may be used to assess the tissue microstructure for evidence of occult dysplasia. In this paper, we have described QFEA, an approach to automated quantification of important features of colorectal epithelial microstructure from microendoscopy image data. One hundred images of normal porcine tissue were used to select, via fivefold cross-validation, the best contrast limit pair combinations for crypt detection sensitivity. In addition, we have also demonstrated a method to automatically calculate, and exclude, images with low signal-to-background ratios (QR) in order to filter and remove images that may yield unreliable quantitative data. In clinical settings, the fiber bundle microendoscopy imaging system used in this study may acquire image data via manual placement of the probe directly into the epithelial surface, or remote placement onto tissue during an endoscopic procedure. In both of these applications, significant image quality variability may occur. The majority of the images demonstrated a crypt detection sensitivity of following analysis via QFEA, indicating potentially reliable calculation of quantitative reporters of tissue microstructure.
A combination of contrast limit pairs was tested with the porcine image data set to determine the best parameters for high crypt detection sensitivity. While the contrast limits were iteratively increased by 10%, it is likely that smaller step increases would have provided increased overall sensitivity. A larger data set would be required to test if smaller iterative increases and contrast limit selection would generalize for microendoscopy images, or if the algorithm would be overtrained. The crypt identification algorithm, using the selected combination of contrast limit pairs, was then applied to both porcine and human images, to determine the aggregate performance when excluding images with low QR (Fig. 12). As seen in Fig. 12(b), the number of images, both porcine and human, with QR values drops below 20, decreasing sample size and leading to undersampling bias in the calculated sensitivities at those values. Figure 12(a) reveals that the PPV of human images is drastically lower than that of the porcine image set. We believe this is due to a more carefully controlled imaging procedure with the porcine tissue, since image quality of human tissue is influenced by variations in human preoperative procedures, and sample handling by clinicians and staff. Potential alternatives to improve image quality include alternative contrast agent or optical sectioning techniques such as structured illumination.34
A pilot comparison of quantified crypt morphology (Sec. 3.3) demonstrated the ability of our quantitative image quality control and image feature extraction algorithm (QFEA) to detect minor differences between two otherwise clinically normal appearing colorectal tissue specimens. While a conclusive statement cannot be made based on image data from eight human patients, it does suggest that an algorithm such as QFEA can quantify crypt features objectively and without significant user input. Similarly, no definite quantitative statements can be made for image mosaics, until a larger data set is acquired. Qualitative inspection of the image mosaics shows them as a viable method for overcoming the FOV size limitations of microendoscopy, while image segmentation of the mosaics shows the potential applicability of QFEA to mosaicked images.
In vivo inspection of gastrointestinal mucosa using high-resolution microendoscopy methods has been previously shown to yield clinically useful data. Recent studies examining manual inspection of microendoscopy images of known diagnoses have demonstrated strong interobserver agreement, indicating that well-trained clinicians should be able to effectively incorporate these methods into clinical practice.20 Qualitative assessment of these images by a highly trained observer is likely to continue to yield clinically relevant data, but widespread adoption of high-resolution, in vivo, microendoscopy methods for improved detection of occult dysplasia remains a challenge. A user-independent method of image analysis could provide additional data to make an informed decision and reduce the burden of training required for the clinician at the point-of-care. The introduction of an image analysis algorithm like QFEA into computer-aided diagnostic methods may yield useful complementary data to make improved clinical decisions. In our pilot study, QFEA demonstrates an ability to detect minor differences between two otherwise clinically normal appearing colorectal tissue specimens, suggesting that discriminative quantitative crypt features can be measured objectively and without significant user input.
Computer-aided diagnostic approaches have been of significant research interest in myriad cancer detection approaches, including imaging. Automated quantitative classification has been applied to breast cancer detection using ultrasound images, digitized mammograms, and histopathology, ovarian cancer detection using matrix-assisted laser desorption/ionization imaging mass spectrometry, and prostate cancer grading using ultrasound images and histopathology.35188.8.131.52.–40 Quantitative classification based on microendoscopy image texture features in Barrett’s esophagus and esophageal squamous cell carcinoma, specifically gray-level co-occurrence matrices, has been previously investigated.4142.–43 A recent publication on automated selection of high-quality frames in microendoscopic images44 points to both the need for quality control, as well as the viability of automated extraction of high-quality frames during video-rate acquisition. Ongoing development work with microendoscopic systems may enable spectroscopic data to be used in conjunction with quantitative image features, such as those described in this paper.45 This quantitative data, coupled with extracted image-based features provided by algorithms such as QFEA, could lead to the adoption of standardized, objective methods of in vivo high-resolution microendoscopy image interpretation and analysis.
In conclusion, the methods described here provide insight into the ability of fiber bundle based high-resolution microendoscopy imaging tools to yield clinically relevant data about the tissue microstructure in colorectal epithelium. Effective clinical translation and application of any quantitative image analysis tools applied to this imaging method will need to account for some unavoidable image quality variation, particularly for in vivo endoscopic application of the device on living, constantly moving tissue. Quantitative assessment and control of high-resolution image quality is essential to ensuring reliable results in any future objective, computer-aided diagnostic algorithm development.
The authors would like to thank Amy J. Powless, Haley M. James, and Mary K. Dierks for their role in image acquisition. This work was supported by the Arkansas Biosciences Institute, the National Institutes of Health (NIH)—1R03CA182052-02 and 1R15CA202662-01, and the Southern Regional Education Board (S.P.P.).
Sandra P. Prieto graduated from Andrews University with a BS in engineering and is currently working towards a PhD in biomedical engineering in the Translational Biophotonics and Imaging Laboratory at the University of Arkansas.
Keith K. Lai received his MD from the Saint Louis University School of Medicine, and completed his residency at the Cleveland Clinic Foundation Program. He completed Gastrointestinal Pathology at the University of Utah.
Jonathan A. Laryea completed his medical training at the University of Ghana Medical School, and later completed his internal medicine and general surgery residency at the Waterbury Hospital Health Center, a Yale University Affiliate. He completed a colon and rectal surgery fellowship at the Georgia Colon and Rectal Surgery Clinic in Atlanta. He is currently an associate professor of surgery at the University of Arkansas.
Jason S. Mizell received his MD and completed a general surgery residency at the Louisiana State Health Sciences Center in Shreveport. He completed his colorectal fellowship at Baylor University Medical Center in Dallas, Texas. He is currently an assistant professor of surgery at the University of Arkansas for Medical Sciences.