Image enhancement based on in vivo hyperspectral gastroscopic images: a case study

Abstract. Hyperspectral imaging (HSI) has been recognized as a powerful tool for noninvasive disease detection in the gastrointestinal field. However, most of the studies on HSI in this field have involved ex vivo biopsies or resected tissues. We proposed an image enhancement method based on in vivo hyperspectral gastroscopic images. First, we developed a flexible gastroscopy system capable of obtaining in vivo hyperspectral images of different types of stomach disease mucosa. Then, depending on a specific object, an appropriate band selection algorithm based on dependence of information was employed to determine a subset of spectral bands that would yield useful spatial information. Finally, these bands were assigned to be the color components of an enhanced image of the object. A gastric ulcer case study demonstrated that our method yields higher color tone contrast, which enhanced the displays of the gastric ulcer regions, and that it will be valuable in clinical applications.


Introduction
Spectral measurements and analyses, which could provide accurate quantifications of the microvascular and morphological properties of gastric mucosa, facilitate the diagnosis of gastric diseases. However, point spectroscopy does not account for the spatial heterogeneity of tissue, as it detects light from a single point, which makes it ineffective for mapping lesion areas. Hyperspectral imaging (HSI), on the other hand, can be used for spatial mapping of tissue morphology and physiology, [1][2][3][4][5][6][7] and it accounts for the spatial heterogeneity of the tissue. This technique involves the acquisition and analysis of a series of reflected two-dimensional images sampled at different wavelengths. [8][9][10][11][12][13] In the gastrointestinal field, most HSI studies have involved ex vivo biopsies, resected tumor tissues, or surface organs, such as the skin, tongue, or larynx. Clancy et al. 13 developed a laparoscope HSI system based on a liquid-crystal tunable filter, Martin et al. 11,12 designed an HSI system that employs fluorescence for larynx imaging, and Leitner et al. 14 established an HSI laryngoscope system based on an acoustic-optical tunable filter, which can collect 51 images with a bandwidth of 5 nm in 1.25 s. Kiyotoki et al. 15 proposed the use of an HSI camera in a diagnostic support system for resected gastric cancer tissues, and indicated that HSI could be employed to measure spectral reflectance in gastric tumors and could differentiate between tumorous and normal mucosa. However, the colors of the resected tissues differed from those observed in vivo, which suggests that the spectral properties of tissue may change after resection.
In the present study, we developed an image enhancement method based on in vivo hyperspectral images for the examination of different types of gastric diseases. Figure 1 shows a diagram of the flexible hyperspectral (HS) gastroscopy system, which can be used to obtain a series of reflected HS images in a noncontact manner in the 405 to 665 nm wavelength range. This system is a modified version of the commercialized endoscope AQ-100 donated by Shanghai Aohua Photoelectric Endoscope Co. Ltd. This system is described in detail in Ref. 16. Twenty-seven sequential narrowband bandpass interference filters are centered at wavelengths between 405 and 665 nm in 10 nm intervals with full widths at half maximum (FWHMs) of 10 nm and ∼75% peak transmission (Shenyang HB Optical Technology Co., Ltd., China) and, together with two all-pass holes, are mounted in two motorized filter wheels positioned in the collimated light beam path. In the conventional white light illumination mode, red (R), green (G), and blue (B) color filters are switched and the all-pass holes in the HS wheels are in the illumination beam path. When the suspected disease area is targeted and the HSI mode is enabled, the 27 monochromatic spectral-channel light beams are produced sequentially in 4.2 s as the filter wheel rotates. An optical fiber bundle directs the beams to the mucosa after the optical taper focuses the light onto the bundle. A combination of lenses is attached to the distal end of the bundle. This combination results in a 5.0-cm-diameter illumination spot at a working distance of 2.0 cm. A 582 × 752-pixel monochromatic charge-coupled device (CCD; ICX279AL, Sony) positioned at the distal end of the optical fiber bundle is used to spectrally resolve the reflected images from the mucosa. Analog signals read from the CCD are sequentially transformed into digital signals in the image processing unit. A cable connects the image processing unit to the medical monitor, which displays the reflected images of the mucosa.

Clinical Data Acquisition
After obtaining ethics approval from Zhongshan Hospital (Shanghai, China, Approval No. 2012-06 (2)) and informed consent from the patients, we acquired HS images of different types of gastric mucosa in a clinical setting. In particular, HS images of gastric ulcers were investigated.

Data Preprocessing
After acquiring sequential HS images containing different spectral information, we performed image preprocessing to reduce the imperfections that arose during imaging and to generate images suitable for analysis. Noise reduction, contrast enhancement, illumination normalization, reflex removal, and soft image registration for moving objects were performed. 17-22

Band Selection
The entropy of an image is defined as 23,24 E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 1 ; 6 3 ; 2 2 0 HðAÞ ¼ − X pðAÞ log pðAÞ; where H is entropy, A is the pixel variable of the image, and p is the probability distribution function of A. The joint entropy is defined as 23 where I indicates the joint entropy, A and B represent the pixel variables of two different spectral images, and pðA; BÞ is the joint probability distribution function of A and B. The joint entropy can also be represented as where θ DI is the DI index and E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 5 ; 3 2 6 ; 6 2 8 HðA 1 ; : : : ; A n Þ ¼ − X pða 1 ; : : : ; a n Þlog 2 pða 1 ; : : : ; a n Þ and E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 6 ; 3 2 6 ; 5 6 8 HðAjBÞ ¼ HðA; BÞ − HðBÞ: The value of θ DI can be used to identify the relevant bands' selection in HS image cubes by performing the following procedure: 23 Step 1: Calculate the entropy for each image in the image cube, and make the image with the highest entropy be the first main component W 1 .
Step 2: Calculate the mutual information (MI) between W 1 and the other images in the image cube, and make the image with the lowest MI be the second main component W 2 .
Step 3: Calculate θ DI for W 1 and W 2 , and the other images in the image cube, make the image that has the minimum θ DI be the third main component W 3 .
Step 4: Continue step 3 until enough components have been selected.

Image Enhancement and Evaluation
After selecting the band subset in this study, the selected bands were assigned to be the new R, G, and B components for the enhanced image. After performing white balancing, the effects of this image enhancement method were investigated by comparing the new image with the conventional color image.

Results
After developing the system, we obtained 29 in vivo HS image cubes for normal gastric mucosa tissues and different types of gastric disease tissues and objects from 12 volunteers in a clinic. Figure 2 shows example images from an HS image cube for a gastric ulcer. The spectral characteristics in the ulcer region are significantly different from those of the normal tissue and the erythematous mucosa around the disease area. Figure 3 presents the spectral curves from three typical regions in the image. As shown, the ulcer region has a reflectance higher than those of the erythematous and normal regions at all visible wavelengths. This increased reflectance is related to the fact that the ulcer region is whiter than its surroundings. Below 595 nm, the reflectance of the erythematous region is lower than that of the normal tissue regions, while above 595 nm, their reflectances are similar. Hence, the erythematous region appears redder than the normal region. We used this gastric ulcer HS image cube in the image enhancement case study. Table 1 presents the band selection results obtained using the DI algorithm, which shows that the first three selected bands are from the G, B, and R wavelength regions. These three bands were assigned to be the new R, G, and B components for the enhanced image. We also assigned the 465 nm band to be the B component for comparison. Figure 4 shows the enhanced imaging results obtained using the new assignments. Figures 4(a) and 4(b) show that the ulcer region appears yellower in the enhanced image than in the conventional image, while the normal tissue regions appear whiter. The erythematous region located between the ulcer region and the normal region appears similar in the enhanced and conventional images. Thus, the enhanced image facilitates differentiation between the ulcer and erythematous regions from the normal tissue region.
From Figs. 4(b) and 4(c), it is evident that the erythematous and normal regions have similar color tones, while the ulcer region exhibits a different color. This finding indicates that different band selections could produce different color tones and enhanced results.

Discussion
For a gastroscopy system with CCDs operating at low speeds, the use of more than three bands would prevent real-time imaging. Hence, three bands should be employed to facilitate diagnosis, as the bands would be selected according to the specific disease and would have FWHMs narrower than those in conventional white light imaging and narrowband imaging.
Despite the interesting results obtained in this study, this method should be employed clinically in the future to verify  its effectiveness in actual application. Moreover, since many factors, such as the organ, location, disease stage, and environment, may cause the reflected spectral images to vary, we do not expect that these band selection results will be universally applicable. In this study, we simply proved that this image enhancement method based on in vivo HS images is useful and that it can improve the color tone contrast of images used for disease diagnosis.
In the future, after analyzing a large number of cases involving in vivo HSI application and the bands selected for each disease and individual, we will be able to determine the optimal spectral band subsets that are effective for most patients with specific diseases.

Conclusion
We proposed and demonstrated an image-enhancement method based on band selection for in vivo HS gastroscopic imaging. A flexible gastroscopy system was modified with HS technology and was used to obtain in vivo HS images for different types of mucosal diseases and objects. A band selection algorithm based on DI was employed to analyze HS images of a gastric ulcer, and three bands (645, 545, and 495 nm) were identified from these images. After defining these selected bands as the new R, G, and B components to be used to obtain the enhanced image, it was found that this new method yielded color tone contrast higher than that of the corresponding conventional color image. Although further study is required, this method has great potential for application in clinics.