KEYWORDS: Endoscopy, RGB color model, Stomach, 3D modeling, Data modeling, Data acquisition, Inspection, Diagnostics, 3D image reconstruction, 3D image processing
Gastroendoscopy is the golden standard procedure that enables medical doctors to investigate the inside of a patient's stomach. Monocular depth estimation from an endoscopic image enables the simultaneous acquisition of RGB and depth data, which can boost the capability of the endoscopy for various potential diagnostic applications, such as the RGB-D data acquisition toward whole stomach 3D reconstruction for lesion localization and local view expansion for lesion inspection. Therefore, deep-learning-based approaches are gaining traction to provide depth information in monocular endoscopy. Since it is very difficult to obtain ground-truth RGB and depth image pairs in clinical settings, computer-generated (CG) data is usually used for training the depth estimation network. However, CG data has a limitation to generate realistic RGB and depth data. In this paper, we propose a novel data generation strategy for self-supervised training to predict the depth in gastroendoscopy. To obtain dense reference depth data for training, we first reconstruct a whole stomach 3D model by exploiting chromoendoscopic images sprayed with indigo carmine (IC) blue dye. We then generate virtual no-IC images from chromoendoscopic images using CycleGAN to make our depth estimation network applicable to general endoscopic images without IC dye. We experimentally demonstrate that our proposed approach achieves plausible depth prediction on both chromoendoscopic and general white-light endoscopic images.
Color correction is one of the most essential camera imaging operations that transforms a camera-specific RGB color space to a standard color space, typically the XYZ or the sRGB color space. Linear color correction (LCC) and polynomial color correction (PCC) are two widely used methods; they perform the color space transformation using a color correction matrix. Owing to the use of high-order terms, PCC generally achieves lower colorimetric errors than LCC. However, PCC amplifies noise more severely than LCC. Consequently, for noisy images, there exists a trade-off between LCC and PCC regarding color fidelity and noise amplification. We propose a color correction framework called tunable color correction (TCC) that enables us to tune the color correction matrix between the LCC and the PCC models. We also derive a mean squared error calculation model of PCC that enables us to select the best trade-off balance in the TCC framework. We experimentally demonstrate that TCC effectively balances the trade-off for noisy images and outperforms LCC and PCC. We also generalize TCC to multispectral cases and demonstrate its effectiveness by taking the color correction for an RGB-near-infrared sensor as an example.
KEYWORDS: Color difference, Image processing, Algorithm development, Image acquisition, Visualization, Image interpolation, Color imaging, Digital cameras, Cameras, RGB color model
A color difference interpolation technique is widely used for color image demosaicking. In this paper, we propose
a minimized-laplacian residual interpolation (MLRI) as an alternative to the color difference interpolation, where
the residuals are differences between observed and tentatively estimated pixel values. In the MLRI, we estimate
the tentative pixel values by minimizing the Laplacian energies of the residuals. This residual image transfor-
mation allows us to interpolate more easily than the standard color difference transformation. We incorporate
the proposed MLRI into the gradient based threshold free (GBTF) algorithm, which is one of current state-of-
the-art demosaicking algorithms. Experimental results demonstrate that our proposed demosaicking algorithm
can outperform the state-of-the-art algorithms for the 30 images of the IMAX and the Kodak datasets.
KEYWORDS: RGB color model, Near infrared, Optical filters, Color difference, Image filtering, Lutetium, Image quality, Cameras, Algorithm development, Linear filtering
Extra band information in addition to the RGB, such as the near-infrared (NIR) and the ultra-violet, is valuable for many applications. In this paper, we propose a novel color filter array (CFA), which we call “hybrid CFA," and a demosaicking algorithm for the simultaneous capturing of the RGB and the additional band images. Our proposed hybrid CFA and demosaicking algorithm do not rely on any specific correlation between the RGB and the additional band. Therefore, the additional band can be arbitrarily decided by users. Experimental results demonstrate that our proposed demosaicking algorithm with the proposed hybrid CFA can provide the additional band image while keeping the RGB image almost the same quality as the image acquired by using the standard Bayer CFA.
Spectral reflectance is an inherent property of objects that is useful for many computer vision tasks. The spectral
reflectance of a scene can be described as a spatio-spectral (SS) datacube, in which each value represents the
reflectance at a spatial location and a wavelength. In this paper, we propose a novel method that reconstructs
the SS datacube from raw data obtained by an image sensor equipped with a multispectral filter array. In our
proposed method, we describe the SS datacube as a linear combination of spatially adaptive SS basis vectors.
In a previous method, spatially invariant SS basis vectors are used for describing the SS datacube. In contrast,
we adaptively generate the SS basis vectors for each spatial location. Then, we reconstruct the SS datacube
by estimating the linear coefficients of the spatially adaptive SS basis vectors from the raw data. Experimental
results demonstrate that our proposed method can accurately reconstruct the SS datacube compared with the
method using spatially invariant SS basis vectors.
Multispectral imaging is highly demanded for precise color reproduction and for various computer vision applications.
Multispectral imaging with a multispectral color filter array (MCFA), which can be considered as a
multispectral extension of commonly used consumer RGB cameras, could be a simple, low-cost, and practical
system. A challenge of the multispectral imaging with the MCFA is multispectral demosaicking because each
spectral component of the MCFA is severely undersampled. In this paper, we propose a novel multispectral
demosaicking algorithm using a guided filter. The guided filter is recently proposed as an excellent structurepreserving
filter. The guided filter requires so-called a guide image. A main issue of the guided filter is how to
obtain an effective guide image. In our proposed algorithm, we generate the guide image from the most densely
sampled spectral component in the MCFA. Then, ohter spectral components are interpolated by the guided
filter. Experimental results demonstrate that our proposed algorithm outperforms other existing demosaicking
algorithms both visually and quantitatively.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.