We performed a comprehensive study of hand–eye calibration approaches for augmented reality (AR) using endoscopes, aiming to find the approaches that yield the best performance and to reveal the mechanism that makes these approaches successful. The two unknown values in this calibration problem are the hand–eye transformation between an endoscope and the endoscope-attached optotracked marker and the transformation between a calibration board with a checker pattern and the board-attached marker. We classify possible approaches to solving hand–eye transformation as direct, simultaneous, and sequential. The effect of the translation components of transformations on an approach’s accuracy is theoretically analyzed using error equations derived from the approaches and demonstrated using both synthetic and real data. We found that sequential approaches performed the best when the magnitude of the translation of hand–eye transformation was larger than that between the board and its marker, which is the general case in implementing AR using endoscopes. In addition, this approach is less sensitive to noise and the number of calibration poses than others. Our results and analyses provide guidance for choosing an optimal hand–eye calibration solution for AR using endoscopes.
To accurately represent the colors in a real scene, a multi-channel camera system is necessary. One of the applications of
the data acquired with the multi-channel camera system is the spectral reflectance estimation. One of the most widely
used methods to estimate the spectral reflectance is the Wiener estimation. While simple and accurate in controlled
conditions, the Wiener estimation does not perform as well with real scene data. Therefore, the adaptive Wiener
estimation has been proposed to improve the performance of the Wiener estimation. The adaptive Wiener estimation
uses a similar training set that was adaptively constructed from the standard training set according to the camera
responses. In this paper, a new way of constructing such similar training set using the correlation between each spectral
reflectance in the standard training set and the first approximation of the spectral reflectance that was obtained by the
Wiener estimation is proposed. The experimental results showed that the proposed method is more accurate than the
conventional Wiener estimations.
Projectors have become common display devices, not only for office and school presentations, but also for home theater
entertainment. Although a completely dark room is the ideal venue for watching a projected image, in most situations
(including classrooms and conference rooms) the viewing conditions are not completely dark, and ambient light falling
on the screen produces a background light level with the image projected on top. As the background light increases, it
becomes more difficult to see the projected image, which becomes dull and may appear washed out. What is really
happening is that the ambient light reduces the contrast of the image. While the amount of light contributing to the image
remains the same, more light has been projected onto the screen by other light sources. This effect can be reduced by
employing the white-peaking function of a digital light-processing (DLP) projector, which adjusts the white segment of
the color wheel, resulting in more natural and vivid images. Although the chromaticity coordinates for an image
projected with and without white peaking are the same, when white is added to the projected image, the perceived hue
changes. This phenomenon is known as the Abney effect. This paper presents a model of this hue-shift phenomenon and
proposes a hue-correction method. For evaluation purposes, an observer-preference test is conducted on several test
images with and without hue shifts, and z-scores are utilized to compare the results.
The spatial gamut-mapping algorithm (SGMA) overcomes the drawbacks of the widely used color-by-color methods.
Spatial gamut mapping can preserve detailed information in original images by performing adaptive gamut mapping in
surrounding pixels within the image. However, spatial gamut mapping can result in hue shift and the halo effect. In
addition, it only preserves the boundary information outside the color gamut; the resulting gamut-mapped image does not
sufficiently preserve the detailed information in the input image. In this paper, we propose an SGMA that utilizes details
of the input image. Our approach improves detail that is not effectively represented with conventional spatial gamut
mapping. This is done by taking an original image and first implementing gamut mapping of the input image. Then, the
details of the input image and gamut-mapped image are extracted. By examining the out-of-gamut region, the details of
the input image can be preserved when these values are added to the gamut-mapped image. The resulting image is
obtained by clipping out-of-gamut pixels, since these pixels are generated in the process of preserving details. We
demonstrated that images obtained using the proposed method are more similar to the input images, compared to images
obtained using conventional methods.
In this paper, subpixel shift estimation method using phase correlation with local region is proposed for registration of
noisy images. Commonly, phase correlation based on the Fourier shift property is used to estimate the shift between
images. Subpixel shift of images can be estimated by the analysis for the phase correlation of downsampled images.
However, in case of images with noise or aliasing artifacts, the error in estimation is increased. Thus, we consider a
small region in a corner of an image instead of the whole, because flat regions with noise and regions with aliasing
induce the error of estimation. In addition, to improve accuracy, the local regions are inversely shifted by varying the
subpixel shift values, and obtaining the peak value of phase correlation between the images. Then, the subpixel shift
value corresponding to the maximum of the peak values is selected. Real-time implementation of this process is possible
because only a local region is used, thereby reducing the process time. In experiments, the proposed method is
compared with conventional methods using several fitting functions, and it is applied for the task of super resolution
imaging. The proposed method shows higher accuracy in registration than other methods, also, edge-sharpness in superresolved
images is improved.