Color filter array demosaicking with local color distribution linearity

Yuanjie Zheng; Stephen Lin; Jie Yang

doi:10.1117/1.1906084

Translator Disclaimer

1 May 2005 Color filter array demosaicking with local color distribution linearity

Yuanjie Zheng, Stephen Lin, Jie Yang

Author Affiliations +

Optical Engineering, 44(5), 050501 (2005). https://doi.org/10.1117/1.1906084

Abstract

We propose a novel demosaicking method based on the linearity property of a local color distribution. With the proposed technique, the color filter array can be demosaicked with less "confetti" types of errors and fringe artifacts than many current demosaicking methods. Furthermore, edge details are well preserved.

1. Introduction

Due to hardware limitations, the single-chip CCD or CMOS solid state sensor array in digital cameras does not measure a complete triplet of red, green, and blue color values for each pixel in an image. Instead, it captures a sparsely sampled image of each of the color planes with a sensor whose surface is covered with a color filter array (CFA). To produce a full RGB image from these subsampled color values, CFA demosaicking is then used to reconstruct the original colors.

The Bayer array¹ shown in Fig. 1 is one of the many typical CFA patterns used in digital still cameras. A variety of methods have been proposed for demosaicking such a pattern. The simplest one is linear interpolation, which does not maintain edge information well. More advanced methods² ³ ⁴ perform CFA interpolation in a manner that preserves edge details.

Fig. 1

Sample Bayer pattern.

A property of many local edge regions is the linearity of its color distribution in RGB space,⁵ which also exists for homogeneous regions. We captialize on the linearity property of local color distributions to produce a novel demosaicking method that can result in fewer demosaicking artifacts while preserving edge details better than many current demosaicking methods.

2. Linearity Property of Local Color Distributions

As described in Ref. 5, because of the limited spatial resolution of the image array, the image plane area of an edge pixel will generally image portions of both regions that bound the edge. For an edge pixel that lies between two regions having distinct RGB color vectors I₁ ^′ and I₂ ^′, its measured RGB color vector I₀ should be a linear combination of the bounding region colors:

Eq. (1)

I_{0} = α I_{1}^{'} + (1 - α) I_{2}^{'}

where α is a value within interval [0,1]. According to this local linearity property, I₀ should be located on the line segment between I₁ ^′ and I₂ ^′ in the 3-D RGB space. The linearity property also suggests that local changes in the three color components should be consistent with one another, expressed as

Eq. (2)

\frac{r_{0} - r_{1}^{'}}{r_{2}^{'} - r_{0}} = \frac{g_{0} - g_{1}^{'}}{g_{2}^{'} - g_{0}} = \frac{b_{0} - b_{1}^{'}}{b_{2}^{'} - b_{0}}

where r_k ^′, g_k ^′, b_k ^′ represent respectively the red, green, and blue values of I_k ^′, and r₀, g₀, b₀ represent respectively the red, green, and blue values of I₀.

In this work, only three consecutive pixels on a line in the CCD array tessellation are regarded as complying with the linearity property. For example, in Fig. 1, I₂₁, I₂₂, and I₂₃ should be linear with regard to 4-connectivity, and I₁₁, I₂₂, and I₃₃ should be linear in the sense of 8-connectivity.

3. Linearity in Demosaicking

The linearity property shown in Eq. (2) describes expected relationships among the color components of neighboring pixels. Missing components can be determined by incorporating the linearity property into the demosaicking problem.

The green channel is first interpolated. Referring to Fig. 1, we estimate G₃₄ of a red CFA pixel by first computing α₁=|G₃₅−G₃₃|, α₂=|G₄₄−G₂₄|, β₁=|B₄₃−B₂₅|, and β₂=|B₄₅−B₂₃|. These quantities are used to determine whether pixel I₃₄ is located on a vertical, horizontal, or diagonal edge. The following estimates are then used for the missing green pixel value:

Eq. (3)

G_{34} = {\begin{array}{l} (G_{33} + G_{35}) / 2 & if α_{1} = M P \\ (G_{24} + G_{44}) / 2 & if α_{2} = M P \\ (G_{24} + G_{33}) / 2 & if [(β_{1} = M P) & (| B_{a v g 1} - B_{23} | < | B_{a v g 1} - B_{45} |)] \\ (G_{35} + G_{44}) / 2 & if [(β_{1} = M P) & (| B_{a v g 1} - B_{45} | < | B_{a v g 1} - B_{23} |)] \\ (G_{24} + G_{35}) / 2 & if [(β_{2} = M P) & (| B_{a v g 2} - B_{25} | < | B_{a v g 2} - B_{43} |)] \\ (G_{33} + G_{44}) / 2 & if [(β_{2} = M P) & (| B_{a v g 2} - B_{43} | < | B_{a v g 2} - B_{25} |)] \end{array}

where MP=min(α₁,α₂,β₁,β₂), B_avg1=(B₂₅+B₄₃)/2 and B_avg2=(B₂₃+B₄₅)/2. In Eq. (3), the last four cases correspond to diagonal edges. For example, a diagonal edge from the lower left to upper right is addressed in the third and fourth cases. For this kind of edge, I₃₄ is first grouped to either the upper left or lower right triangle formed by the edge in the 8-neighborhood, depending on which triangle has the more similar blue value. Then G₃₄ is estimated by the known green values in the selected triangle. The green channel value for a blue CFA pixel can be interpolated similarly.

After demosaicking the green color plane, the blue and red values of green CFA pixels are then estimated using the linearity property as follows, using I₄₄ as an example.

Eq. (4)

B_{44} = {\begin{array}{l} (B_{45} + T B * B_{43}) / (1 + T B) & (T B \neq - 1) & (T B \neq Inf) \\ (G_{44} / G_{43}) B_{43} & (T B = - 1) & (G_{43} \neq 0) \\ B_{43} & (T B = - 1) & (G_{43} = 0) \\ B_{43} & T B = Inf \end{array}

where

T B = {\begin{array}{l} (G_{45} - G_{44}) / (G_{44} - G_{43}) & G_{44} \neq G_{43} \\ Inf & G_{44} = G_{43} \end{array}

R₄₄ can be determined similarly to Eq. (4) by the known green and red components value of I₅₄, I₄₄, and I₃₄.

Linearity is also used to estimate the missing red values for blue CFA pixels, and the blue values for red CFA pixels. Using I₅₄ as an example, the blue value of a red CFA pixel is interpolated as

Eq. (5)

B_{54} = (B_{54}^{H} + B_{54}^{V}) / 2

where B₅₄ ^H is estimated from pixels I₅₃, I₅₄, and I₅₅ with the method in the Eq. (4), and B₅₄ ^V is determined similarly from pixels I₄₄, I₅₄, and I₆₄.

4. Results

In our experiments, all test images are sampled with the Bayer CFA pattern and then reconstructed using demosaicking methods under comparisons in RGB color space.

In Fig. 2, we display the results of the Hamilton method,³ the Gunturk method,² bilinear interpolation, and our method on a real color image. For greater clarity, we highlight a patch in the image and zoom in to obtain a larger scale. Bilinear interpolation produces many “confetti” types of artifacts. Fringe artifacts, also known as zipper artifacts, are obvious in the results of the Gunturk method. For this image, the Hamilton method performs as well as our method, both having much fewer artifacts.

Fig. 2

Demosaicked results by some current demosaicking methods and ours on a real image.

More than 50 real images were tested in our experiments, and we found our method to be less susceptible to edge artifacts than these selected state-of-the-art demosaicking methods² ³ ⁴ in most cases. At the same time, our method reasonably preserves edge details. Some of the test images and demosaicking results are available on our webpage.⁶

Acknowledgments

This work was done while the first author was an intern at Microsoft Research Asia.

REFERENCES

1.

B. E. Bayer, “Color imaging array,” U.S. Patent No. 3971065 (1976).

2.

B. K. Gunturk , Y. Altunbasak , and R. M. Mersereau , “Color plane interpolation using alternating projections,” IEEE Trans. Image Process. , 11 (9), 997 –1013 (2002). Google Scholar

3.

J. F. Hamilton and J. E. Adams, “Adaptive color plane interpolation in single sensor color electronic camera,” U.S. Patent No. 5629734 (1997).

4.

R. Ramanath , W. E. Snyder , and G. L. Bilbro , “Demosaicking methods for Bayer color arrays,” J. Electron. Imaging , 11 (3), 306 –315 (2002). Google Scholar

5.

S. Lin, J. Gu, S. Yamazaki, and H.-Y. Shum, “Radiometric calibration from a single image,” Proc. IEEE Computer Society, Conference on Computer Vision and Pattern Recognition, pp. 938–945 (2004).

6.

www.pami.sjtu.edu.cn/people/yjzheng/research-Demosaicking.htm

Citation Download Citation

Yuanjie Zheng, Stephen Lin, and Jie Yang "Color filter array demosaicking with local color distribution linearity," Optical Engineering 44(5), 050501 (1 May 2005). https://doi.org/10.1117/1.1906084

Published: 1 May 2005

JOURNAL ARTICLE
2 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY