1 April 2009 Visual-oriented morphological foreground content grayscale frames interpolation method
Author Affiliations +
Abstract
A new and improved visual-oriented grayscale frames interpolation method consists of partially changing, step by step using growing structuring elements, the morphological S transforms of the foreground content of an input frame with the morphological S transforms of the foreground content of an output frame. Better performance comes at the expense of not very great computational complexity. Computer simulations illustrate the results.
Vizireanu and Udrea: Visual-oriented morphological foreground content grayscale frames interpolation method

1.

Introduction

The grayscale frames interpolation problem is the process of creating intermediary grayscale frames between two given frames.1 The interpolated grayscale frame (Xi) , 0iM , M2 depends on the content of an input grayscale frame and an output grayscale frame. For i=0 , the interpolated grayscale frame is equal to the input grayscale frame X0 ; for i=M , the interpolated grayscale frame is equal to the output grayscale frame XM .2

The simplest method involves linear interpolation.3, 4 In Ref. 5, a dynamically elastic surface interpolation scheme was proposed. A hybrid approach that combined elastic interpolation, spline theory, and the surface consistency theorem was proposed to produce further improvement.6, 7 Grevera and Udupa8 and Herman 9 interpolated the distance and proposed shape-based methods by encoding the segmented image with distance codes. Guo developed a morphology-based interpolation method and successfully resolved the problems in objects with holes and large offsets.10 Lee proposed another morphology-based scheme that was simpler in computational complexity.11 Goshtasby selected feature points from successive frames to control the gray-level interpolation.12 In Ref. 13, feature points were used that were based on a fuzzy measure of the boundaries and medial axis transforms. In Ref. 14, a morphological skeleton interpolation was proposed in which interpolated sets were generated from a succession of skeletons derived from the matching of two neighboring set skeletons. Berier and Neely computed the interpolation with feature line-segment control.15 An example of object-based methods that used features is shape-based interpolation,16 which was extended by allowing registration between frames17 and using feature guidance.18 For grayscale images, mathematical morphology provides a well-founded theory.19

The new interpolation grayscale frame method presented in this paper is based on the morphological S transform (MST) and consists of partially changing, step by step using growing structuring elements (SEs) the morphological S transforms of the foreground content of an input grayscale frame with the MSTs of the foreground content of an output grayscale frame. Better performance comes at the expense of not very great computational complexity.

This paper is organized as follows. Section 2 defines the grayscale MST. Section 3 presents the grayscale frames interpolation algorithm. Section 4 provides experimental results. The conclusions of this study are drawn in Sec. 5.

2.

Morphological S Transform (MST)

The MST of a grayscale image X can be calculated by means of morphological operations.19 For any SE B , we have

1

(n+1)B=nBB,n=0,1,2,and0B=(0,0).

The morphological S(X,nB) transform of a grayscale image X by SE nB is given by

2

S(X,nB)=XnB(XnB)B.

A grayscale image can be represented using the morphological formula

3

X=(S[X,(N2)B]+{S[X,(N1)B]+S(X,NB)B}B)B,
where N is the maximum value defined by

4

S(X,nB)=,n>NandS(X,NB).

3.

Interpolation Method

The new and improved grayscale frame interpolation method presented in this paper consists of partially changing, step by step using growing SEs nB for n=N,,1,0 with 0B=(0,0) , the MST of the foreground content of a grayscale input frame with the MST of the foreground content of an output grayscale frame.

Based on the grayscale input frame X0 and grayscale output frame XM , the foreground contents are defined as

5

X0f=X0.*sign(X0XM)andXMf=XM.*sign(X0XM),
where

6

sign(x)={1,ifx00,ifx=0}.
Using the morphological S transform and SE B , the foreground content X0f of input grayscale frame X0 and the foreground content XMf of output grayscale frame XM can be perfectly reconstructed using 3.

The interpolated foreground content Xif of grayscale interpolated frame Xi , for 1iN and N2 for N=max(N0,NM) , obtained by using the morphological formula

7

Xif(X0f,XMf,B)=(S[X0f,(Ni)B]++{S[XMf,(N1)B]+S(XMf,NB)B}B)BB
for 1imin(N0,NM)1 . Finally, the grayscale interpolated frame Xi is given by

8

Xi(X0,XM,B)=[X0X0.*sign(X0XM)]+Xif

Using morphological formula 7, we can build a large number, min(N0,NM) , of not necessarily different interpolated grayscale frames.

4.

Experimental Results

We used the proposed algorithm for grayscale frame interpolation by constructing a frame using growing SEs and different i values. We performed the experiments using grayscale images.

An example is presented in Fig. 1 . Test input and output grayscale frames are presented in Figs. 1 and 1, and the “missing” frame Xref is in Fig. 1. This frame will be used as reference to compare our results. Our frames have dimensions [256, 256].

Fig. 1

Input, output, and reference grayscale frames for the interpolation example: (a) grayscale input frame Xi , (b) grayscale output frame XM , and (c) grayscale reference frame Xref .

020502_1_1.jpg

Using formula 5, we obtained the foreground contents for input and output frame X0f and XMf for input and output frame X0 and XM [Figs. 2 and 2 ]. The background content for both the input and output frames (content the frames have in common), Xb [Fig. 2], is given by

9

Xb=X0X0.*sign(X0XM).

Fig. 2

Foreground and background contents for the input frame and output frames: (a) foreground contents for input frame X0f , (b) foreground contents for output frame XMf , and (c) background content of both frames (content the frames have in common) Xb .

020502_1_2.jpg

Using the morphological S transform and crosses or squares as SE B , we obtained N0c=38 , NMc=38 , and Nc=38 for the crosses and N0s=30 , NMs=30 , and Ns=30 for the squares. Interpolated foreground content Xif , with 1i38 (for the crosses) and 1i30 (for the squares), was obtained using morphological formula 7. Finally, grayscale interpolated frame Xi was given by formula 8. For the crosses, we built 34 interpolated grayscale frames, but only 21 were different. For the squares, we built 30 interpolated grayscale frames, but only 17 were different. Using a standard computer and an optimized C routine implementation, we obtained a grayscale interpolated frame at 40ms . For size(Xref)=[AB] , we can define error function ri by

10

ri=sum(sum{sign[abs(XrefXi)]})(A*B).

Some error function results for the crosses (ric) and squares (ris) SEs are presented in Tables 1, 2 .

Table 1

Error function ric .

i 468101214161820
ric (%)17.6719.8923.1118.4516.0717.3422.1521.8316.95

Table 2

Error function ris .

i 468101214161820
ris (%)19.2321.2817.2515.8718.4819.6225.2322.3723.74

Interpolated frames using different SEs are presented in Figs. 3 and 4 .

Fig. 3

Interpolated frame ( Xi , i=12 ) using crosses as the SE.

020502_1_3.jpg

Fig. 4

Interpolated frame ( Xi , i=10 ) using squares as the SE.

020502_1_4.jpg

After simulations using different SEs and different i values, we observed that one of the interpolated frames offered by our method was similar to the original reference frame.

5.

Conclusions

This paper has addressed the grayscale frames generalized interpolation by means of mathematical morphology.

After describing the algorithm, we provided experimental results that were very encouraging. We applied the proposed scheme to real-world data. For many situations, the proposed scheme was experimentally shown to successfully resolve complex interpolation problems. Some preprocessing could improve the results. The algorithm is fully morphological and can be applied quickly. This entire process is efficient without significant computational complexity. However, it is difficult to state rigorously that our method produces theoretically correct interpolated results for any frames.

Acknowledgments

This research was founded by UEFISCSU, contract PII, IDEI, no. 100/2007, D.N. VIZIREANU.

references

1.  M. Iwanowski and J. Serra, “Morphological interpolation and color images,” in Proc. of Intern. Conf. on Image Proc., Venice, Italy (1999). Google Scholar

2.  A.-M. Huang and T. Q. Nguyen, “A multistage motion vector processing method for motion-compensated frame interpolation,” IEEE Trans. Image Process. 17(5), 694–708 (2008). 10.1109/TIP.2008.919360 Google Scholar

3.  C. C. Liang, C. T. Chen, and W. C. Lin, “Intensity interpolation for reconstructing 3-D images from serial cross-sections,” in Proc. IEEE Eng. Med. Bio. Soc. 10th Int. Conf., New Orleans, LA, Paper CH2566-8, pp. 1389–1390 (1988). Google Scholar

4.  C. C. Liang, C. T. Chen, and W. C. Lin “Intensity interpolation for branching in reconstructing three-dimensional objects from serial cross-sections,” Proc. SPIE 1445, 456-467 (1991). 10.1117/12.45242 Google Scholar

5.  W.-C. Lin, C.-C. Liang, and C.-T. Chen, “Dynamic elastic interpolation for 3-D image reconstruction from cross sections,” IEEE Trans. Med. Imaging 7(9), 225–232 (1988). 10.1109/42.7786 Google Scholar

6.  S.-Y. Chen and W.-C. Lin, “Automated surface interpolation technique for 3-D object reconstruction from serial cross sections,” Comput. Med. Imaging Graph. 15(4), 265–276 (1991). 10.1016/0895-6111(91)90085-A Google Scholar

7.  Y.-H. Liu, Y.-N. Sun, C.-W. Mao, and C.-J. Lin, “Edge-shrinking interpolation for images,” Comput. Vis. Graph. Image Process. 21(2), 91–101 (1997). Google Scholar

8.  G. J. Grevera and J. K. Udupa, “Shape-based interpolation of multi-dimensional grey-level images,” IEEE Trans. Med. Imaging 15(12) 881–892 (1996). 10.1109/42.544506 Google Scholar

9.  G. T. Herman, J. Zheng, and C. A. Bucholtz, “Shape-based interpolation,” IEEE Comput. Graphics Appl. 12, 69–79 (May 1992). 10.1109/38.135915 Google Scholar

10.  J.-F. Guo, Y.-L. Cai, and Y.-P. Wang, “Morphology-based interpolation for 3D image reconstruction,” Comput. Med. Imaging Graph. 19(3), 267–279 (1995). 10.1016/0895-6111(95)00007-D Google Scholar

11.  T.-Y. Lee and W.-H. Wang, “Morphology-based three-dimensional interpolation,” IEEE Trans. Med. Imaging 19(7), 711–721 (2000). 10.1109/42.875193 Google Scholar

12.  A. Goshtasby, D. A. Tuner, and L. V. Ackerman, “Matching tomographic slices for interpolation,” IEEE Trans. Med. Imaging 11(4), 507–516 (1992). 10.1109/42.192686 Google Scholar

13.  D. T. Puff, D. Eberly, and S. M. Pizer, “Object-based interpolation via cores,” Proc. SPIE 2167, 143–150 (1994). 10.1117/12.175049 Google Scholar

14.  V. Chatzis and I. Pitas, “Interpolation of 3-D binary images based on morphological skeletonization,” IEEE Trans. Med. Imaging 19(7), 699–710 (2000). 10.1109/42.875192 Google Scholar

15.  T. Berier and S. Neely, “Feature-based image metamorphosis,” in Proc. Comput. Graphics (SIGGRAPH’92) 26(2), 35–42 (1992). Google Scholar

16.  S. P. Raya and J. K. Udupa, “Shape-based interpolation of multidimensional objects,” IEEE Trans. Med. Imaging 9(3), 32–42 (1990). 10.1109/42.52980 Google Scholar

17.  W. E. Higgins, C. Morice, and E. L. Ritman, “Shape-based interpolation of tree-like structures in three-dimensional images,” IEEE Trans. Med. Imaging 12(9), 439–450 (1993). 10.1109/42.241871 Google Scholar

18.  T.-Y. Lee and C.-H. Lin, “Feature-guided shape-based image interpolation,” IEEE Trans. Med. Imaging 21(12), 1479–1489 (2002). 10.1109/TMI.2002.806574 Google Scholar

19.  E. R. Dougherty and R. A. Lotufo, Hands-on Morphological Image Processing, SPIE Press, Bellingham, WA (2003). Google Scholar

© (2009) Society of Photo-Optical Instrumentation Engineers (SPIE)
Nicolae Dragos Vizireanu, Radu Udrea, "Visual-oriented morphological foreground content grayscale frames interpolation method," Journal of Electronic Imaging 18(2), 020502 (1 April 2009). https://doi.org/10.1117/1.3134142 . Submission:
JOURNAL ARTICLE
3 PAGES


SHARE
Back to Top