A common feature of wavelet domain image resolution upscaling algorithms is the assumption that the image to be enhanced is the low-pass filtered subband of a wavelet-transformed high-resolution (HR) image. Then the detail wavelet coefficients in subbands containing high-pass spatial frequency information are estimated and the HR image is obtained by inverse wavelet transform. In Ref. 1, detail coefficients are estimated using the evolution of wavelet transform extrema in coarser subbands. A similar but less computationally expensive approach is advocated in Ref. 2. In Ref. 3 a technique is proposed that takes into account the hidden Markov tree (HMT) approach. The HMT-based methods have been further developed not to require any training data set.4
The decimated wavelet transform is not shift-invariant and, as a result, inaccurate representation of wavelet coefficients—for example, due to quantization or non-exact estimation—introduces cyclostationarity into the image, which manifests itself as ringing in the neighborhood of discontinuities. Cycle-spinning (CS), introduced in Ref. 5, has been shown to be an effective method against ringing when used for denoising purposes. In Refs. 6, 7, it was shown that CS, applied as a postprocessing operation, yields improvements in the framework of JPEG and JPEG2000. Recently, it has been shown that it is also a powerful alternative to competing methods, when applied to image resolution upscaling problems in the wavelet domain.8
Since its introduction, the basic CS algorithm has hardly evolved apart from an iterative variant in Ref. 9. This involves iterated applications of a processing chain consisting of forward translation, wavelet denoising, and backward translation. Although this scheme was shown to improve upon the basic CS in wavelet denoising problems, it doesn’t lend itself to wavelet domain resolution upscaling. In the resolution upscaling context, the upscaled image, which is the output of the previous iteration, would need to be downsampled before being fed as input into the next, nullifying the effects of the preceding iteration.
We introduce the concept of directional cycle spinning. In particular we take into account local edge orientation information derived from a wavelet decomposition of the available low-resolution (LR) image to influence key parameters of the CS algorithm.
Image Resolution Upscaling Using Cycle Spinning
Image resolution upscaling involving CS typically consists of two steps, namely, wavelet-domain zero padding (WZP) followed by the application of a CS operator.
In the first step, an initial approximation to the unknown HR image is generated using WZP: Using the available LR image of size , the unknown HR image is reconstructed by using zero padding of high-frequency (HF) subbands followed by inverse wavelet transform:is an all-zero matrix of size and is the inverse discrete wavelet transform. This implies a simplified image formation process whereby the point spread function is associated with the low-pass wavelet filter kernel used for the above transformation. The HR approximation obtained as above commonly exhibits artifacts such as smoothing and ringing. The ringing emerges at the vicinity of discontinuities as alternating undershoots and overshoots of the intensity level. CS has been shown to be an effective means to reduce these artifacts by averaging out the translation dependence.5 For the image resolution upscaling purposes, a variant of the CS methodology is applied as a second step as follows:8 First a number of LR images are generated from by (1) spatial shifting, (2) wavelet transformation, and (3) discarding the HF coefficients: where represents discarding of HF coefficients, denotes wavelet transform, and is a shift operator applying horizontal and vertical shifts of in the range . Then, Eq. 1 is applied to all yielding images, where . Finally, these intermediate HR images are re-aligned and averaged to give the final HR reconstructed image: where is the inverse of the shifting operator .
Directional Cycle Spinning
In conventional CS as described above, all possible shifts within a range that defines a local neighborhood are used. Nevertheless, a closer inspection of artifacts in a typical HR image generated using WZP reveals that ringing artifacts not only occur in the vicinity of strong edges but, more importantly, they are strongly correlated with the orientation of those edges. In particular, for an edge of a given orientation, ringing is more pronounced in the normal direction. This is due to the fact that ringing artifacts materialize as a result of undershoots and overshoots of sharp intensity level changes. This observation suggests that CS should be predominantly applied across edges (i.e., vertically in the vicinity of a horizontal edge and vice versa) avoiding other orientations and thus preventing unnecessary smoothing of image detail.
To test this conjecture, we have used the synthetic image shown in Fig. 1 . We consider this synthetic image as the ground truth HR image we seek to reconstruct. Then we generate an LR version of it by low-pass filtering followed by downsampling. Finally we obtain approximations to the original HR image using WZP followed by CS applied at different orientations: nondirectional (NCS), horizontal (HCS), vertical (VCS), or diagonal (DCS). Figure 2 indicates the patterns used for CS (shift locations used are shown as shaded elements) with the no shift location occupying (0,0). For example HR reconstructions using HCS and VCS are obtained respectively usingare the intermediate HR images corresponding to a shift of as explained in Sec. 2.
PSNR values of the HR reconstruction for each quadrant and for each CS orientation are shown in Fig. 1 while the best value is shown in boldface. Our results confirm that the best reconstruction always occurs by CS in the direction of the normal to dominant edges while in the absence of such edges nondirectional CS is preferable. The gain is more pronounced in HCS and VCS, while DCS brings a more modest improvement.
Our approach uses a local estimate of image directional activity to influence the orientation in which CS is applied. This estimate is obtained by using a wavelet decomposition of the available LR image .
Let and denote respectively the HF horizontal and vertical detail subbands of and and denote the wavelet coefficients at position of and respectively. We partition into nonoverlapping blocks and estimate horizontal and vertical activity measures for the ’th block as and respectively; summation is over all wavelet coefficients contained in block . Finally, block in the HR image is reconstructed by using these activity values as weights to the corresponding reconstructions generated using Eq. 2:
The proposed method has been tested on a number of well-known test images. An HR version of these images was used as ground truth for performance evaluation purposes. The wavelet transform was implemented using the well-known Daubechies filters, the maximum shift was set to 5, and block sizes were . The proposed method has been compared with other wavelet based methods as well as an edge directed10 and a fractal-based11 method. PSNR values are tabulated in Table 1 for both and enlargement factors. Figures 3 and 4 respectively show the enlarged images and amplified residual images for subjective comparisons with bilinear interpolation and WZP. Figure 5 shows a cross section of an edge (taken from Fig. 1) and reconstructed versions as well as the absolute error of these reconstructions.
PSNR (dB) values for 2× (256×256to512×512) and 4× (128×128to512×512) enlargement.
|WZP and CS8||34.93||29.27||33.56||30.78||24.28||21.54||34.32||29.87|
|WZP and Directional CS||35.09||29.55||33.73||30.98||24.37||21.67||34.50||30.14|
It can be seen that directional CS adapts better to edge orientation and avoids jagged edge (staircase) artifacts. Overall our results show that the directional CS outperforms the competing methods and offers modest but consistent improvements over baseline CS. Additionally, the proposed scheme offers a reduction in computational complexity relative to conventional CS because only shifts in the normal to salient edges are required, involving WZP calculations compared with in the conventional method.
A directional variant of the CS methodology was considered for image resolution upscaling in the wavelet domain. The proposed method estimates local edge orientation from a wavelet decomposition of the available LR image and uses this information to influence CS parameters. Our experimental results confirm that the proposed method outperforms competing methods for a wide range of images both in objective and subjective terms.