1.

Introduction

In many scientific applications, images are often corrupted by noise because of either data acquisition or data transmission. Therefore, the problem of recovering an original image from noisy data has received ever-increasing attention in recent years, but it is still very challenging.¹ It has been popular to denoise an image by Markov random field (MRF)-based methods. Recently, a new fields of experts (FoE)-based approach² was introduced that considered high-order MRF cliques to grasp the complex structural information in image data and utilized a maximum-a-posteriori (MAP) framework to get the clean image. It is one of the most promising methods in generic MRF-based image denoising approaches.³

The typical objective in denoising problems is a recovery of an image with minimal mean-squared error (MSE). However, in practice, we do not have the original image to compare to, and thus we cannot know what choice of parameters minimizes the MSE. Therefore, the parameters are often tuned manually by looking at the reconstructed result.

In FoE image denoising algorithms, the tradeoff parameter and stopping criterion must be specified before iteration. However, tuning of the parameters is not an easy task. Assuming that the noise level is known a priori, Roth and Black adopted a fixed number of iterations and experientially determined the appropriate tradeoff parameter between the likelihood term and the FoE prior term using a complicate training procedure. But in real image denoising, accurate noise variance estimation is unavailable, even though there are many types of noise variance estimate algorithms.⁴

In this work, we have developed an automatic blind method in FoE real image denoising, which can take effect without accurate noise variance estimation and automatically terminate at a good result.

2.

Brief Review of Field of Experts Image Denoising

The basic problem of image denoising is the recovery of a latent clear image $X$ from an observed noisy image $Y:Y=HX+N$ , where $N$ represents additive noise, assumed to be white Gaussian noise (WGN) with zero mean and known standard deviation $\sigma$ . Using MAP rules to estimate the clear image $X$ , we can maximize the posterior probability ${\widehat{X}}_{\mathrm{MAP}}=\mathrm{Arg}\underset{X}{\max }P\left(X\right|Y)=\mathrm{Arg}\underset{X}{\max }P\left(Y\right|X)P\left(X\right)$ , in which the conditional possibility is

3.

Parameter Selection in Roth/Field-of-Experts

In Roth-FoE, two important parameters, the tradeoff parameter $\lambda$ and stopping criterion, have an important impact on the final image quality and its computational complexity, so they must be specified before iteration begins. However, tuning of the parameters is not an easy task. In computer vision tasks, two stopping criteria were widely used: a fixed number of iterations or a predefined threshold, which the difference between adjacent iterations becoming smaller. However, these two strategies both have to choose the threshold or the number of iterations by experience, so they cannot be done completely automatically for all images.

Assuming a known noise distribution in synthetic image denoising, Roth experimentally determined the appropriate $\lambda$ using a complicated training procedure with synthetic data and specified a fixed number of iterations. The Roth-FoE can generate satisfied results in the synthetic experiment with known noise variance.

We simulated noisy data by adding WGN to the same test image Castle from the Berkeley segmentation dataset, as in Roth’s work. The true noise level we added was $\sigma =15$ , and the noise levels while using Roth-FoE are four different hypotheses with $\widehat{\sigma }∊\{10,15,30,75\}$ . Other parameters were selected as recommended in Ref. 2. The peak signal-to-noise ratio (PSNR) curves of four experiences with 5000 iterations are given in Fig. 1. It shows that the noise variance estimation is vital in the performance of the final results using the stopping criterion for fixed numbers of iterations. However, if we can find an effective automatic stopping criterion to stop the algorithm at optimal times, such as using $\widehat{\sigma }∊\{30,75\}$ , we can get the satisfied results, or even better than the results by using the perfect noise variance estimation with $\widehat{\sigma }=15$ . Unfortunately, in real image denoising, there is no ground truth and the accurate noise variance is unavailable, even though there are many types of noise variance estimate algorithms.⁴ So the Roth-FoE is restricted to simulated experiments, and cannot be directly applied to real image denoising applications.

Fig. 1

Denoising performance of noisy image $(\sigma =15)$ using the $5\times 5$ FoE models with varying noise level $(\widehat{\sigma }∊\{10,15,30,75\})$ shown in terms of PSNR.

4.

Automatic Stopping Criterion

According to Eq. 3, Roth-FoE can be understood as a regularization algorithm in inverse problems, and $\lambda ∕{\sigma }^2$ can be considered as the regularization factor to balance the impact between the two terms: a data fidelity term that measures the likelihood of the input image given the output, and a prior term that encodes prior assumptions about the output. In this work, motivated by Fig. 1, we considered the parameter $\lambda ∕{\sigma }^2$ as fixed and adopted a no-reference image quality assessment based on singular value decomposition (SVD) to design an optimal automatic stopping criterion for FoE-based image denoising.

Considering an $n\times n$ window ${\omega }_k$ at point $(i,j)$ of image $X$ , the gradient matrix is defined as

5.

Experimental Results

We design an experimental scheme to illustrate the image quality performance of the proposed method on both synthetic and real images. For the synthetic data, we simulated noisy data by adding WGN $(\sigma =30)$ to the test image Castle. We denoised it by Roth-FoE and the proposed ASC-FoE under seven different noise-level hypotheses with $\widehat{\sigma }∊\{25,30,35,40,50,75,100\}$ . We used the $5\times 5$ FoE model with 24 filters, and other parameters were selected as recommend in Ref. 2. Table 1 shows the performance comparison in terms of PSNR and structural similarity (SSIM) index as measured between the denoised image and the ground truth. Also, the iteration numbers are given out. It can be seen that the ASC-FoE outperforms the Roth-FoE and has less computational cost. It should be noted that, for fair comparison of the two methods in the synthetic ASC-FoE experiment, we used six different noise level hypotheses the same as the Roth-FoE instead of setting $\lambda ∕{\sigma }^2=0.001$ , as described in ASC-FoE. In addition, using $\lambda ∕{\sigma }^2=0.001$ is almost equal to using $\widehat{\sigma }=75$ , and the result is also satisfied.

Table 1

PSNR, SSIM, and iteration number for Roth-FoE and ASC-FoE.

For the real noise image, the experimental results are shown in Fig. 2. We use the test image JFK $(367\times 343)$ ⁷ that suffers from the real noise shown in Fig. 2. The noise comes from film grain, scanning, and compression processes, and may not be pure Gaussian—indeed, it may be space variant. We estimated the standard deviation of the noise through the commonly used median absolute deviation (MAD) method for Roth-FoE. The measured value is $\sigma =4.2$ . The results of Roth-FoE and ASC-FoE are shown in Figs. 2 and 2, respectively. It can be seen that the ASC-FoE outperforms the Roth-FoE in noise-suppressing capability, and obtains a more pleasant result. Moreover, the Roth-FoE requires 5000 iterations according to recommended parameter settings, while in this experiment we terminated the ASC-FoE at iteration 208 when the image quality achieved the best. Therefore, the iteration time reduction of ASC-FoE scores around 96% and would result in a speedup factor of more than 24.

Fig. 2

Denoised images and partially magnified images of Roth-FoE and ASC-FoE compared in this experiment on an image with real noise. (a) Real noisy image. (b) Roth-FoE $(Q=27.3598)$ . (c) Proposed ASC-FoE $(Q=28.5272)$ .

6.

Conclusion

For real image denoising applications, we develop an automatic stopping criterion in FoE image denoising. We demonstrate that our ASC-FoE can obtain a faster and more pleasant result than the original Roth-FoE without the explicit need to know the noise variance a priori.