## 1.

## Introduction

An image captured by an imaging system represents the degraded version of an original image due to blurring and additive noise.^{1, 2} For a size
$M\times N$
image, a typical degradation model can be written as

Regularized constrained least squares (RCLS) have been widely used to obtain a solution for Eq. 1, and the solution is obtained by minimizing the following function with respect to
$\mathbf{x}$
,^{1, 2}

## Eq. 2

$$M\left(\mathbf{x}\right)={\Vert \mathbf{y}-\mathbf{H}\mathbf{x}\Vert}^{2}+\alpha {\Vert \mathbf{C}\mathbf{x}\Vert}^{2},$$The prior knowledge used in RCLS is that the original image is smooth. However, this is a global requirement and therefore not effective in terms of local smoothness. Also, when an iterative technique is used to obtain the solution of Eq. 2, the iterative solution may suffer from noise amplification after a certain iteration step when the additive noise is serious.^{2, 3, 4} Therefore, a more desirable solution can be obtained by imposing reasonable constraints into the solution space of RCLS.^{3, 4, 5} Projection onto convex sets (POCS) has been widely used in many areas, since it is effective to impose nonlinear local properties into the solution space.^{3, 6}

In this work, an iterative adaptive hybrid image restoration algorithm using a local smoothing constraint is presented. We follow the formulation represented by Eq. 2 and propose to bring knowledge about local properties of the original image into the restoration process, so that prior knowledge and spatial adaptivity are incorporated in the solution. The basic motivation is to constrain local ranges of values that the restored image can take, leading to increased convergence speed of the iterative algorithm and performance improvement compared to RCLS.

This work is organized as follows. In Sec. 2, the proposed hybrid gradient-projection restoration algorithm is explained. Experimental results and conclusions are presented in Sec. 3.

## 2.

## Proposed Algorithm

The gradient iteration of the regularized smoothing functional in Eq. 2 can be written as

## Eq. 3

$${\mathbf{x}}_{k+1}={\mathbf{x}}_{k}+[{\mathbf{H}}^{\mathbf{T}}\mathbf{y}-({\mathbf{H}}^{\mathbf{T}}\mathbf{H}+\alpha {\mathbf{C}}^{\mathbf{T}}\mathbf{C}){\mathbf{x}}_{k}]=G{\mathbf{x}}_{k}.$$^{1, 2, 7}According to Ref. 7, we determine the regularization parameter by

## Eq. 4

$$\alpha \left({\mathbf{x}}_{k}\right)={\Vert \mathbf{y}-\mathbf{H}{\mathbf{x}}_{k}\Vert}^{2}\u2215(\theta -{\Vert \mathbf{C}{\mathbf{x}}_{k}\Vert}^{2}),$$## 5.

## 6.

## Eq. 7

$${\widehat{x}}_{k,W,\mathrm{max}}(i,j)={\mathrm{max}}_{(p,q)\u220a{S}_{(i,j)}}{\widehat{x}}_{k}(p,q),$$## 8.

## Eq. 9

$$B(i,j)=\frac{{\sigma}_{{\widehat{x}}_{k},\mathit{LW}}^{2}(i,j)}{{\sigma}_{{\widehat{x}}_{k},\mathit{SW}}^{2}(i,j)}\times \frac{{\widehat{x}}_{k,\mathit{SW},\mathrm{max}}(i,j)}{{m}_{{\widehat{x}}_{k},\mathit{SW}}(i,j)}.$$In general, noise amplification contributes to the problem of restoration, and therefore desirable properties such as detection of noisy pixels and the degree of local activity should be incorporated into the restoration process to effectively suppress noise amplification. It is clear how the local statistics affect
$B(i,j)$
. Let us assume that
${\widehat{x}}_{k}(i,j)$
is a noisy pixel belonging to an element of a homogeneous activity region. In such a case,
${\sigma}_{{\widehat{x}}_{k,SW}}^{2}(i,j)$
is greater than
${\sigma}_{{\widehat{x}}_{k,LW}}^{2}(i,j)$
, and therefore local smoothing is necessary to suppress the noise amplification. As the noise increases, the variance ratio decreases, leading to smaller
$B(i,j)$
. In addition, when the same noise is added to low and high activity regions, the noise within the low activity region is more visible than that within the high activity region. Therefore, tighter bounds should be applied to the noisy pixel within the low activity region than to that within the high activity region.
${\widehat{x}}_{k,SW,\mathrm{max}}(i,j)\u2215{m}_{{\widehat{x}}_{k,SW}}(i,j)$
in Eq. 9 is used as a way to represent the degree of local activity in this work. As the local activity decreases,
$B(i,j)$
decreases. A smaller
$B(i,j)$
leads to tighter bounds, while looser bounds are for a larger
$B(i,j)$
, so that the noise amplification is effectively suppressed. They are in agreement with the noise masking property in areas of high spatial activity of the human visual system.^{8}

Local information has the limit to effectively suppress noise amplification. Therefore, global information about the noise can help to effectively reduce noise amplification. For example, tighter bounds are more desirable as the additive noise increases. ${T}_{k}$ in Eq. 8 is used to incorporate global information into the restoration process, so that it can control the degree of bounds that is computed at each iteration step, such as

## Eq. 10

$${T}_{k}=\mathrm{exp}[Z\times \frac{\sum _{i}\sum _{j}\left|{x}_{k-1}(i,j)\right|}{\sum _{i}\sum _{j}\left|{n}_{k-1}(i,j)\right|}],$$The proposed algorithm is used to obtain a solution that is an element of the intersection set between a solution space using the gradient approach reflecting the global smoothing constraint, and a projection set incorporating the local smoothing constraint.

## 3.

## Experimental Results and Conclusions

The proposed adaptive hybrid image restoration algorithm is tested with various noisy blurred images and degradation, and it is compared to typical RCLS. In the set of such experiments, the
$256\times 256\phantom{\rule{0.3em}{0ex}}\text{pixels}$
Lena and Cameraman images are described here. The original images were blurred by
$7\times 7$
motion blur and Gaussian blur with variance of 5. A Gaussian-distributed noise signal was added to the blurred images. We tested the proposed algorithm for various SNRs. In addition, a 2-D Laplacian operator was used for the high-pass operator in Eq. 2.^{2} For evaluating the objective and subjective performance of the algorithm, the mean squared error (MSE) and structural similarity index (SSIM) in Ref. 9 were used, where SSIM takes a value between
$-1$
(the worst value) and 1 (the best value). Also,

## Eq. 11

$$\frac{{\Vert {\mathbf{x}}_{k+1}-{\mathbf{x}}_{k}\Vert}^{2}}{{\Vert {\mathbf{x}}_{k}\Vert}^{2}}\u2a7d{10}^{-m}$$Figures 1, 1, 1 show the degraded Lena image ( $7\times 7$ Gaussian blur with variance of 5- and $10\text{-}\mathrm{dB}$ Gaussian noise), the restored image with RCLS, and the restored image with the proposed method, respectively. For $Z=0.07$ , the proposed algorithm converges after ten iterations ( $\mathrm{MSE}=260$ and $\mathrm{SSIM}=0.677$ ), while RCLS requires 39 iterations ( $\mathrm{MSE}=493$ and $\mathrm{SSIM}=0.432$ ). The results show that RCLS leads to noise amplification in the restored image. On the other hand, the proposed hybrid algorithm effectively suppresses the noise amplification. When tighter bounds (smaller $Z$ ) are used, the convergence becomes faster. However, tighter bounds result in oversmoothed images. On the basis of our experiments, $0.05\u2a7dZ\u2a7d0.1$ is a good range with convergence speed and performance.

Figures 2 and 2 show MSE and convergence speed comparisons as a function of iteration number for Lena image when $10\text{-}\mathrm{dB}$ Gaussian noise is added. The results show that the proposed algorithm has the capability to keep a minimum mean squared error, while RCLS is very sensitive to the terminating criterion, since the noise is amplified after a certain iteration number. Also, it is verified that the proposed algorithm is faster than RCLS for all cases.

Tables 1, 2 summarize the performance comparisons (MSE, SSIM, and iteration number) at convergence for Gaussian blur and motion blur, respectively. As expected, the performance gain of the proposed algorithm increases as the additive noise increases. However, as the additive noise decreases, looser bounds are necessary to avoid oversmoothed images. As shown in Tables 1, 2, the proposed algorithm performs similarly to RCLS for additive noise less than $30\phantom{\rule{0.3em}{0ex}}\mathrm{dB}$ .

## Table 1

Performance comparisons at convergence (Lena).

Noise(dB) | Method | Gaussian blur | Motion blur | ||||
---|---|---|---|---|---|---|---|

MSE | SSIM | Iteration Number | MSE | SSIM | Iteration Number | ||

5 | RCLS | 1799 | 0.230 | 66 | 2336 | 0.204 | 66 |

Hybrid | 303 | 0.612 | 10 | 288 | 0.633 | 9 | |

10 | RCLS | 493 | 0.432 | 39 | 797 | 0.352 | 49 |

Hybrid | 260 | 0.677 | 10 | 245 | 0.703 | 9 | |

20 | RCLS | 212 | 0.647 | 64 | 241 | 0.598 | 74 |

Hybrid | 189 | 0.730 | 38 | 174 | 0.734 | 53 | |

30 | RCLS | 139 | 0.772 | 123 | 125 | 0.773 | 127 |

Hybrid | 139 | 0.772 | 123 | 125 | 0.773 | 127 |

## Table 2

Performance comparisons at convergence (Cameraman).

Noise(dB) | Method | Gaussian blur | Motion blur | ||||
---|---|---|---|---|---|---|---|

MSE | SSIM | Iteration Number | MSE | SSIM | Iteration Number | ||

5 | RCLS | 2423 | 0.161 | 64 | 3071 | 0.150 | 63 |

Hybrid | 471 | 0.495 | 9 | 458 | 0.506 | 8 | |

10 | RCLS | 691 | 0.304 | 38 | 1049 | 0.252 | 47 |

Hybrid | 409 | 0.592 | 9 | 393 | 0.605 | 8 | |

20 | RCLS | 324 | 0.490 | 66 | 350 | 0.446 | 74 |

Hybrid | 305 | 0.642 | 35 | 280 | 0.636 | 54 | |

30 | RCLS | 202 | 0.677 | 135 | 182 | 0.671 | 134 |

Hybrid | 201 | 0.678 | 133 | 181 | 0.671 | 134 |

The novelty of the proposed algorithm is that it leads to faster convergence speed, and subjectively and objectively better performance than typical RCLS without prior information of an original image by incorporating the local smoothing constraint.

We propose an iterative adaptive hybrid image restoration algorithm using a local smoothing constraint. Each pixel in an image is projected onto a local smoothing set determined by the local mean, variance, and maximum intensity value of the partially restored image. The ratio of these parameters is used to define the convex set, resulting in effective suppression of the noise amplification and faster convergence speed.

## Acknowledgments

This research was supported by the Korea Science and Engineering Foundation (KOSEF) grant funded by the Korea government (MEST) (number 2010-0000397) and the Ministry of Knowledge Economy, Korea under the Information Technology Research Center support program supervised by the National IT Industry Promotion Agency [NIPA-2010-(C1090-1011-0003)].

## References

**,” IEEE Signal Process. Mag., 14 (2), 24 –41 (1997). https://doi.org/10.1109/79.581363 1053-5888 Google Scholar**

*Digital image restoration***,” Opt. Eng., 31 (10), 2174 –2190 (1992). https://doi.org/10.1117/12.59978 0091-3286 Google Scholar**

*Robust estimation techniques in regularized image restoration***,” Opt. Eng., 41 (10), 2515 –2524 (2002). https://doi.org/10.1117/1.1503072 0091-3286 Google Scholar**

*Iterative regularized least-mean mixed-norm image restoration***,” IEEE Signal Process. Lett., 15 (3), 192 –195 (2009). https://doi.org/10.1109/LSP.2008.2012227 1070-9908 Google Scholar**

*A novel multiple image deblurring technique using fuzzy projection onto convex sets***,” IEEE Trans. Image Process., 4 (5), 594 –602 (1995). https://doi.org/10.1109/83.382494 1057-7149 Google Scholar**

*General choice of the regularization functional in regularized image restoration***,” IEEE Trans. Syst. Man Cybern., SMC-6 845 –853 (1976). https://doi.org/10.1109/TSMC.1976.4309481 0018-9472 Google Scholar**

*Image restoration based on a subject criterion***,” IEEE Trans. Image Process., 13 (4), 600 –612 (2004). https://doi.org/10.1109/TIP.2003.819861 1057-7149 Google Scholar**

*Image quality assessment: From error visibility to structural similarity*