## 1.

## Introduction

Image noises are difficult to model due to the limitation of the imaging system which makes the image restoration not an easy task. In many scientific applications, a robust filter is desirable and crucial for the image pre-processing. Moreover, in some special applications, for example in medical diagnosis, the visibility of structure details is also very important, so the destination of denoising is not only to pursue a higher signal-to-noise ratio, but also to produce a satisfying visual quality.

A number of denoising algorithms have been presented, such as statistical and diffusion-based filters,^{1, 2} that restore the image through calculating local statistics or gradient information. The nonlocal means (NL-means)^{3} indicates that each pixel is restored as an average weighted by the intensity similarity of neighboring pixels. The block-wise NL-means approach^{4} based on a Bayesian framework is very efficient in reducing speckle noises while preserving edges. The mean shift (MS) method^{5} weighting the neighboring pixels uses a kernel comprised of a spatial component and an intensity component. The spatial component makes it more robust than replacing a pixel's value with a simple weighted average of intensity. In this paper, we propose a nonlocal oriented method which is more robust than the classical MS and NL-means in smoothing homogeneous areas while preserving tiny textures.

## 2.

## Related Works

The weights of MS filters are formatted with two components:^{5}

## Eq. 1

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{equation} K_{h_s,h_r } (x) = Ck\left( \left\| {\frac{{x^s }}{{h_s }}} \right\|^2 \right)k\left(\left\| {\frac{{x^r }}{{h_r }}} \right\|^2 \right).\end{equation}\end{document} $${K}_{{h}_{s},{h}_{r}}\left(x\right)=Ck\left({\Vert \frac{{x}^{s}}{{h}_{s}}\Vert}^{2}\right)k\left({\Vert \frac{{x}^{r}}{{h}_{r}}\Vert}^{2}\right).$$The NL-means^{3} estimates the pixel *i*with a weighted average of all the pixels,

## Eq. 2

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{equation} NL[v](i) = \sum_{j \in I} {w(i,j)v(j)}, \end{equation}\end{document} $$NL\left[v\right]\left(i\right)=\sum _{j\in I}w(i,j)v\left(j\right),$$## Eq. 3

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{equation} w(i,j) = Ce^{ - \frac{{\| {N_i - N_j }\|_{2,a} ^2 }}{{h^2 }}}. \end{equation}\end{document} $$w(i,j)=C{e}^{-\frac{\parallel {N}_{i}-{N}_{j}{\parallel}_{2,a}^{2}}{{h}^{2}}}.$$*w*(

*i*,

*j*) depends on the similarity of the intensity gray level vectors

*v*(

*N*

_{i}) and

*v*(

*N*

_{j}).

*N*

_{k}denotes a square neighborhood centered at a pixel

*k*. The block-wise method

^{4}restores the center block instead of the center pixel and the final restored intensity of the pixel is the mean of the current restored block. This fact allows a more robust comparison than local smoothing filters.

## 3.

## Nonlocal Oriented Weights Estimator

Based on NL-means, we propose the nonlocal oriented mean shift filter (NL-OMS). The weights estimator is the improved MS kernel and computed as a weighted value of all the image blocks. This method can be considered as a block-wise NL-means with variable bandwidths adapted to local features. The current block is recovered as,

## Eq. 4

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{equation} {\rm NL\!-\! OMS}(u)(B_i) = \sum_{j \in I} {K(B_j,B_i)u(B_i)},\end{equation}\end{document} $$\mathrm{NL}-\mathrm{OMS}\left(u\right)\left({B}_{i}\right)=\sum _{j\in I}K({B}_{j},{B}_{i})u\left({B}_{i}\right),$$## Eq. 5

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{equation} K(B_j,B_i) = Ce^{ - |j - i|^2/h_1 ^2} e^{ - || {u(B_j) - u(B_i)}||_2^2 / h_2 ^2}, \end{equation}\end{document} $$K({B}_{j},{B}_{i})=C{e}^{{-|j-i|}^{2}/{h}_{1}^{2}}{e}^{-\left|\right|u\left({B}_{j}\right)-u\left({B}_{i}\right){\left|\right|}_{2}^{2}/{h}_{2}^{2}},$$## Eq. 6

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{equation} {\rm NL\hbox{-}OMS}(u)(x_i) = \overrightarrow G (B_i)*{\rm NL\hbox{-}OMS}(u)(B_i), \end{equation}\end{document} $$\mathrm{NL}\text{-}\mathrm{OMS}\left(u\right)\left({x}_{i}\right)=\stackrel{\u20d7}{G}\left({B}_{i}\right)*\mathrm{NL}\text{-}\mathrm{OMS}\left(u\right)\left({B}_{i}\right),$$*u*(

*B*

_{i}) = [

*u*

^{(1)}(

*B*

_{i}), …,

*u*

^{(p)}(

*B*

_{i}), …]

^{T}is the intensity vector of the block

*B*

_{i},

*i*,

*p*∈

*Z*.

*h*

_{1}and

*h*

_{2}control the decay of the exponential function. [TeX:] $\overrightarrow G (B_i)$ $\stackrel{\u20d7}{G}\left({B}_{i}\right)$ is an oriented filter which is determined by eigenvectors with eigenvalue decomposition of the local Hessian matrix.

^{2}The proposed weights estimator considers not only the local texture information but also the spatial relationship. Any image block with a similar gray level and a short spatial distance to the center block gives a large weight. The center pixel is restored by a convolution between [TeX:] $\overrightarrow G (B_i)$ $\stackrel{\u20d7}{G}\left({B}_{i}\right)$ and the current block, and the size of each block is determined by local variation.

^{4}NL-OMS needs few parameters to tune. We simply set the image block as the bounding box of [TeX:] $\overrightarrow G (B_i)$ $\stackrel{\u20d7}{G}\left({B}_{i}\right)$ in the following experiments. Each block may have variable bandwidths, so we redefine the

*L*

_{2}-norm as a generalized distance, only considering the pixels in the intersection of two blocks

*B*

_{i}and

*B*

_{j},

## Eq. 7

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{equation} \| {u(B_j) - u(B_i)} \|_2^2 \equiv \sum\limits_{p \in B_j \cap B_i } {[u^{(p)} (B_j) - u^{(p)} (B_i)]} ^2\vadjust{\pagebreak} \end{equation}\end{document} $$\parallel u\left({B}_{j}\right)-u\left({B}_{i}\right){\parallel}_{2}^{2}\equiv \sum _{p\in {B}_{j}\cap {B}_{i}}{[{u}^{\left(p\right)}\left({B}_{j}\right)-{u}^{\left(p\right)}\left({B}_{i}\right)]}^{2}$$^{6}which automatically discovers the statistical properties of the signal by decreasing the joint entropy of neighborhoods.

## 4.

## Experiments and Validation

We assume a speckle noise model^{4} to perform the simulation. In the experiment on textured data (Fig. 1), the “roof” image is used as the ground truth to test the performance of preserving textures with different methods. The noisy image is obtained by adding speckle noise under three different noise-level hypotheses with σ ∈ {15, 30, 50} and other parameters are selected as recommend in Refs. 4 and 5. The bottom row demonstrates the results processed with NL-means and NL-OMS respectively. Figure 1 is more similar to the ground truth visually than Fig. 1 and the edges are significantly protected. Table 1 lists the comparison in terms of PSNR (peak signal-to-noise ratio) index including MS and nondenoising. The numeral results objectively show the performance of the proposed method in denoising. Figure 2 is a real cone-beam computer tomography (CBCT) image. There are distinct speckle noises in the homogeneous region and the structures are polluted by granules. The noise level is reduced with NL-means, but the important anatomical details are also blurred, as shown in Fig. 2. Using NL-OMS, it can be clearly seen that the whole image quality is improved, and the anatomical details are preserved in Fig. 2. Not considering the computational burden, it is worthy to gain quality improvement to avoid inaccuracy in diagnosis. Moreover, we will focus on promoting the efficiency of the proposed algorithm in a future work.

## Table 1

Comparison of PSNR with different methods.

PSNR | ||||
---|---|---|---|---|

σ | Non | MS | NL-means | NL-OMS |

15 | 14.52 | 16.86 | 17.67 | 19.24 |

30 | 11.88 | 16.46 | 16.84 | 17.14 |

50 | 10.28 | 15.99 | 16.05 | 16.12 |

## 5.

## Conclusions

Combining with the nonlocal means and mean-shift methods, we derive a nonlocal weights estimator for image noise reduction. The novel estimator considers not only the similarity of intensity, but also the spatial relationship between image blocks. Denoising is performed with an oriented filter adapted to the variations of local intensity. This novel method is more robust than the classical nonlocal means filter and mean-shift filter in smoothing homogeneous areas while preserving edges.

## References

**,” IEEE Trans. Pattern Anal. Mach. Intell., 7 (2), 165 –177 (1985). https://doi.org/10.1109/TPAMI.1985.4767641 Google Scholar**

*Adaptive noise smoothing filter for images with signal dependent noise***,” IEEE Trans. Image Processing, 16 1412 –1424 (2007). https://doi.org/10.1109/TIP.2007.891803 Google Scholar**

*Oriented speckle reducing anisotropic diffusion***,” 60 –65 (2005). https://doi.org/10.1109/CVPR.2005.38 Google Scholar**

*A nonlocal algorithm for image denoising***,” IEEE Trans. Image Processing, 18 (10), 2221 –2229 (2009). https://doi.org/10.1109/TIP.2009.2024064 Google Scholar**

*Nonlocal means-based speckle filtering for ultrasound images***,” IEEE Trans. Pattern Anal. Mach. Intell., 24 (5), 603 –619 (2002). https://doi.org/10.1109/34.1000236 Google Scholar**

*Mean-shift: A robust approach toward feature space analysis***,” IEEE Trans. Pattern Anal. Mach. Intell, 28 (3), 364 –376 (2006). https://doi.org/10.1109/TPAMI.2006.64 Google Scholar**

*Unsupervised, information-theoretic, adaptive image filtering for image restoration*