Moran’s I for impulse noise detection and removal in color images

Abstract. An approach for impulse noise detection and removal in color images based on Moran’s I (MI) statistic is proposed. The proposed method consists of detection and removal components and is called Moran’s I vector median filter (MIVMF). The detection module is able to determine if a pixel is noise or noise-free. If it is a noise pixel, the vector median filter (VMF) will be used to remove the noise. This detection capability meets the so-called “switching” mechanism, which only selects noisy pixels for denoising. Hence, this proposed filter will expedite the processing time with the reduced number of vector calculations in the VMF due to this detection function. This type of detection is achieved with MI index and the indication of one-dimensional Laplacian kernels. We compare the proposed MIVMF with other well-developed vector-type median filters in the literature. Our experimental results show that the proposed filter is not only faster in the filtering process but also efficient in removing random impulse noise with different noise levels in color images. The MIVMF demonstrates a promising denoising result based on the criteria of peak signal-to-noise ratio and structural similarity index metric. With the visualization of processed images, the MIVMF can avoid image blurring, preserve the edge details, and achieve superior noise reduction.


Introduction
The major sources of noise corrupted in most digital images are from the process of image acquisition, quantization, and transmission.As one common example, when an image is transmitted through some wireless mobile networks, it may be ruined with noisy signals from atmospheric disturbances, such as thunder and lightning in the environment. 1hus, an image may be degraded such that it will not have the same original quality.The consequence of degraded images can create potential problems for further image processing and analysis.For example, a clustering algorithm for segmenting an image usually measures the relationships in the pixel space by categorizing the pixels into different classes. 2 Therefore, the existence of noise pixels will create different attributes for pixels that originally belonged to the same cluster.This may generate different clustering outcomes.Our goal is to remove noise in color images and restore the image, which, as a result, will be as similar as possible or identical to the original image.
In general, algorithms for removing noise in images can be divided into two methods: one for grayscale images and one for color images.4][5][6][7] As digital imaging technology has advanced, the necessity for color image processing has been in great demand.Although grayscale image processing is still needed in specific fields, such as medical imaging on x-ray computed tomography, 8 the wide usage of color and multichannel images is very common in many areas of imaging applications.In this study, we consider color images our primary focus in developing the denoising filter.
Prior to building the algorithm for denoising filters, one of the most important prerequisites is to take on the noise models for color images.Image noise occurs in a wide variety of forms. 1,5It is very common for noise to contaminate the pixels of images taken by the sensor.During the process of digitization and transmission, noise can be introduced into each pixel wherein one or more bit-errors will be embedded in the pixels of images.The Gaussian noise model and impulse noise model are two widely used models for characterizing the noise information for denoising filters in digital image processing.The Gaussian model is convenient for the simulation of noise to design and test the efficiency of a denoising algorithm.If the noise causes pixel data loss or saturation, the impulse noise model, also called salt-and-pepper noise, will be suitable for the noise characterization.Gaussian noise is used as a model of dark noise and often as a crude model of shot noise, which is governed by the Poisson distribution.The main sources of Gaussian noise in digital images arise during acquisition, for example, sensor noise caused by factors such as poor illumination, high temperature, and transmission noise.There are also different noises caused by devices, such as dark noise, which is due to the thermal fluctuations of stationary charge carriers.Dark noise frequently occurs in image sensors, such as chargecoupled devices.Noise degrades image quality and causes difficulty in further processing.Hence, an image denoising algorithm requires an image noise model for improving the signal-to-noise ratio in digital images.In this work, we assume that the impulse noise model is the major type of noise to be detected and removed.Impulse noise randomly and sparsely corrupts pixels to two intensity levels, relative high or relative low, compared with its neighboring pixels.
0][11] Let us assume that the density level of noise probability is δ and k value is the color channel index for red, blue, and green.For convenience, we assume that the red, green, and blue channels are assigned 1, 2, and 3, respectively, with an 8-bit pixel image for the following discussion.If δ is assigned 0.2, then 20% of the entire pixels of a given image is noise.The first impulse noise model, which is uncorrelated impulsive noise, 10 is formulated as follows: E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 1 ; 3 2 6 ; 7 5 2 SAP with probability δ or o k with probability 1 − δ where k ¼ 1; 2; or 3; (1) where r k SAP stands for the random vector that represents the impulse noise corruption with r k SAP ∈ ½0 or r k SAP ∈ ½255 (with equal probability) and o k represents the original color vectors.The subscript SAP represents the salt-and-pepper noise.Those two vectors result in the final outcomes x k of the corrupted image with impulse noise restricted to 0 or 255.This model is called a "salt-and-pepper" impulse noise type.This model is limited to only one channel corruption at a time for all three channels.Hence, we use lower case symbols r k SAP , o k , and x k to represent this model.The second impulse noise model is based on the correlation of impulse noise distribution among the color channels.This model can be expressed as follows: E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 2 ; 6 3 ; 5 2 0 X ¼ ½o 1 ; o 2 ; o 3 with probability 1 − δ ½r 1 SAP ; o 2 ; o 3 with probability δ 1 for channel 1 ðunder δÞ ½o 1 ; r 2 SAP ; o 3 with probability δ 2 for channel 2 ðunder δÞ ½o 1 ; o 2 ; r 3 SAP with probability δ 3 for channel 3 ðunder δÞ ½r 1 SAP ; r 2 SAP ; r 3 SAP with probability δ 4 for all channels ðunder δÞ ; where X represents a pixel vector and r i SAP and o i are similarly defined as in Eq. ( 1).δ 1 , δ 2 , and δ 3 are the respective probabilities of each channel corruption (these parameters were set up as δ 1 ¼ δ 2 ¼ δ 3 ¼ δ 4 ¼ 0.25 in Ref. 10).For example, if we assume that the noise level is 0.1 (i.e., 10% noise density), the δ value is 0.1.Each individual channel δ i will be given 0.25 probabilities under the overall 10% probability.Consequently, the combination of single channel corruptions from δ 1 , δ 2 , and δ 3 and all channel corruption will add up to 0.1 probability in total.The problem with this model is that it limits the noise corruption either on a single channel only or three channels simultaneously but does not permit two channels for the noise corruption simultaneously.This may not be realistic in practical applications.
The noise model that we used in our research is not limited to one channel for noise corruption.The noise signal used is also not limited to two relatively high and low values (i.e., values close to 0 or 255).Our model assumes random-valued noise corruption on the multiple channels simultaneously with a randomized noise level from 0 to 255.We assume that all channels are corrupted with the probability δ.The impulse noise model we used is shown in Eq. ( 3), where the subscript rv in r i rv stands for the random value.This model allows any combination of channels (can be 1, 2, or 3) to be corrupted with impulse noises.This is a more general impulse noise model compared with noise models 1 and 2. As a result, the comparison in terms of visual perception of noise corruption level is more severe than noise model 2. Please note that noise model 1 as shown in Fig. 1(b) looks more noisy than our model in Fig. 1(d) due to the difference in the threechannel noise (i.e., color) and salt-and-pepper.The latter makes dark and white noises more prominent compared with the color.E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 3 ; 6 3 ; 7 5 2 X ¼ 8 > < > : ½o 1 ; o 2 ; o 3 with probability 1 − δ or ½r 1 rv ; r 2 rv ; r 3 rv with probability δ : (3) There are many state-of-the-art developments and publications in the literature for impulse noise removal in color images.Similar to grayscale methods, many of them have their own advantages and limitations in terms of performance issues, such as edge-preserving capability and efficiency on noise removal.5][6] This type of MFs selectively suppresses noise pixels, leaving those uncorrupted pixels intact.It can preserve the edge details and avoid blurring the image.This mechanism is also used in some versions of the vector median filter (VMF) for color images. 10,12,13Our proposed idea on Moran's I (MI) (statistic) and four one-dimensional (1-D) Laplacian kernels is designed to improve the performance on the capability of noise detection, which will then be used for noise removal function.Our proposed filter is also a type of filter with a switching mechanism.MI is one of the oldest indicators of spatial autocorrelation frequently used in social sciences, such as geography and sociology.Based on our research in the literature, two groups of researchers have proposed the MI statistic in their work.Chen et al. 14 used the MI statistic for measuring the image quality of reconstructed images.Chuang and Huang 15 assume that the byte of a pixel in an image can be divided into two parts, signal and noise; the noise occupies the lower bits of a byte.The MI and join-count statistics then examine the noise bits to filter them out.In our study, the MI statistic is used as a measure in a defined window for detecting noise pixels.The motivation for this study is to measure the strength of spatial autocorrelation of pixels in the local neighborhood of the image.MI indicates the different spatial structures of the smooth and rough surfaces, which provide an index for impulse noise determination and preserve original noise-free pixels.In Sec. 3, we will introduce MI and its application on impulse noise detection in detail.The rest of this paper is organized as follows.The related work in Sec. 2 will briefly review research in impulse noise removal of color images and present our initial thought and motivation for improving contemporary filtering methods for color images.A comparison of the classical MF and VMF is given in this section based on our experimental results.In Sec. 3, the proposed algorithm with spatial autocorrelation associated with MI and Laplacian kernels is provided.Section 4 describes the simulation results, and Sec. 5 gives the concluding remarks and future work.

Classical Median Filter and Vector Median Filter
The classical MF is an efficient filtering algorithm for removing impulse noise in gray-level, color, and multichannel images.It is one of the most common nonlinear type filters.Due to its distinctive property in the impulse response of the MF, this filter has been widely used in suppressing impulsive noise. 16Many authors have analyzed the statistical properties of the filter, and some modifications and generalizations of the MF have been introduced. 16This MF is a simple and efficient algorithm that replaces the center pixel with the median value of the corresponding neighborhood window, such as a size of 3 × 3. A general MF is formulated as E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 4 ; 3 2 6 ; 5 5 5 where f is the filtered image, ðx; yÞ is the location of a pixel, W is the neighborhood window in the image, and the s and t values are determined by the window size m and the floor function (i.e., bm∕2c) for integer truncation.
Even though this filtering algorithm was originally developed for the grayscale image, it can intuitively be used in color images with a component-wise extension.The socalled marginal standard median filter (MSMF) 8,17,18 deals with each red, green, and blue channel separately, by calculating the median value of each channel within a neighborhood window.The restored image is then constructed by combining all RGB channels together, as shown in Fig. 2. Consequently, the filtered pixel will have the median value for each channel as a component in the pixel vector.This filter considers the median value in each separated channel without considering the correlation that may exist between color channels.As a result, the restored image may have color distortion (artifacts). 8,18stola et al. 16 proposed a VMF for impulse noise removal for color images in 1990.The VMF is based on ordering the vectors within the neighborhood window by calculating the cumulative pairwise distance and taking the vector corresponding to the lowest-ranked distance as the vector median value for the output.The VMF is formulated as in Eq. ( 5) Fig. 2 The MSMF applies to each component separately.A pixel consists of RGB components.The median value of each component is calculated, and the output is a new vector with the median value in each of the RGB components.
where the norm L can be implemented as either L 2 (i.e., Euclidean distance) or other different norms E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 5 ; 6 3 ; 7 3 0 X VMF ¼ arg min where X j (X k ) represents a pixel vector in the window and X VMF is the pixel vector with the lowest-ranked distance.So far, we reviewed two basic filtering algorithms used for impulse noise removal.Most of the state-of-the-art filters evolved from these two basic filters.Our filter is also motivated by these two basic filters.In Sec.2.2, we give a brief comparison of these two basic filters from the perspective of both the performance and time complexity of the filters.
Please note that we will denote the MSMF as MF since it is just an extension of the traditional MF for color images.

Brief Comparison Between the Median Filter and Vector Median Filter
Prior to moving into constructing new algorithms for impulse noise filtering, we briefly compared the MF and VMF for their similarities and differences on the filtering performance.Many publications have pointed out the advantage of the VMF for multichannel data due to the correlation that exists between the multichannel data.However, in some color images, such an advantage may not present.For example, if we take a 3 × 3 neighborhood window in a color image with all eight pixel values [255, 255, 0] and one noise pixel [0, 255, 255], both MF and VMF will have identical denoising results.This might be the reason that the MF shows a very competitive result compared with those of VMF in the following experiments based on the peak signal-to-noise ratio (PSNR) and structural similarity index metric (SSIM) measures.
Multiple sample images were tested to compare the denoising quality and time complexity.To assess the denoising quality, we used the PSNR 19 and SSIM 20 as the quantitative measure.The PSNR is calculated below E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 6 ; 6 3 ; 3 3 2 PSNR ¼ 10 × log 255 2 MSE ; where E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 7 ; 6 3 ; 2 7 6 MSE ¼ and M and N are the width and height of an image, I ij for an individual pixel of the noise-free image, and Ǐij is the corresponding pixel of the filtered image.The higher value indicates a better denoising result.The SSIM is designed to measure image quality considering human subjectivity in the context of luminance, contrast, and structural similarity. 20he comparison between original image x and filtered image y can be calculated using the following equation: ; t e m p : i n t r a l i n k -; e 0 0 8 ; 3 2 6 ; 6 6 2 SSIMðx; where μ x and μ y are the average intensities and σ x , σ y are the standard deviations corresponding to images x and y, respectively, and μ xy is the covariance.The values C 1 and C 2 are constants determined by the dynamic range value.
To conduct a comparison study for MF and VMF, Lena, pepper, and baboon images (Fig. 3) were tested with 5%, 10%, and 15% noise levels.The size of the filtering neighborhood window is 3 × 3 and each image is 512 × 512 pixels.In terms of image complexity, we calculated the entropy for each image using the below equation, which is based on the average (f) of three color channels E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 9 ; 3 2 6 ; 4 9 3 where HðfÞ is the sum of entropy for each gray-level probability, p i , in the histogram of the average of three color channels, f, and L − 1 is the maximum gray level for an image.(Here, L is the total number of gray levels.)In general, as an image complexity increases, its entropy value also increases.
Our empirical study on the MF and VMF revealed some interesting outcomes.As shown in Tables 1-3, the overall results from VMF are not significantly different from those of MF.This is shown in Table 3 for the baboon image, in particular, which has the highest entropy among the three images tested.In terms of time complexity, VMF is more than 10 times slower than MF in all test cases.However, VMF is generally considered to be more appropriate for color image processing because it considers inherent correlation between the red, green, and blue color channels of the image and prevents image color distortion. 8,10,17,18Even with some advantages over MF, the essential problem of slow filtering speed is the major weakness of VMF although some high-speed versions have been proposed. 21If we consider the VMF for the realtime application, its time complexity needs be improved.From this perspective, our proposed filter design will not only possess the advantages of the VMF but also improve its filtering process speed.

Brief Review on Various Vector Median Filters
Since VMF was introduced in the early 1990s, many variations of VMF have been proposed, and some of them use different distance measures. 8,9,18,22The basic vector directional filter (BVDF) utilizes the angle between two pixel vectors within the window. 8The summation of angular distances used in this filter is calculated below E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 1 0 ; 3 2 6 ; 3 6 0 AðX i ; X j Þ i ¼ 1; 2; 3; : : : ; 9; (10)   where AðX i ; X j Þ denotes the angle between pixel vectors X i and X j using the arc cosine function from the following equation: The problem with the basic vector median approach is that it can alter some pixels even if there are possibilities that those pixels are actually noise free.Consequently, the excessive filtering causes blurring effects on the denoised image.To solve this problem, the center weighted vector median filter (CWVMF) 8,9 was proposed to use a weight value for the center pixel in the window.This algorithm, however, lacks an impulse noise detection scheme.Consequently, the adaptive center weighted vector median filter (ACWVMF) 18,23 was proposed.The main difference between the ACWVMF and other filters introduced so far is that it has noise detection capability, instead of performing noise removal on every pixel.This ACWVMF filter provides a switching mechanism that differentiates between impulse noise and noise-free pixels.In the ACWVMF, the noise  detection step, which serves as a switching mechanism, is based on the concept of aggregated distances assigned to the pixels in the filtering window. 18The difference between the accumulated distances assigned to the central pixel and to the pixel with the lowest rank serves as an indicator of the noise.If the indicator is greater than (or equal) a fixed threshold, the output of the ACWVMF is a weighted mean of the central pixel of the filtering window and the vector median of its samples.Otherwise, the pixel is noise-free.In our proposed algorithm, MI statistic has a better indication than the mechanism used in the ACWVMF for noise or noisefree pixel.This statement will be verified later in the experimental results, as shown in Table 21.
Another approach is based on the "peer group" concept. 21,24,25This type of filter excludes corrupted pixels in the window to calculate the median vector value.The peer group, in short, is the group of pixels in the filtering window that minimizes the total sum of distances from the center pixel of the group to adjacent pixels.Therefore, the peer group vector median filter (PGVMF) is based on the trimmed sum of distances.If the number of pixels in the peer group (denoted by α) is equal to 9 in the window size of 3 × 3, it will be identical to the basic VMF algorithm (Fig. 4).The replaced pixel vector is determined by the pixels located within a peer group of pixels, which shows the minimum distance or minimum dispersion.
Compared with various filters introduced so far, the robust switching vector median filter (RSVMF) 12 has a solid noise detection algorithm that enables a filtering process; if a pixel is determined as a noisy pixel, the RSVMF will use the VMF to remove the noise pixel.Otherwise, the pixel value remains unchanged.The RSVMF works in a similar way as VMF except for the following modification: E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 1 2 ; 3 2 6 ; 5 6 6    if d center ≤ α • medðd 1 ; d 2 ; d 3 ; : : : ; d m Þ then no filtering otherwise; the filtering is same as in VMF ; ( where m is the total number of pixels in a window, d center is the cumulative Euclidean distances from the center pixel to pixel 1;2; : : : ; m in the window, and α is a constant value.The med function is to select median value from the cumulative distances of d 1 ; d 2 ; d 3 ; : : : ; d m using the way we calculate the cumulative distance in the VMF.Another denoising method is to process vector-based filtering through the trimming scheme. 26The basic idea of trimming schemes is to select a group of pixels in the filtering window that may exclude noise pixels through the ranking order.In a sense, this filter is very similar to the PGVMF.However, the RSVMF has some limitation on removing the noise pixels if the noise level is increased to a certain percentage, such as 30% noise that is used in our experiments.
All of the algorithms reviewed so far have their own unique characteristics and limitations for filtering impulse noise through vector-based processing.There are several different variations of algorithms reviewed above to improve the result in such a way that the filter can preserve edge details and minimize blurring effects. 13A comprehensive survey on impulse noise removal filters was given in Ref. 10.Our objective is to design an efficient algorithm for detection and removal of the random-valued impulse noise in color images based on the fundamentals of MI statistics and Laplacian kernels.Our preliminary study on multispectral images shows that this filter is effective in removing impulse noise. 27In summary, the vector-type MFs listed in Table 4 were tested and compared in this study.
3 Proposed Algorithm: Moran's I Vector Median Filter Our proposed algorithm for noise detection and removal in color images consists of three major components: MI for spatial autocorrelation, four 1-D Laplacian kernels, and the VMF.The first two components are used for noise detection in the algorithm, and the VMF is used for denoising.

Moran's I for Spatial Autocorrelation
The concept of spatial autocorrelation can be traced back to the first law of geography-"Everything is related to Fig. 4 A comparison of VMF and PGVMF. 25In this example, P1 is the vector median because P1 has the minimum distance from itself to all others with the VMF.For PGVMF, peer group vector median is P2 because alpha (α) value is 5; P2 is the minimum distance.everything else, but near things are more related than distant things."By definition, spatial autocorrelation is clarified as follows: "Spatial autocorrelation refers to the fact that the value of a variable at one point in space is related to the value of that same variable in a nearby location." 28In fact, the spatial information concept has long been used in pattern recognition and image analysis with different types of formats.To determine the spatial autocorrelation, Moran introduced MI to check the degree of spatial autocorrelation in areal data. 29MI is calculated by the following equation, and its range is between −1 and þ1: Here, we assume that there are m regions, and ω ij is a measure of the spatial proximity between regions i (with value y i ) and j (with value y j ).The parameter ȳ is the mean value of m regions.The interpretation of the I value is categorized into two specific spatial patterns.If the value closes to þ1, it indicates a strong homogeneous spatial autocorrelation where local similarity or homogenous spatial patterns exist.A value nearing −1 indicates the strong heterogeneous spatial autocorrelation, but it is a negative pattern in which heterogeneous values are grouped together (e.g., a mixture of very high and very low values).This negative spatial autocorrelation is unusual in nature.For example, high-income households are not usually located in urban twilight zones.
If the I value is zero, there is an absence of any strong spatial autocorrelation or no distinct spatial pattern, such as checkerboard. 28The calculation of MI can be exemplified with the areal data in Fig. 5(a), which is similar to the example in Ref. 27.
To find out the weight value of ω ij , it is necessary to check the connectivity between regions i and j (i.e., immediate neighbors).For example, region A has connectivity with B and C. Region D has connectivity with B, C, E, and F, as shown in Fig. 5(a).Hence, a binary connectivity weight matrix [shown in Fig. 5(b)] can be constructed corresponding to the data in Fig. 5(a).
From the denominator of Eq. ( 13), P m i ðy i − ȳÞ 2 represents the variance of the regional value.As a result, m is replaced by 6 [since there are six regions in Fig. 5(a)], MI is calculated as (the detail is omitted here) E Q -T A R G E T ; t e m p : i n t r a l i n k -; s e c 3 .1 ; 3 2 6 ; 7 5 2 6 Hence E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 1 4 ; 3 2 6 ; 6 6 3 Moran's I ¼ 6ð100Þ 18ð244Þ ¼ 0.1488: ( From the above result, the I value gives a small positive spatial autocorrelation for the given data. The implementation of MI for noise detection in color images can be done with the same procedure as shown in the example above.The difference now is that a region used in the previous example will be replaced by a pixel in the filtering window for impulse noise detection.An example is given in Table 5, which shows a 3 × 3 filtering Table 5 (a) A hypothetical example of 3 × 3 filtering window (one channel only), (b) its pixel numbering scheme for the pixels in (a), and (c) the binary connectivity (i.e., adjacency) matrix for (a) that shows which pixel is adjacent (marked by 1) to pixel number from 0 to 8. Otherwise, 0 will be used for nonadjacency.For example, for pixel 0, pixels 1, 3, and 4 are adjacent to it.window where the intensity range of a given pixel is between 0 and 255 for an 8-bit color image.Based on the 8-connectivity concept, 1 we can construct a weight matrix similar to Fig. 5(b).The weight matrix is given in Table 5. Please note that only one channel is shown here.
Based on the weight matrix formulated, for each color channel of RGB, MI can be calculated to check the degree of spatial autocorrelation.If all MI values from three channels have near þ1 value, then all the pixels in the window are relatively similar.In other words, if any MI value from three channels is close to −1, there is a high probability that impulse noises exist in the filtering window.The threshold value ε 0 for determining the noise area is set up through our experiments, which will be explained in detail in Sec.3.3.

Four One-Dimensional Laplacian Kernels
By means of a single detector based on MI index, it may not be robust to detecting impulse noise precisely.This is because MI measures a region's spatial autocorrelation.In other words, MI focuses on all pixels of the filtering window, not the central pixel itself in the window.Hence, it is required to have the next level detector for impulse noise.The application of Laplacian kernels is one of the precise and simple detecting algorithms for impulse noise.In fact, several median-based impulse noise detectors often misclassify a noise-free pixel into the noise category. 4,17In our proposed filter, if only MI is used for the detection, there is a chance that pixels located on the lines and edges can be categorized as impulse noise also.Using the Laplacian kernels as the second-level detection will eliminate this problem.In a hypothetical example as shown in Figs.6(a) and 6(b), we assume this image patch is a line with a direction in 135 deg.We also assume that the patch is identical in three channels.MI value is 0.079 in (a), and the center pixel is noise.Based on our proposed algorithm, if we set the threshold value ε 0 to 0.0, the center pixel will be detected as a noise pixel (MI value is not less than the threshold) and will go through the denoising process.However, the minimum response from four 1-D Laplacian kernels is 0. Based on the algorithm in Ref. 4, the response is not considered a noise pixel since its value is so small (i.e., 0).The reason is that we usually set the threshold value T 0 greater than 0 for the response of Laplacian kernels.Hence, MI can overcome the weakness of the 1-D Laplacian kernels in this example.On the other hand, if a line is given as shown in (b), MI value is −0.087.Based on our proposed algorithm, if we set threshold value ε 0 to 0.0, with the second level of detection (i.e., four 1-D Laplacian kernels), it will be detected as a noise-free pixel based on step 4 in our proposed algorithm (the condition ε 0 is satisfied, but the condition T 0 is not satisfied).The minimum response from four 1-D Laplacian kernels is 0 (with the 135-deg direction of the 1-D Laplacian kernel); based on the algorithm in Ref. 4, the response is not considered a noise pixel since its value is small (i.e., 0).This center pixel will be considered a    noise-free pixel.Hence, the weakness of MI can be complemented by the four 1-D Laplacian kernels.In Fig. 6, four 3 × 3 1-D Laplacian kernels are used in our experiments for the second-level noise detection.In fact, both 3 × 3 and 5 × 5 1-D Laplacian kernels 4 were tested simultaneously to verify which size of the kernel is more appropriate for the second-level detection.In our experiments, the kernels of different sizes do not show any significantly different results.At the first level of noise pixel detection, the corrupted image is tested through MI index; then if necessary, four 1-D Laplacian kernels will be used in the second-level detection.Four 1-D Laplacian kernels are sensitive to edges with four directions.The minimum absolute value from the four kernel operators is chosen as the detection value.
The Laplacian kernel response is denoted as C i;j , and ⊗ is a kernel operator as shown below E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 1 5 ; 6 3 ; 5 6 5 C i;j ¼ minfx i;j ⊗ K d ∶d ¼ 1 to 4g; (15)   where x i;j is a pixel and K d is a Laplacian kernel.Each kernel is applied to choose the minimum absolute value in the window.This value is compared with the threshold (T 0 ) value to determine whether it is a noise pixel or not [Eq.( 16)].Please note that we use C to represent C i;j for the pixel ði; jÞ in the examination in Eq. ( 16).If the value is larger than the threshold, it is an impulse noise.If the value is smaller than (or equal to) the threshold, then the current pixel is either a noise-free or edge pixel. 4The determination of this threshold will be discussed in Sec. 4 E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 1 6 ; 6 3 ; 4 2 4 fðCÞ ¼ C > T 0 ; Impulse noise C ≤ T 0 ; Noise free or edge : From the detection based on MI and Laplacian kernel response, a pixel is either classified as a noise pixel, in which the VMF will be called in to remove the noise, or the pixel value remains unchanged.

Proposed Filtering Process with Vector Median Filter
The remaining issue related to the implementation of Moran's I vector median filter (MIVMF) is the determination of the threshold value because the improper threshold setting can produce unwanted filtering results.First, the threshold value ε 0 for MI should be properly set up to check if pixels in the sliding window are located at the noise area or the relatively homogenous, noise-free area.We chose three images as shown in Fig. 3 to verify the proper threshold value.We tested all three images with different noise levels for 5%, 10%, and 15%, and the threshold value ε 0 can be properly estimated.If MI is less than ε 0 , then the Laplacian kernels will be used to determine if it is a noisy pixel.Otherwise, it is a noise-free pixel.Algorithm 1 MIVMF.
Step 1: Read in a noise color image and set threshold values ε 0 and T 0 .(Please note that the selection of threshold values ε 0 and T 0 was discussed in Sec.3.3 in detail and is given in Table 14.) Steps 2 to 5 are repeated for each pixel in the image for each iteration.
Step 2: Calculate MI value in the neighborhood of the pixel within the defined filtering window for each color component of the color image.
Step 3: Evaluate the four 1-D Laplacian kernel response values in the neighborhood of the pixel and select the minimum kernel response value (i.e., Min) for each color component of the color image.
Step 4: If any MI value from all three color components is less than ε 0 (first-level detection), and if the Min value from all three color components is larger than T 0 (second-level detection), go to step 5 (for denoising).Otherwise, go to step 2 (it is a noise-free pixel).
Step 5: Use the VMF for removing noise and then go to step 2.  As shown in Tables 6 and 7, the best threshold value ε 0 is somewhere between −0.1 and 0.1 from the results (bold print) of PSNR and SSIM.In Table 8, the best threshold value is between −0.2 and 0. From those experimental results, this information can be used as an indicator for choosing a threshold value for MI index for a color image with different complexity in entropy.Consequently, the promising PSNR and SSIM results from Lena, peppers, and baboon images are located around ε 0 ¼ 0. Hence, we set up our threshold value as ε 0 ¼ 0.0 for all images in our experiments.Furthermore, the difference in the range of possible threshold values for ε 0 is so small such that the selection of this index is not critical to the filtered results.
The next threshold to be determined is the response of four 1-D Laplacian kernels.We tested many images to find out whether there exists any common threshold value which can be used for an image with different complexity.However, we could not find such a common threshold value with our initial experiments.Therefore, we set up the hypothesis that image complexity might be related Fig. 9 Filtering results of Lena image.To see the filtering result in detail, it has been zoomed at the center area: (a) an original image, (b) 15% impulse noise added, and (c), (d), (e), (f), (g), (h), and (i) are the results of VMF, BVDF, CWVMF, ACWVMF, PGVMF, RSVMF, and MIVMF (i.e., our proposed filter), respectively.
Table 14 Some additional parameters for filters used in the experiments.

Filters
Extra parameters used besides 3 × 3 window CWVMF Center weight ¼ 4 with the threshold value of Laplacian kernels.We chose five images, as shown in Fig. 7, with different entropy levels and filtered with various threshold values.In our empirical study on the images tested, the threshold range is between 20 and 120, as shown in Fig. 8.In addition, the noise levels of 5%, 10%, and 15% were tested.In Fig. 8, only Laplacian kernels were used to determine the threshold (T 0 ) for the VMF.For the noise level 10% and 15%, the filtering process was run twice to improve overall outcomes measured by PSNR.For the 5% case, one filtering was good enough for removing impulse noise.The number of iterations is written next to the noise level, as shown in Tables 9-13.By observing experimental results from Tables 9-13, we discovered a trend in the pattern the setup of the threshold value for Laplacian kernels response for an image.This trend is to increase the threshold value based on the entropy of an image being denoised, as shown in Fig. 8. Therefore, it is quite straightforward to calculate the entropy of an image to estimate the threshold value.Based on this assumption, we used the interpolation method for assessing a threshold value for an image being denoised in our simulation in Sec. 4.
From the discussions above, our proposed denoising Algorithm 1 (MIVMF) is summarized in the steps.Please note that the algorithm can be repeated if needed.In other words, the output from the first pass of the algorithm becomes the input for the second pass and so on.

Experimental Results
To evaluate the performance of the proposed algorithm and compare it with other well-developed denoising filters, we tested all the filters listed in Table 4. Various test parameters, including the filtering window size and threshold values, are listed in Table 14.All test conditions except our proposed filter are based on the suggested parameter values from the literature reviewed.The size of the neighborhood window used for all filters is 3 × 3. To compare the denoising performance, PSNR, SSIM, and time complexity measured in seconds are calculated.Three images from Fig. 3 are used in the experiments.Some of denoised images are shown in Figs.9-12.Please note that in the noise levels 10% and 15%, both RSVMF and MIVMF were run twice to obtain the best outcome measured by PSNR.Other filters were run only once.(Otherwise, the image will be blurred if they were run more than once.)For the repetition, the output from the first pass of the algorithm becomes the input for the second pass and so on.
In our experiments, there is no significant difference for a filter in terms of the time complexity for processing an image with any noise level or image complexity based on the entropy measure.This can be seen from Tables 15-17.The proposed MIVMF is the best among all the filters tested with the PSNR measure.For the SSIM measure, which is for human eye perception, all the filters are almost identical, except that BVDF is degrading when the noise level is increased.
On the contrary, for an image with medium entropy, such as the peppers image, the PSNR measure for all filters is decreasing when the noise level is increased, as shown in Table 16.This phenomenon is also shown in the SSIM measure for the BVDF.Otherwise, the SSIM is very similar for all the filters tested.The proposed MIVMF has the best PSNR measure for an image with low, medium, and high entropy.For images with higher entropy, a trend exists for all filters that the SSIM is decreasing when the noise level is increasing, except for the VMF in which the 15% noise shows a high SSIM value, as shown in Table 17.In terms of the PSNR measure, all of the filters tested decrease if the image complexity is increased.However, the MIVMF has the overall highest PSNR value at different noise levels and with almost any complexity of images measured in entropy.
Based on the results shown in Tables 15-17, there is only slight difference on the performance of RSVMF and MIVMF for the low noise level.Hence, to measure the effectiveness of RSVMF and MIVMF for heavy noises, both filters were tested on 20%, 25%, and 30% noise levels on three images, and the results are shown in Tables 18-20.(Please note that the results are from three iterations of each filter, and overall time spent is the sum of three iterations.)Test threshold (α) for RSVMF is 1.5 based on Ref. 12.While the SSIM is similar for both filters, the proposed MIVMF has a better PSNR measure with the higher noise level.The MIVMF has the      overall highest PSNR value in those noise levels.In addition, the running time for the MIVMF is less than half of the RSVMF.
To further demonstrate the effectiveness of the proposed algorithm in denoising, we tested on some different images and performed a number of iterations for each filter listed in Table 14.Images tested are shown in Fig. 13 which includes parrot (entropy 15.25), penguin (entropy 15.40), and koala images (entropy 16.18).As mentioned in Table 14, we used 0.0 for MI threshold ε 0 and threshold T 0 is interpolated based on the image entropy.Table 21 shows the PSNR score for each filter tested.To each image 5% noise was added, and each filter was run only once in the experiments.Experimental results show that the proposed algorithm, MIVMF, has the best PSNR scores.
In our study, some filters will cause the distortion of images and blur the image details if the filter is run multiple times on the image.To test the efficiency of the proposed algorithm, we ran each filter with a number of iterations to compare the filtered results.The results were obtained with six iterations for each filter and are shown in Table 21.All PSNR scores for VMF, BVDF, CWVMF, ACWVMF, PGVMF, and RSVMF are decreasing when the number of iterations is increased.This phenomenon will continue if the filter is iterated again.This indicates that the filtered image is distorted and image details are not preserved.The PSNR scores with our proposed MIVMF either decrease or increase slightly, as shown in Table 21.In either case, the MIVMF will reach a saturation state, and its PSNR score will not change again.For example, in Tables 21(a) and 21(d) (first column), the PSNR of the MIVMF will stay in 23.57 and 26.43, respectively, no matter how many more iterations are repeated.Similarly, in Tables 21(b) and 21(c) (first column), the PSNR will stay in 29.59 and 30.33, respectively.This important characteristic shows that the MIVMF is an efficient filter with the good detection mechanism for determining if a pixel is noise or not.Hence, the image details will not be damaged or blurred.We also observed that the MIVMF has the highest PSNR value no matter how many iterations were repeated.In addition, the MIVMF runs much faster than other filters, as shown in Tables 15-20.
By observing the experimental results in Tables 15-21 and Figs.9-12, the proposed MIVMF shows a promising result compared with other vector-based filters.We developed an efficient denoising filter for accomplishing better time complexity and efficient denoising results, which was the goal set up at the beginning of this study.Our two-level detection function based on MI and 1-D Laplacian kernels works properly, so it can reduce the processing time and produce better filtered results compared with other similar type of vector-based filters.

Conclusion and Future Work
We propose a noise detection and removal filter that uses MI statistics (i.e., MI value) to calculate the spatial autocorrelation indices and combine with four 1-D Laplacian kernels for random-valued impulse noise detection for color images.As far as we know, this may be the first attempt to use MI statistics for color image denoising.This is an adaptive filter in which the MI value and response of four 1-D Laplacian kernels are calculated for each 3 × 3 neighborhood window.This two-level noise detection scheme works efficiently as it has the capability of distinguishing a given pixel as noise or not.The VMF is then used to process those noisy pixels and leave noise-free pixels untouched.Our proposed filter can be considered an efficient extension of the VMF in a sense.As an extension, the proposed filter can achieve    better denoising results and efficient time complexity for the color images, which are corrupted with random-valued impulse noises.To determine if a pixel is noise or not, two levels of detection are used.The first level is controlled by the MI value, and the second level is measured by the response of four 1-D Laplacian kernels.An empirical study was able to estimate the threshold value for MI statistics.An interpolation method, which can roughly estimate the threshold value that is used in the comparison with the response of four 1-D Laplacian kernels for an image being denoised based on the entropy measure, was also established.
The threshold value calculated with this interpolation method was tested in our experiments and illustrated with good denoising results.MI statistics shows an interesting result in our exploration of the application of color images for detecting random-valued impulse noises.We compared the proposed MIVMF with other vector-type MFs, including VMF, BVDF, CWVMF, ACWVMF, PGVMF, and RSVMF, using the criteria of PSNR and SSIM.The proposed MIVMF shows promising denoising results based on these criteria.The MIVMF is faster than the other filters tested in the experiments regardless of the noise level and image complexity.For the PSNR, the MIVMF has a higher value than the compared filters.By the visualization, the proposed MIVMF can avoid the image blurring results, preserve edge details in the image, and achieve superior noise reduction.We found that this kind of spatial statistic is very useful in the color image denoising algorithm.
Although we were able to estimate the threshold values that were successfully used in our proposed filter, the issue of determining an optimal and adaptive threshold value for detecting a noise pixel remains a challenging problem in our exploration of MI statistics.If this threshold value can be estimated with an adaptive approach, it would be more useful to generate better denoising outcomes.We only applied the concept of spatial autocorrelation based on MI statistics and 1-D Laplacian kernels for the impulse noise detection in color images.We concentrated on the nonfuzzy vector-type MFs in this work.Several fuzzy vectortype filters that work well for noise detection and removal also exist.For future work, we plan to investigate the following four tasks: (1) to develop an adaptive method that can estimate the threshold values more accurately, (2) to determine if the MIVMF is suitable and efficient for hyperspectral remote sensing images, (3) to extend our detection scheme to other noise models, such as Gaussian noise, and (4) to explore the fuzzy vector-type MFs and determine if the fuzzy mathematics can be used to improve the proposed MIVMF performance for denoising.

Fig. 1 A
Fig. 1 A comparison of different impulse noise models: (a) an original image, (b) corrupted with the noise model 1, (c) corrupted with the noise model 2, and (d) corrupted with the noise model 3.The probability δ is set to 10% for all noise models.All testing simulations in this paper will be based on noise model 3.

Fig. 5
Fig. 5 (a) An area of six regions that are represented by A, B, C, D, E, and F with a value associated with each region 26 and (b) corresponding binary connectivity matrix.

Fig. 6 A
Fig. 6 A hypothetical example shows a line with noise in (a) with 0 as the noise in the center pixel and without noise in (b).Four 1-D Laplacian kernels with a size of 3 × 3: (c) horizontal, (d) vertical, (e) 45-deg, and (f) 135-deg directions.

Fig. 8 A
Fig. 8 A comparison chart for the threshold values used in four 1-D Laplacian kernels for images tested.
25 for light noise, α ¼ 1.5 for heavy noise Proposed 3 × 3 window size, Laplacian kernel threshold value, T 0 is interpolated based on the image complexity with entropy measure as shown in Fig. 8 MI threshold ε 0 ¼ 0.0 Journal of Electronic Imaging 023023-12 Mar∕Apr 2017 • Vol.26(2) Hung and Chang: Moran's I for impulse noise detection. . .

Fig. 12 (
Fig.12(a) Specific filtering results of 30% noise added cases for the comparison among PGVMF, RSVMF, and MIVMF (i.e., our proposed filter).To see the filtering result in detail, it has been zoomed at the center area.Three original images are shown in the first column, the second column are results of PGVMF, the third column are results of RSVMF, and the last column are results of MIVMF.(b) The entire baboon image is shown to illustrate the denoising effect of PGVMF, RSVMF, and MIVMF.We can clearly see that the filtered result by PGVMF is distorted; in particular, the moustache of the baboon became very thick.Each filter was repeated three times to obtain the result.

Table 2
A comparison of MF and VMF for peppers image with entropy value 15.30.

Table 3
A comparison of MF and VMF for baboon image with entropy value 15.90.

Table 1
A comparison of MF and VMF for Lena image with entropy value 15.12.
Note: s denotes seconds.

Table 4
Different filters being compared in this study.

Table 6
Threshold testing with Lena image: entropy 15.12.

Table 7
Threshold testing with peppers image: entropy 15.30.

Table 8
Threshold testing with baboon image: entropy 15.90.

Table 10
Results on F-16 image.The number next to each noise level column shows the number of iterations in the filtering process.
Note: The best threshold is highlighted with bold font.

Table 9
Results on ocean image.The number next to each noise level column shows the number of iterations in the filtering process.
Note: The best threshold is highlighted with bold font.

Table 13
Results on IC motherboard image.The number next to each noise level column shows the number of iterations in the filtering process.
Note: The best threshold is highlighted with bold font.

Table 12
Results on caster image.The number next to each noise level column shows the number of iterations in the filtering process.
Note: The best threshold is highlighted with bold font.

Table 11
Results on mountain image.The number next to each noise level column shows the number of iterations in the filtering process.

Table 18 A
comparison of RSVMF and MIVMF on heavy noise added Lena image.

Table 20 A
comparison of RSVMF and MIVMF on heavy noise added baboon image.

Table 19 A
comparison of RSVMF and MIVMF on heavy noise added peppers image.

Table 21
The PSNRs for different numbers of iterations and filters are listed: (a) parrot was corrupted with 5% noise, (b) penguin with 5% noise, (c) koala with 5% noise, and (d) parrot with 15% noise.Each filter was run six times in the experiments and is represented with the respective number in the first column, and then each PSNR was calculated for each run.The saturated modes were highlighted with italicized font in the table for the MIVMF filter.
Note: The saturated modes were highlighted with italicized font for the MIVMF filter.