A challenging problem of blind estimation of inherent sensor noise parameters [mainly its variance or standard deviation (STD)] from image data has been extensively studied by researchers for the last decade (see Ref. 1 and references therein). For a given sensor, the problem is to estimate noise parameters directly from noisy images acquired by this sensor.126.96.36.199.188.8.131.52.184.108.40.206.–15
Sensor noise must be detected and quantified prior to the majority of subsequent image processing tasks. Such information can help to properly select a suitable technique or adjust a method parameter to a current noise level (unknown in advance), with the final goal of making these techniques operate well enough. For example, different image filtering methods are needed to deal with either additive (signal-independent) noise (see Ref. 16 and references therein) or signal-dependent Poisson noise [typical for charge-coupled device (CCD) sensors].17220.127.116.11.18.104.22.168.26.27.–28 A threshold that is a function of noise standard deviation can be used with an edge detector.13 In Refs. 29 and 30, a method is proposed for estimating the denoising bounds for nonlocal filters from a noisy image, where noise statistics are to be known or accurately preestimated from the same noisy data. Similar results for local filters were obtained in Ref. 31, with the same requirement for noise statistics.
A signal-independent spatially uncorrelated noise model was the first possibility widely considered in the literature for modeling sensor noise in a very large number of image processing applications. In these applications, the noise is typically assumed as a zero mean stationary Gaussian distributed random process. Such process is fully described in terms of second-order statistics: its variance or standard deviation and a two-dimensional (2-D) Dirac delta function for its spatial autocorrelation function. For such noise models, the methods designed for estimating noise standard deviation can be roughly divided into two groups: the methods operating in the spatial domain and those operating in the spectral domain. Spatial methods, also called homogeneous area (HA) methods,32 make an essential use of image homogeneous areas characterized by a negligible level of texture spatial variation compared to the noise level. Spectral methods utilize a suitable orthogonal transform to better separate image texture and noise; the former is assumed to be smoother compared to image noise.33 They can have certain advantages compared to spatial methods, as the latter also can be applied to nonintensive texture, in addition to homogeneous areas. In Refs. 15, 3435.–36, the estimation of additive noise standard deviation is performed by analyzing the observed data in blocks of fixed size in the discrete cosine transform (DCT) domain. Estimation using wavelet transform is also possible.37,38
However, with advances in CCD-sensor technology, the applicability of the signal-independent noise model is diminishing, and signal-dependent photonic noise is becoming more and more dominant.14,39 Nevertheless, the level of signal-independent thermal noise remains nonnegligible. Then, one has to deal with mixed signal-independent and signal-dependent noise, which, in general, can also be treated as signal-dependent. Such noise is intrinsically nonstationary, but it can be locally approximated by stationary additive noise, with variance being a function of image intensity. In this case, the dependence of local variance on image intensity (called noise level function, or NLF, in Ref. 10) is of interest. If this dependence can be approximated by a polynomial, then one has to estimate the parameters (coefficients) of such a polynomial. For CCD-sensor noise, a first-order polynomial is considered to be a proper model40 characterized by two parameters, later referred to as signal-independent and signal-dependent component variances.
It is important to mention that signal-dependent noise is significantly more restrictive compared to signal-independent noise: homogeneous areas of sufficient size, with intensities covering the whole image intensity range, are needed to accurately estimate noise variance as a function of image intensity. This problem was addressed, for example, in Ref. 10, where a priori information on an estimated nonlinear NLF was used to compensate for the lack of homogeneous areas and to stabilize the estimation process. Note that such a priori information is not available for all sensors. In this situation, the estimation of noise parameters (coefficients of a polynomial and signal-independent and signal-dependent component variances) should be performed in a blind manner, relying only on observed noisy data. To reach high performance in such a situation, a blind noise parameter estimator should satisfy the following requirements:1
1. To provide unbiased estimates with as little variance as possible.
2. To perform well enough at different noise levels.
3. To be insensitive to image content; i.e., to provide appropriate accuracy, even for textural images.
By saying “with as little variance as possible,” we mean that there is a certain theoretical limit on the estimation accuracy (in terms of standard deviation) of noise parameters from image data.15 It is desirable for an estimator to perform close to this limit. The true image content is certainly nonuniform and may include different texture patterns, edges, objects of small size, etc. All these heterogeneities should not essentially influence the performance of a noise parameter estimator; i.e., estimated bias and standard deviation should not increase significantly (over a theoretical limit).
Potentially, the performance of blind methods can be very high—certainly higher than that of a human operator. The reason is that these methods can use subtler differences between image content and noise, which might be not visible to the human eye. But satisfying the above requirements altogether (and reaching a high level of estimation accuracy) for signal-dependent noise has been a difficult problem.12
Extended versions of approaches, originally proposed for the estimation of signal-independent noise standard deviation, have been considered to deal with signal-dependent noise, among them the well-known scatterplot approach.39 Recall that a scatterplot is a collection of points, each representing image local variance versus local mean. To reduce the influence of outliers, a robust fit to the scatterplot data points has typically been considered by different authors (e.g., Refs. 8, 10, and 39). Unfortunately, robust fit in the presence of a large percentage of abnormal local variance estimates (obtained from heterogeneous fragments) may appear very unstable. As a result, different fit strategies can lead to notably different estimation results.12
To meet all the requirements discussed above, we extend here the promising approach that we recently proposed in Ref. 15 to estimate the standard deviation (or variance) of signal-independent noise (assumed to be spatially uncorrelated). Our approach introduces two specific maps: the image texture informative (TI) map and the noise informative (NI) map. These two maps are complementary; i.e., for a given image, each scanning window (SW) belongs to one or another map. Assigning a given SW to one of these maps is decided based on the Fisher information (FI) on image texture parameters and noise standard deviation contained in this single window. In the NI map, a large amount of FI on the noise standard deviation is contained in each selected SW forming this map. Such SWs contribute in solving the main task, i.e., the accurate blind estimation of noise parameters. On the contrary, in the TI map, a large amount of FI on image texture parameters is contained in corresponding SWs. Such SWs are involved in solving an auxiliary task (i.e., the estimation of image texture parameters).
Note that both tasks are not mutually exclusive since texture parameters for NI SWs and noise parameters for TI SWs remain unknown, due to mutual masking of texture and noise. Therefore, the core of our approach is to use both maps simultaneously in an iterative manner. Noise parameters are estimated from NI SWs and then are applied to improve the estimation accuracy of texture parameters from neighboring TI SWs. The latter parameters, in turn, are used to improve the estimation accuracy of noise parameters.
To implement this scheme, we carry out the maximum likelihood (ML) estimation of a parametrical 2-D fractal Brownian motion (fBm) model, selected for describing locally the texture of a 2-D noisy image SW. The FI on the estimated parameter vector (including fBm-model parameters and noise standard deviation) can then be derived. To obtain the final estimation of noise standard deviation from the NI map, two alternatives, either fBm- or DCT-based estimators (, ), are proposed. The first one performs direct parametrical ML estimation of noise standard deviation from NI SWs, with a texture correlation matrix defined according to the fBm model and parameters derived from neighboring TI SWs. The second one applies DCT transform to NI SWs and uses only a fixed and limited number of high-frequency DCT coefficients to estimate noise standard deviation.
Now, under signal-dependent noise hypothesis, the main difficulty is to take into account the variation of noise standard deviation with regard to image intensity (due to the signal-dependent noise component), while still resorting to image NI/TI maps. For this purpose, we analyze how image NI SWs are distributed over the intensity range. This distribution allows the finding of narrow intensity intervals where noise variance can be accurately estimated and the discarding of noninformative intensities (those without any NI SW detected). Final estimates of noise signal-independent/dependent component variances are obtained by linear fit applied to these accurate variance estimates localized with regard to intensity. The whole procedure ensures the absence of outliers among local estimates of noise variance. Therefore, robust fit procedures that can be unstable are no longer needed.
This paper is organized as follows. Section 2 briefly introduces the fBm model and then recalls and signal-independent noise variance estimators that we previously proposed. Later in this section, the signal-dependent noise parameter estimation problem is introduced from an informational point of view, including FI distribution over the available intensity range for a given image. In Sec. 3, and estimators are extended for signal-dependent noise and the potential accuracy of such noise parameter estimates is provided. In Sec. 4, the performance of and estimators is comparatively assessed against two other modern methods in a large image database and real noise from a CCD sensor. Finally, in the last section, conclusions are offered.
Signal-Dependent Noise Parameters Estimation from Noise and TI Maps
FBm Model for Describing Texture Locally
To describe image texture locally, we propose to use the fBm model . Fractal analysis based on mathematical fBm processes has been suggested as a useful technique for characterizing real-life images, as they can describe quite complicated textures or shapes of natural scenes with a minimal parameter set.41
By definition, , , is a Gaussian process [the original coordinates are at the point (0,0): ] with the correlation function:42
In spatial terms, the Hurst exponent describes fBm-texture roughness ( for rough texture, for smooth),43 and describes fBm amplitude.
By , , and , we denote a single-component image with columns and rows (for multi-component or color images, the approach proposed here should be applied component-wise). Suppose that the image is affected by signal-dependent noise according to the following model:
However, in the section of this paper that describes the experiment, we concentrate on a reduced-order noise model for recent CCD sensors. In this case, the noise is supposed to be the sum of two components: . The first one, , is signal-independent with variance , and the second one, , is signal-dependent with variance (Poisson-like noise). This corresponds to a polynomial model of the first order () and, thus the coefficient vector simply reduces to :
Structure of and Signal-Independent Noise Variance Estimators
At this point, let’s briefly recall the main ideas suggested in the and signal-independent noise variance estimators () proposed by the authors in Ref. 15. Their generalized structure is recalled in Fig. 1. This structure used the fBm model and both image NI and TI maps. In the estimation process, a processed sample corresponds to a noisy textural fragment or SW of the image. The sample is described by a statistical parametrical model. The model parameters vector includes both texture parameters (fBm-model parameters) and signal-independent noise variance . After initialization (stage 1), NI/TI maps and noise standard deviation are simply assumed to be equal to initial guesses or current refined estimates in stage 2. At this stage, texture parameters are estimated for each TI SW first. Then, these estimates act as texture parameter estimates in neighboring NI SWs. This stage results in estimates of the parameter vectors for all SWs for further use in stage 3.
In stage 3, those SWs that can be efficiently used for estimating either noise or texture parameters are identified. We propose to perform this task by considering the corresponding FI on involved parameters or, equivalently the Cramér-Rao Lower Bound (CRLB), to each SW. By setting a proper threshold on CRLB, it becomes possible to find a subset of NI/TI SWs that provide noise/texture parameters estimates with a predefined accuracy. Finally, in stage 4, the noise standard deviation is estimated for each NI SW using either a fBm-model-based ML estimator () or DCT (). Note that by using only NI SWs ensures that all individual noise standard deviation estimates in stage 4 achieve a predefined accuracy and that there are no outliers among them. As a result, a simple nonrobust estimation procedure (for example, weighted mean) is sufficient in stage 4 to obtain the final estimate .
FI Distribution over Images
When solving the estimation problem for signal-independent noise, the total amount of FI on noise standard deviation contained in all noise-informative SWs determines the potential performance of an estimator (in terms of its variance).15 For signal-dependent noise, when noise standard deviation is a function of true image intensity, the distribution of FI over the image intensity range becomes of major importance. Specifically, if all noise-informative SWs have the same intensity mean , only the value of can be estimated. This allows estimating the pure Poisson-like () or multiplicative () signal-dependent noise standard deviation by and , respectively (with accuracy that does not depend on ). If a mixture of signal-independent noise and either Poisson-like or multiplicative noise is assumed, the best accuracy can be achieved when FI is distributed equally between two intensities, and , with being as large as possible. When no a priori information about is available, then FI uniformly distributed over all image intensity ranges is the best option.
Unfortunately, for a particular image, this distribution cannot be modified. What can be done, in practice, is to establish the actual distribution for a given image and to obtain the corresponding accuracy of noise parameter vector estimation, taking into account available a priori information. This problem will be addressed next, and and estimators will be extended to estimate the function.
The corresponding FI about the parameter vector on texture parameters and noise standard deviation was introduced in Ref. 15 for a single SW () as
Assume that noise-informative SWs have been detected, and assign mean intensity to the th NI SW. For the subset of these windows with , we assume a signal-independent noise model with standard deviation . Then, joint FI matrix on extended parameter vector can be obtained in a similar way:
Using , we define CRLB per unit intensity range as
To get better insight on the meaning of variables defined above, the possible shape of and its relation to the image content are illustrated in Fig. 2. Figure 2(a) displays the green component of image 3 from the NED2012 color images database (Noise Estimation Database), later referred to as the test image. We created the NED2012 database especially for testing the noise parameter estimator. It is described in detail in Sec. 4.1, later in this article.
The function has been calculated for the test image, and it is displayed in Fig. 2(b). In this experiment, we fix to calculate . A way to calculate automatically will be described in Sec. 3.1. Intensities of the test image cover range approximately from 0 to 1,200. In Fig. 2(b), four image intensity intervals with the lowest can be seen, with image intensities around 10 (1), 200 (2), 400 (3), and 800 (4).
Objects in the test image with intensities falling within intervals 1–4 are marked in Fig. 2(a) with the corresponding numbers. Interval 1 corresponds to the nonintensive dark tree texture, intervals 2 and 3 relate to homogeneous house fronts, and interval 4 relate to cloudy sky. One can see that indicates image areas that provide accurate noise standard deviation estimation and their localization with regard to image intensity.
and Estimators for Signal-Dependent Noise Parameters
Preliminaries of the Proposed Estimators of Polynomial SD Noise Variance
While designing the signal-dependent noise parameter estimator, we assume piecewise-constant approximation to the function:
As utilizing NI SWs along with either fBm-based or DCT-based noise standard deviation estimators ( or ) was proved to be efficient in Ref. 15 for signal-independent noise, we propose here to apply these two estimators to estimate in each NI SWs subset, with mean intensity falling within a given interval .
Before using Eq. (8), the set of intervals should be properly selected. This can be done by taking into account the relationship between and . Indeed, for noninformative image intensities with high , the interval has to be increased to provide sufficiently accurate noise standard deviation estimates. Conversely, for informative image intensities with low (areas 1–4 in Fig. 2), the same accuracy can be achieved for smaller . It is natural to estimate noise standard deviation with a predefined accuracy for each considered intensity interval ; i.e., to require9) is an extension of the thresholding procedure used in Ref. 15 to select a given SW as NI in the signal-independent noise case. Whereas this original constraint requires the signal-independent noise standard deviation to be estimated with a predefined accuracy from a single NI SW, Eq. (9) requires the same standard deviation estimation, but from the subset of NI SWs with mean intensities within the interval .
Therefore, is just a scaled-down version of . In the example considered in Fig. 2, we set . For this value, varies from 1 to 10 in informative intensity intervals 1–4 and exceeds 200 in noninformative intervals [shown by the peaks in Fig. 2(a)]. It follows that for informative intensities, noise variance can be accurately estimated from very narrow intensity intervals, enabling fine NLF analysis.
Based on Eq. (10), the set of intervals is obtained by the following procedure:
1. Find that minimizes : ;
2. Calculate by substituting to (10);
3. Save the first interval , and set .
4. Find a new pair such that is subject to constraints , . Again, is given by Eq. (10) for .
5. Add a new interval to the list of intervals found at previous iterations. Increase by unit.
6. Repeat this process until no new pair can be found, and then terminate.
For each interval , is estimated using either the or the estimator (under signal-independent hypothesis). At the same time, is calculated for future use. The output of the proposed signal-dependent noise estimator is the set of noise standard deviation estimates obtained for each interval and the corresponding set of mean intensities .
Now, the maximum likelihood estimation (MLE) of coefficients vector is obtained by
Proposed Estimators of Polynomial SD Noise Variance
Note that the signal-dependent noise variance estimate obtained by Eq. (11) requires availability of the values of and , that are based on true noise variance . Therefore, the signal-dependent noise variance estimation procedure should operate iteratively as described below. The structure of the proposed estimator for texture parameters and signal-dependent noise standard deviation is summarized in Fig. 3. In the following, we provide a description of this estimator and illustrate its behavior based on the test image [Fig. 2(a)] for . True parameter vector is obtained for this image in the experiment via a calibration procedure (see Sec. 4.1 of this article for details).
In the first stage, noise is assumed to be signal-independent and an initial guess is calculated as specified in Ref. 15. It is equal to the minimum of sample standard deviation estimates over all image nonoverlapping SWs. For the test image, the value was obtained. The initial guess for the vector assumes the initial value .
Texture and noise parameter estimates and NI and TI maps are updated in stage 2. In this stage, we fix noise parameters to be equal to either an initial guess or the previously estimated value (here, defines the iteration index). The signal-dependent noise variance for each SW (both NI and TI) is calculated according to the retained polynomial noise model (a mixture of signal-independent and Poisson-like signal-dependent noises in this case) as , substituting true image intensity by mean intensity over current SW. Then, fBm-model parameters for TI and NI SWs are estimated, and discrimination between texture/NI SWs can be refined as specified in Ref. 15.
Next, in stage 3, a current value of relative CRLB per unit image intensity () and the corresponding set of intervals are calculated as described above. For illustration purposes, the obtained in the first iteration is shown in Fig. 4(a) as a straight black line.
The set of intervals allows the estimating of values using either the fBm- or DCT-based estimator and refining the estimate according to Eq. (11). The estimates and function are both shown in Fig. 4(a) for the first iteration of the algorithm as black dots and a solid black line, respectively.
The estimate is iteratively refined by repeating stages 2–4 until convergence. The convergence of the algorithm can be observed from the comparison of Fig. 4(a) with 4(b), where the first and the final (fifth) iteration estimates of , , and for the test image are shown. Note that for each iteration, Fig. 4 shows estimates at the beginning of the iteration (signal-independent noise for the first iteration), not at the end. It can be seen that estimates are concentrated at intensities where takes on minimal values. Noise variance estimate for from 600 to 1200 does not change significantly with iterations. Conversely, for from 0 to 300 that corresponds to areas (1) and (2) in Fig. 2(b), the noise variance estimate decreases from about 170 [the first iteration, Fig. 4(a)] to about 5 [the last iteration, Fig. 4(b)], and the number of estimates decreases significantly (a smaller number of SWs is considered NI by the algorithm).
Now let us comment on the algorithm behavior for the textured area of the test image with an intensity close to 60 (dark trees in the lower-right part of the test image). With iterations, noise variance estimates are decreasing for this area. Consequently, the signal-to-noise ratio (measured locally) is constantly increasing. As a result, this area progressively becomes less and less noise informative [this is reflected by the increase of function clearly seen in Fig. 4]. Finally, in iteration 5 [Fig. 4(b)], this area is excluded from the noise variance estimation process (it becomes related to the TI map).
The results obtained at final iteration confirm that, given a degraded image, indicates that image areas that can provide accurate noise variance estimation and their localization with regard to image intensity. It can be clearly seen that the estimated curve is very close to noise variance obtained by the calibration procedure. For the considered test image, there are enough NI intensities to provide a high estimation accuracy of signal-dependent noise variance. For images with not enough NI (homogeneous) areas for estimating noise parameters, may become inaccurate. Based on our experiments, we have found that this extreme case can always be detected by checking the accuracy of at the current iteration or the entries of the matrix [see Eq. (12) for details].
Experimental Results Using the NED2012 Database
This section deals with the application of the two designed estimators, and , of signal-dependent noise parameters to the NED2012 database of images that have been corrupted by signal-dependent noise. Estimator performance is analyzed and compared to that of the recently developed Automatic Scatterplot Reference Points Fitting with Restrictions (ASRPFR) method (discussed in Ref. 12) and the ClipPoisGaus_stdEst2D method published in Ref. 9.
The primary goal of this study is to determine the potential accuracy of signal-dependent noise parameter estimation (variance of signal-independent and -dependent components) from real images of different types, and to compare the performance of existing estimators to this bound.
Image Database for Testing Signal-Dependent Noise Estimation Algorithms: NED2012
A key item in testing noise variance estimators is the availability of images with known noise parameters. Artificial noise-free images with synthetic noise raise questions about the applicability of the obtained results to the real images corrupted by real noise. Another possibility lies in using real-life images with a low level of noise, with synthetic noise added for test purposes. In this area, the TID200844 database has been extensively used.45 However, we have identified four drawbacks of TID2008 that prevent us from exploiting it in our study:
1. Restricted image size. Indeed, the size of the images in the TID2008 database is . This corresponds to . However, modern digital cameras typically have more than 10 Mpx sensors.
2. Images from the TID2008 database are in 8-bit representation, while 12 or 14 bits is typical currently for images stored in raw format (for both digital cameras and remote sensing acquisition systems).
3. TID2008 images are subject to the demosaicing procedure to convert them from a color filter array (CFA) to RGB representation. Demosaicing unfortunately affects the spectral properties of both image texture (due to smoothing) and inherent noise (because it becomes spatially correlated). It is highly preferable to deal with CFA representation to assess data directly from a camera sensor.
4. TID2008 images contain inherent noise that, strictly speaking, does not allow to consider them as noise-free.15 This noise variance cannot be accurately estimated due to issues 1 and 3. Automatic analysis shows that its variance is about 4,15 but manual analysis shows it can locally reach 4 to 10. Such values are critical for our situation, as the standard deviation of the estimation error of additive noise variance provided by the estimator (as we will show next) can be as low as 0.2.
To overcome these drawbacks, we have decided to base our study on 12-bit raw images from the Nikon D80 DSLR camera with a 10.2-Mpx CCD sensor. No extra noise was generated; we only dealt with the parameter estimation for noise that was originally present in D80 images. Our main assumption is that the noise parameters remain the same with time and camera operational conditions. Absence of noise spatial correlation is also assumed. Our experiments have shown no violation of these assumptions at the attained level of accuracy.
To accurately estimate true noise parameters of Nikon D80 sensor for ISO100, we have used the following semi-automatic calibration procedure:
1. A series of 17 images of a white sheet of paper was taken in raw Nikon electronic file format. For these images, we had a fixed International Standardization Organization (ISO) value (equal to 100), as well as other camera settings (using a manual regime), except for shutter speed. By selecting different shutter speeds, a full image intensity range (in 12 bits) was covered. To suppress image texture (due to paper surface), strongly defocused images were collected. Then paper texture was smoothed, leaving sensor noise unaffected.
2. This series of images was partitioned into nonoverlapping SWs. A 2-D DCT transform was then applied to each window. The highest 16 coefficients (with indices from 5 to 8 for both dimensions) from each window were stored for further processing. By relying on these high-frequency coefficients, we additionally diminished the influence of image texture. For each such group of 16 coefficients, we calculated the image mean intensity in the corresponding SW.
3. The available intensity range from 0 to 4098 (12 bits) was divided into narrow intervals. DCT coefficients were grouped according to their corresponding mean intensities. In this manner, for each th intensity interval, DCT coefficients were collected. The sample variance of these coefficients, , was an estimate of the signal-dependent noise variance in ‘th intensity interval.
4. The coefficient vector was obtained by Eq. (11), with .
Visual analysis of noise variance dependence on image intensity shows notable deviation from the theoretic linear shape (first-order polynomial), especially for the blue component (Fig. 5). To take this into account, the order of the approximation polynomial was set to . The obtained estimates are given in Table 1 (we call them calibration lines). The noise parameters obtained via the calibration procedure will be marked with index “0” below.
Sensor noise parameters for the Nikon D80 digital SLR camera (statistical error at one sigma level is specified)
|Component||c0 (σn.SI2)||c1 (σn.SD2)||c2|
It can be seen that the green channel is the least noisy one, followed by the red and blue channels. Quadratic terms for all channels are nonzero. These values are statistically significant (more than or 14.3 sigma) and cannot be neglected. They probably appear due to the internal regulations of the camera.
For testing the two proposed estimators, we selected 25 D80 images taken from the same camera during a two-year interval and organized them into the NED2012 database. The database includes images with different content. Some of them have large homogeneous areas (e.g., sky), while others are quite textural, and defocused areas are present (Fig. 6). All images are presented as a CFA array of in size. Red, green, and blue channel data were extracted from the CFA array by subsampling. Images have different ISO values (from 100 to 320), different shutter speeds (from to ), different apertures (from f/4 to f/14) and different focal lengths (from 18 mm to 135 mm). From this set of parameters, it is the ISO parameter that directly affects noise parameters for both signal-independent and -dependent components. In order to compensate for this influence and convert all images to the reference ISO100, we have simply normalized each image by a factor of 100/ISO before processing.
We will first prove that estimation results obtained by the method agree with the calibration data shown above. For this goal, we applied the method to all images from the NED2012 database simultaneously by processing 1,500 SWs of randomly selected from each NED2012 image (a total of windows). In this manner, very high estimation accuracy of the , , and coefficients can be reached. Estimation results for all three channels are shown in Table 1 (the line). Figure 7 illustrates these results for the blue channel.
One can clearly see that there is no statistically significant difference between estimates of noise parameters obtained for two different datasets (the NED2012 database and the set of 17 calibration images) by the calibration procedure and the method (they differ by less than ). It is worth noting that the accuracy provided by the method is only slightly worse than the one obtained by the calibration procedure. In this test, the method shows its potential ability to deal with signal-dependent noise that is more complex than mixtures of additive and Poisson/multiplicative noises.
In the next experiment, we restricted ourselves to the case of a mixture of signal-independent and Poisson-like signal-dependent noises. The overall noise variance is thus signal-dependent: . As was shown above, the noise in the Nikon D80 images does not strictly follow this linear model. Therefore, in order to nullify quadratic term before estimation, we normalized the image intensity in each SW byTable 1. After such normalization, noise parameters become as specified in Table 1, with .
Accuracy Analysis of Signal-Dependent Noise Parameter Estimation
The considered set of four estimators, , , ASRPFR, and ClipPoisGaus_stdEst2D, were applied to each of the 25 images of the NED2012 database in a component-wise manner (red, green, and blue components were processed independently). A SW of in size was selected for both and . The choice of this particular SW size is justified in this subsection. For each component, the two noise variance components and were estimated. Overall, 25 estimates were obtained for each channel. Their empirical probability density functions (pdfs), pdf () and pdf (), are shown in Figs. 8 and 9, respectively. The main statistical characteristics of these estimates [mean , bias and standard deviation ] are given in Table 2. The bias measure is calculated as , where is the true value of .
Mean value M(·), bias and standard deviation STD(·) of noise variance estimates on the NED2012 database.
|(a) Signal-independent component|
|NI+DCT (without quadratic term correction)||Red||7.94||7.84||−1.22||1.33|
|ASRPFR||Red||7.94||10.51 (median)||32.37||7.53 (MAD)|
|Green||5.06||31.11 (median)||514.77||31.11 (MAD)|
|Blue||11.36||25.52 (median)||124.72||14.09 (MAD)|
|ClipPoisGaus_stdEst2D||Red||7.94||10.60 (median)||37.83||13.09 (MAD)|
|Green||5.06||11.51 (median)||126.35||59.10 (MAD)|
|Blue||11.36||13.83 (median)||12.13||10.62 (MAD)|
|(b) Signal-dependent component|
|NI+DCT (without quadratic term correction)||Red||0.142||0.149||4.550||0.010|
|ASRPFR||Red||0.142||0.153 (median)||7.429||0.031 (MAD)|
|Green||0.134||0.138 (median)||2.831||0.079 (MAD)|
|Blue||0.174||0.216 (median)||24.135||0.047 (MAD)|
|ClipPoisGaus_stdEst2D||Red||0.142||0.148 (median)||1.100||0.052 (MAD)|
|Green||0.134||0.138 (median)||2.135||0.170 (MAD)|
|Blue||0.174||0.195 (median)||14.210||0.062 (MAD)|
Bold values indicates values with the lowest bias magnitude and STD.
Preliminary conclusions can be drawn from these results. Among these four estimators, ASRPFR and ClipPoisGaus_stdEst2D provide worse performance than the and with regard to both signal-independent and -dependent components. The main factor that degraded the performance of the ASRPFR and ClipPoisGaus_stdEst2D estimators is the significant number of outliers. However, they form a pronounced mode in the vicinity of the true value of signal-independent and -dependent noise component variances anyway. Therefore, we have decided to characterize the performance of these estimators by calling on extra robust measures. Specifically, median is used instead of mean value, and median absolute deviation (MAD) instead of standard deviation.
Estimates of signal-independent noise component variance are biased for all four estimators. The absolute value of the bias is the smallest for the and methods (i.e., less than 6%). It increases to more than 30% for the ASRPFR method and to more than 12% for the ClipPoisGaus_stdEst2D. The estimates of the signal-dependent noise component variance are practically unbiased for the method, negatively biased by about 6% for the NI+fBm method, and exhibit a positive bias less than for the ASRPFR and ClipPoisGaus_stdEst2D methods.
It is worth highlighting that the method provides the best estimation accuracy on both signal-independent and -dependent components. More precisely, with regard to the standard deviation of the noise variance estimates (specified in Table 2), it outperforms the method by approximately 1.25 to 2.6 times, the ASRPFR method by 3.6 to 10 times, and by an even greater degree for the ClipPoisGaus_stdEst2D. We explain the reduced performance of the estimator with regard to the one by the sensitivity of the former to errors on Hurst exponent estimation. A second possible reason could be deviations of real image texture from the assumed fBm-model.
It is important to mention here that both ASRPFR and ClipPoisGaus_stdEst2D have been applied to NED2012 images without normalization [Eq. (13)] for quadratic term compensation. The reason for this is that such normalization operates at the SW level and depends on image partitioning during processing. We had no opportunity to modify the original implementation of ASRPFR and ClipPoisGaus_stdEst2D to take this into account. Therefore, in an additional experiment, we quantified the noncompensated term influence on (Table 2). Overall, it led to negative bias of signal-independent component variance and positive bias of signal-dependent component variance. For red and green channels, this additional bias was negligible and does not exceed 5% in magnitude. For the blue channel, this bias was more significant, with a magnitude of about 15%. We thus believe this is evidence that the noncompensated term cannot explain the decreased performance of ASRPFR and ClipPoisGaus_stdEst2D.
In Fig. 10, we detail the signal-independent and -dependent noise variance estimates obtained with the best estimator on each component (red, green, and blue) of all images from the NED2012 database. For most of the images, very accurate estimates were obtained. For these images, the potential estimation accuracy on the signal-independent noise component variance is about 0.2–1, and it is about on the signal-independent noise component variance. But for some of the images, namely 13, 15, 16, 22, and 23, an increased estimation error was observed. This is reflected by a corresponding increase in the potential estimation accuracy for signal-independent and -dependent noise components to about 2 and , respectively.
Let us now assess the efficiency of the analyzed estimators with regard to the diagonal terms of the correlation matrix of coefficient vector defined above by Eq. (12). For this purpose, two normalized errors, one for and another one for , respectively, are to be considered:
Note that for an efficient estimator, both and should be distributed close to the normal distribution . The statistical efficiency of the analyzed estimators with regard to these bounds can be estimated as14) over all available estimates.
Figure 11 displays and pdfs, respectively, for the best method (with ). The theoretical pdf is added for comparison purposes. Efficiencies for both noise components are shown in Table 3 for four different window sizes: , 9, 11, and 13.
Statistical characteristics of normalized estimates for the NED2012 database (tree channels).
|Estimator||Channel||Bias, %||STD(σ^n.SI2)||e^||Bias, %||STD(σ^n.SD2)||e^|
|NI+DCT||Signal-independent component||Signal-dependent component|
Bold values indicates values with the lowest bias magnitude and STD.
As is shown, the estimator exhibits rather high efficiency for different SW sizes, with a value of about 10 for both noise components. Note that for the estimator, the corresponding efficiency (not included in Table 3) is about 2%, and it is less than 0.1% for both the ASRPFR and ClipPoisGaus_stdEst2D estimators. More detailed analysis shows that the performance of the estimator is highest for and 11. It is slightly degrading for and the degradation is much more significant for . This can be explained by the influence of two factors compensating each other. On one side, the accuracy of texture parameter estimation reduces for smaller sized SWs. This factor is responsible for the decline in performance for . On the other hand, the image texture becomes more heterogeneous for larger windows, making the fBm model less adequate. This factor is responsible for the slight decrease in performance for . Taking into account that the processing time is increasing fast with the SW size, we suggest as the best setting.
It is important to note that even for the estimator, there exists an essential gap between the current level of accuracy and potential accuracy . This gap can be attributed equally to inherent shortcomings of the estimator and to the bound itself. For example, the latter does not properly take into account errors associated with Hurst exponent interpolation from TI map to NI map. Further research needs to be undertaken to reduce this gap and to better assess the potential of blind noise parameter estimation.
The problem of parameter estimation of signal-dependent noise has been considered in this paper. It has been shown that distribution of FI with regard to noise standard deviation over the available image intensity range is mainly responsible for the accuracy of signal-dependent noise parameter estimation. This is in contrast to signal-independent noise, for which estimation accuracy is defined by overall FI. This feature has been utilized to extend the and estimators previously proposed by the authors to the case of signal-dependent noise.
The performance of the and estimators has been assessed using the newly developed NED2012 database, with real noise originating from the CCD sensor. True values of noise parameters in images from the NED2012 database have been found via the calibration procedure.
For images from the NED2012 database, the proposed and estimators are shown to notably outperform the recently published ASRPFR and ClipPoisGaus_stdEst2D estimators. Between these two estimators, is considerably more effective than in terms of bias and variance of noise parameter estimates.
Overall estimation accuracy of signal-dependent noise parameters for images of the NED2012 database is high: estimates are slightly biased most of the time, with a bias value of less than 2% and a standard deviation of about 10% to 25% with regard to true noise parameter value. However, for some images with complex content, component-wise processing fails to provide sufficiently accurate results.
The distinctive feature of our approach is its ability to calculate the potential estimation accuracy of signal-dependent noise parameters for a given noisy image. We have found that there exists an essential gap between the obtained accuracy with the best estimator and the potential estimation accuracy thus provided ( statistical efficiency is about 10%). Such a gap indicates the necessity of further research in this area to design more efficient estimators and to obtain a more accurate lower bound on the performance of such estimators.
In general, the proposed estimation algorithms (preferably the estimator) can be used for blind evaluation of an arbitrary polynomial dependency of the noise standard deviation on image intensity. Such experiments for second-order polynomials have been successfully carried out. In all cases, the potential estimation accuracy of polynomial coefficients can be obtained.
Here, we would like to thank S. Abramov for passing us results for the ASRPFR estimator and A. Foi for discussions, assistance and making implementation of his method available at http://www.cs.tut.fi/~foi/sensornoise.html. This work has been partly supported by the French-Ukrainian program Dnipro (PHC DNIPRO 2013, Projet No. 28370QL).
Mykhail L. Uss graduated from National Aerospace University (Ukraine) in 2002 and got a diploma with honor in radio engineering. Since then, he has been with the Department of Aircraft Radioelectronic Systems Design of National Aerospace University. He obtained the Candidate of Technical Science degree in DSP for remote sensing from National Aerospace University (Ukraine) in 2006, and then a PhD from the University of Rennes in France in 2011. Currently, he is head of the Department of Aircraft Radioelectronic Systems Design. His research interests include the statistical theory of radio-technical systems, digital signal/image processing, blind estimation of noise characteristics, and theory of fractal sets with applications to image processing and remote sensing.
Benoit Vozel obtained the State Engineering degree and the MSc degree in control and computer science from École Centrale de Nantes (France) in 1991, and then a PhD from the same university in 1994. As a PhD student, he was with the Signal Processing Group at Institut de Recherche en Communication et Cybernétique de Nantes, where he worked on the detection of abrupt changes in signals. Since 1995, he has been with the Engineering School of Applied Sciences and Technology, University of Rennes in France, where he is currently working in the Signal and Multicomponent/Multimodal Image Processing Laboratory. His research interests generally concern blind estimation of noise characteristics, image filtering and restoration, and adaptive image and remote sensing data processing.
Vladimir V. Lukin graduated from Kharkov Aviation Institute (now National Aerospace University, Ukraine) in 1983 and got a diploma with honors in radio engineering. Since then, he has been with the Department of Transmitters, Receivers, and Signal Processing of National Aerospace University. He defended the thesis of Candidate of Technical Science in 1988 and the thesis of Doctor of Technical Science in 2002 in DSP for remote sensing. Since 1995, he has been in cooperation with Tampere University of Technology. Currently, he is the department’s vice chairman. His research interests include digital signal/image processing, remote sensing data processing, and image filtering and compression.
Kacem Chehdi received the PhD and “Habilitation à diriger des recherches” degrees in signal processing and telecommunications from the University of Rennes in France in 1986 and 1992, respectively. He is currently with the University of Rennes, where he was an assistant professor from 1986 to 1992, has been a professor of signal and image processing with the Engineering School of Applied Sciences and Technology since 1993, has been the head of the Signal and Multicomponent/Multimodal Image Processing Laboratory (TSI2M) since 2004, and was the head of the Analysis Systems of Information Processing Laboratory from 1998 to 2003. His research interests include adaptive processing at every level in the pattern recognition chain by vision, blind restoration, and blind filtering (in particular, the identification of the physical nature of image degradations and the development of adaptive algorithms), and segmentation and registration topics (in particular, the development of unsupervised, cooperative, and adaptive systems). The main applications currently under investigation are multispectral and hyperspectral image processing and analysis.