Image informative maps for component-wise estimating parameters of signal-dependent noise

. We deal with the problem of blind parameter estimation of signal-dependent noise from mono-component image data. Multispectral or color images can be processed in a component-wise manner. The main results obtained rest on the assumption that the image texture and noise parameters estimation problems are interdependent. A two-dimensional fractal Brownian motion (fBm) model is used for locally describing image texture. A polynomial model is assumed for the purpose of describing the signal-dependent noise variance dependence on image intensity. Using the maximum likelihood approach, estimates of both fBm-model and noise parameters are obtained. It is demonstrated that Fisher information (FI) on noise parameters contained in an image is distributed nonuniformly over intensity coordinates (an image intensity range). It is also shown how to find the most informative intensities and the corresponding image areas for a given noisy image. The proposed estimator benefits from these detected areas to improve the estimation accuracy of signal-dependent noise parameters. Finally, the potential estimation accuracy (Cramér-Rao Lower Bound, or CRLB) of noise parameters is derived, providing confidence intervals of these estimates for a given image. In the experiment, the proposed and existing state-of-the-art noise variance estimators are compared for a large image database using CRLB-based statistical efficiency criteria.


Introduction
A challenging problem of blind estimation of inherent sensor noise parameters [mainly its variance or standard deviation (STD)] from image data has been extensively studied by researchers for the last decade (see Ref. 1 and references therein).[4][5][6][7][8][9][10][11][12][13][14][15] Sensor noise must be detected and quantified prior to the majority of subsequent image processing tasks.Such information can help to properly select a suitable technique or adjust a method parameter to a current noise level (unknown in advance), with the final goal of making these techniques operate well enough.8][19][20][21][22][23][24][25][26][27][28] A threshold that is a function of noise standard deviation can be used with an edge detector. 13n Refs.29 and 30, a method is proposed for estimating the denoising bounds for nonlocal filters from a noisy image, where noise statistics are to be known or accurately preestimated from the same noisy data.Similar results for local filters were obtained in Ref.31, with the same requirement for noise statistics.
A signal-independent spatially uncorrelated noise model was the first possibility widely considered in the literature for modeling sensor noise in a very large number of image processing applications.In these applications, the noise is typically assumed as a zero mean stationary Gaussian distributed random process.Such process is fully described in terms of second-order statistics: its variance or standard deviation and a two-dimensional (2-D) Dirac delta function for its spatial autocorrelation function.For such noise models, the methods designed for estimating noise standard deviation can be roughly divided into two groups: the methods operating in the spatial domain and those operating in the spectral domain.Spatial methods, also called homogeneous area (HA) methods, 32 make an essential use of image homogeneous areas characterized by a negligible level of texture spatial variation compared to the noise level.Spectral methods utilize a suitable orthogonal transform to better separate image texture and noise; the former is assumed to be smoother compared to image noise. 33They can have certain advantages compared to spatial methods, as the latter also can be applied to nonintensive texture, in addition to homogeneous areas.In Refs.15, 34-36, the estimation of additive noise standard deviation is performed by analyzing the observed data in blocks of fixed size in the discrete cosine transform (DCT) domain.Estimation using wavelet transform is also possible. 37,38owever, with advances in CCD-sensor technology, the applicability of the signal-independent noise model is diminishing, and signal-dependent photonic noise is becoming more and more dominant. 14,39Nevertheless, the level of signal-independent thermal noise remains nonnegligible.Then, one has to deal with mixed signal-independent and signaldependent noise, which, in general, can also be treated as signal-dependent.Such noise is intrinsically nonstationary, but it can be locally approximated by stationary additive noise, with variance being a function of image intensity.In this case, the dependence of local variance on image intensity (called noise level function, or NLF, in Ref. 10) is of interest.If this dependence can be approximated by a polynomial, then one has to estimate the parameters (coefficients) of such a polynomial.For CCD-sensor noise, a first-order polynomial is considered to be a proper model 40 characterized by two parameters, later referred to as signal-independent and signal-dependent component variances.
It is important to mention that signal-dependent noise is significantly more restrictive compared to signal-independent noise: homogeneous areas of sufficient size, with intensities covering the whole image intensity range, are needed to accurately estimate noise variance as a function of image intensity.This problem was addressed, for example, in Ref. 10, where a priori information on an estimated nonlinear NLF was used to compensate for the lack of homogeneous areas and to stabilize the estimation process.Note that such a priori information is not available for all sensors.In this situation, the estimation of noise parameters (coefficients of a polynomial and signalindependent and signal-dependent component variances) should be performed in a blind manner, relying only on observed noisy data.To reach high performance in such a situation, a blind noise parameter estimator should satisfy the following requirements: 1 1.To provide unbiased estimates with as little variance as possible.2. To perform well enough at different noise levels.3. To be insensitive to image content; i.e., to provide appropriate accuracy, even for textural images.
By saying "with as little variance as possible," we mean that there is a certain theoretical limit on the estimation accuracy (in terms of standard deviation) of noise parameters from image data. 15It is desirable for an estimator to perform close to this limit.The true image content is certainly nonuniform and may include different texture patterns, edges, objects of small size, etc.All these heterogeneities should not essentially influence the performance of a noise parameter estimator; i.e., estimated bias and standard deviation should not increase significantly (over a theoretical limit).
Potentially, the performance of blind methods can be very high-certainly higher than that of a human operator.The reason is that these methods can use subtler differences between image content and noise, which might be not visible to the human eye.But satisfying the above requirements altogether (and reaching a high level of estimation accuracy) for signal-dependent noise has been a difficult problem. 12 Extended versions of approaches, originally proposed for the estimation of signal-independent noise standard deviation, have been considered to deal with signaldependent noise, among them the well-known scatterplot approach. 39Recall that a scatterplot is a collection of points, each representing image local variance versus local mean.To reduce the influence of outliers, a robust fit to the scatterplot data points has typically been considered by different authors (e.g., Refs.8, 10, and 39).Unfortunately, robust fit in the presence of a large percentage of abnormal local variance estimates (obtained from heterogeneous fragments) may appear very unstable.As a result, different fit strategies can lead to notably different estimation results. 12o meet all the requirements discussed above, we extend here the promising approach that we recently proposed in Ref. 15 to estimate the standard deviation (or variance) of signal-independent noise (assumed to be spatially uncorrelated).Our approach introduces two specific maps: the image texture informative (TI) map and the noise informative (NI) map.These two maps are complementary; i.e., for a given image, each scanning window (SW) belongs to one or another map.Assigning a given SW to one of these maps is decided based on the Fisher information (FI) on image texture parameters and noise standard deviation contained in this single window.In the NI map, a large amount of FI on the noise standard deviation is contained in each selected SW forming this map.Such SWs contribute in solving the main task, i.e., the accurate blind estimation of noise parameters.On the contrary, in the TI map, a large amount of FI on image texture parameters is contained in corresponding SWs.Such SWs are involved in solving an auxiliary task (i.e., the estimation of image texture parameters).
Note that both tasks are not mutually exclusive since texture parameters for NI SWs and noise parameters for TI SWs remain unknown, due to mutual masking of texture and noise.Therefore, the core of our approach is to use both maps simultaneously in an iterative manner.Noise parameters are estimated from NI SWs and then are applied to improve the estimation accuracy of texture parameters from neighboring TI SWs.The latter parameters, in turn, are used to improve the estimation accuracy of noise parameters.
To implement this scheme, we carry out the maximum likelihood (ML) estimation of a parametrical 2-D fractal Brownian motion (fBm) model, selected for describing locally the texture of a 2-D noisy image SW.The FI on the estimated parameter vector (including fBm-model parameters and noise standard deviation) can then be derived.To obtain the final estimation of noise standard deviation from the NI map, two alternatives, either fBm-or DCT-based estimators (NI þ fBm, NI þ DCT), are proposed.The first one performs direct parametrical ML estimation of noise standard deviation from NI SWs, with a texture correlation matrix defined according to the fBm model and parameters derived from neighboring TI SWs.The second one applies DCT transform to NI SWs and uses only a fixed and limited number of high-frequency DCT coefficients to estimate noise standard deviation.Now, under signal-dependent noise hypothesis, the main difficulty is to take into account the variation of noise standard deviation with regard to image intensity (due to the signal-dependent noise component), while still resorting to image NI/TI maps.For this purpose, we analyze how image NI SWs are distributed over the intensity range.This distribution allows the finding of narrow intensity intervals where noise variance can be accurately estimated and the discarding of noninformative intensities (those without any NI SW detected).Final estimates of noise signal-independent/dependent component variances are obtained by linear fit applied to these accurate variance estimates localized with regard to intensity.The whole procedure ensures the absence of outliers among local estimates of noise variance.Therefore, robust fit procedures that can be unstable are no longer needed.This paper is organized as follows.Section 2 briefly introduces the fBm model and then recalls NI þ fBm and NI þ DCT signal-independent noise variance estimators that we previously proposed.Later in this section, the signal-dependent noise parameter estimation problem is introduced from an informational point of view, including FI distribution over the available intensity range for a given image.In Sec. 3, NI þ fBm and NI þ DCT estimators are extended for signal-dependent noise and the potential accuracy of such noise parameter estimates is provided.In Sec. 4, the performance of NI þ fBm and NI þ DCT estimators is comparatively assessed against two other modern methods in a large image database and real noise from a CCD sensor.Finally, in the last section, conclusions are offered.
In spatial terms, the Hurst exponent H describes fBmtexture roughness (H → 0 for rough texture, H → 1 for smooth), 43 and σ x describes fBm amplitude.
By yðt; sÞ, t ¼ 1: : : N c , and s ¼ 1: : : N r , we denote a single-component image with N c columns and N r rows (for multi-component or color images, the approach proposed here should be applied component-wise).Suppose that the image yðt; sÞ is affected by signal-dependent noise according to the following model: yðt; sÞ ¼ xðt; sÞ þ n½t; s; xðt; sÞ; (2) where xðt; sÞ is the original noise-free image, n½t; s; xðt; sÞ is an ergodic Gaussian noise with zero mean, signal-dependent variance σ 2 n ½xðt; sÞ and a 2-D Dirac delta function for its spatial autocorrelation function.For a general case, for signaldependent noise variance σ 2 n ðIÞ, we assume a polynomial model with degree n p : σ 2 n ðI; cÞ ¼ c ⋅ ½1; I; I 2 ; : : : ; I n p , where I is true image intensity and c ¼ ðc 0 ; c 1 ; : : However, in the section of this paper that describes the experiment, we concentrate on a reduced-order noise model for recent CCD sensors.In this case, the noise n½t; s; xðt; sÞ is supposed to be the sum of two components: n½t; s; xðt; sÞ ¼ n SI ðt; sÞ þ n SD ½t; s; xðt; sÞ.The first one, n SI ðt; sÞ, is signal-independent with variance σ 2 n:SI , and the second one, n SD ½t; s; xðt; sÞ, is signal-dependent with variance σ 2 n:SD ⋅ I (Poisson-like noise).This corresponds to a polynomial model of the first order (n p ¼ 1) and, thus the coefficient vector simply reduces to c ¼ ðσ 2 n:SI ; σ 2 n:SD Þ: This structure used the fBm model and both image NI and TI maps.In the estimation process, a processed sample corresponds to a noisy textural fragment or SW of the image.The sample is described by a statistical parametrical model.The model parameters vector includes both texture parameters (fBm-model parameters) and signal-independent noise variance σ 2 n:SI .After initialization (stage 1), NI/TI maps and noise standard deviation are simply assumed to be equal to initial guesses or current refined estimates in stage 2. At this stage, texture parameters are estimated for each TI SW first.Then, these estimates act as texture parameter estimates in neighboring NI SWs.This stage results in estimates of the parameter vectors for all SWs for further use in stage 3.
In stage 3, those SWs that can be efficiently used for estimating either noise or texture parameters are identified.We propose to perform this task by considering the corresponding FI on involved parameters or, equivalently the Cramér-Rao Lower Bound (CRLB), to each SW.By setting a proper threshold on CRLB, it becomes possible to find a subset of NI/TI SWs that provide noise/texture parameters estimates with a predefined accuracy.Finally, in stage 4, the noise standard deviation is estimated for each NI SW using either a fBm-model-based ML estimator (NI þ fBm) or DCT (NI þ DCT).Note that by using only NI SWs ensures that all individual noise standard deviation estimates in stage 4 achieve a predefined accuracy and that there are no outliers among them.As a result, a simple nonrobust estimation procedure (for example, weighted mean) is sufficient in stage 4 to obtain the final estimate σ2 n:SI .

FI Distribution over Images
When solving the estimation problem for signal-independent noise, the total amount of FI on noise standard deviation contained in all noise-informative SWs determines the potential performance of an estimator (in terms of its variance). 15For signal-dependent noise, when noise standard deviation is a function of true image intensity, the distribution of FI over the image intensity range becomes of major importance.Specifically, if all noise-informative SWs have the same intensity mean I 0 , only the value of σ n ðI 0 Þ can be estimated.This allows estimating the pure Poisson-like (σ 2 n ðIÞ ¼ σ 2 n:SD I) or multiplicative (σ 2 n ðIÞ ¼ σ 2 μ I 2 ) signal-dependent noise standard deviation by σn:SD ¼ σn ðI 0 Þ∕ ffiffiffiffi I 0 p and σμ ¼ σn ðI 0 Þ∕I 0 , respectively (with accuracy that does not depend on I 0 ).If a mixture of signal-independent noise and either Poisson-like or multiplicative noise is assumed, the best accuracy can be achieved when FI is distributed equally between two intensities, I min and I max , with I max − I min being as large as possible.When no a priori information about σ n ðIÞ is available, then FI uniformly distributed over all image intensity ranges is the best option.
Unfortunately, for a particular image, this distribution cannot be modified.What can be done, in practice, is to establish the actual distribution for a given image and to obtain the corresponding accuracy of noise parameter vector c estimation, taking into account available a priori information.This problem will be addressed next, and NI þ fBm and NI þ DCT estimators will be extended to estimate the σ n ðI; cÞ function.
The corresponding FI about the parameter vector θ ¼ ðσ x ; H; σ n Þ on texture parameters ðσ x ; HÞ and noise standard deviation ðσ n Þ was introduced in Ref. 15 for a single SW (N SW ¼ 1) as where is information on fBm-model parameters, I σ n σ n is information on signal-independent noise standard deviation, and Assume that noise-informative SWs have been detected, and assign mean intensity Īi to the ith NI SW.For the subset N SW ≥ 1 of these windows with Īi ∈ ½I − ΔI∕2; I þ ΔI∕2, we assume a signal-independent noise model with standard deviation σ n ¼ σ n ðIÞ.Then, joint FI matrix I θ ðI; ΔIÞ on extended parameter vector θ ¼ ðσ x.1 ; H 1 ; σ x.2 ; H 2 ; : : : ; where I σ n σ n :N SW ¼ P N SW j¼1 I σ n σ n :j .From the matrix I θ ðI; ΔIÞ, CRLB on σ n can be found to be the corresponding element of the inverse matrix I θ ðI; ΔIÞ −1 (marked by "n" in the lower index below): Using σ 2 σ n ðI; ΔIÞ, we define CRLB per unit intensity range ½I − 0.5; I þ 0.5 as and relative CRLB per unit intensity range as where σ σ n :rel ðI; ΔIÞ ¼ σ σ n ðI; ΔIÞ∕σ n ðIÞ.
To get better insight on the meaning of variables defined above, the possible shape of Δσ 2 σ n :rel ðIÞ and its relation to the image content are illustrated in Fig. 2. Figure 2(a) displays the green component of image 3 from the NED2012 color images database (Noise Estimation Database), later referred to as the test image.We created the NED2012 database especially for testing the noise parameter estimator.It is described in detail in Sec.4.1, later in this article.
The Δσ 2 σ n :rel ðIÞ function has been calculated for the test image, and it is displayed in Fig. 2(b).In this experiment, we fix ΔI ¼ 1 to calculate Δσ 2 σ n :rel ðIÞ.A way to calculate ΔI automatically will be described in Sec.n ðIÞ function: where I min :k ≥ I min and I max :k ≤ I max , ½I min ; I max is the available image intensity range.
As utilizing NI SWs along with either fBm-based or DCT-based noise standard deviation estimators (NI þ fBm or NI þ DCT) was proved to be efficient in Ref. 15 for signal-independent noise, we propose here to apply these two estimators to estimate σ 2 n:k in each NI SWs subset, with mean intensity falling within a given interval ½I min :k ; I max :k .
Before using Eq. ( 8), the set of intervals I k should be properly selected.This can be done by taking into account the relationship between I k and Δσ 2 σ n :rel ðIÞ.Indeed, for noninformative image intensities with high Δσ 2 σ n :rel ðIÞ, the interval ΔI k ¼ I max :k − I min :k has to be increased to provide sufficiently accurate noise standard deviation estimates.Conversely, for informative image intensities with low Δσ 2 σ n :rel ðIÞ (areas 1-4 in Fig. 2), the same accuracy can be achieved for smaller ΔI k .It is natural to estimate noise standard deviation with a predefined accuracy σ 2 σ n ðI; ΔIÞ ¼ σ 2 n:rel: max for each considered intensity interval I k ; i.e., to require where Eq. ( 9) is an extension of the thresholding procedure used in Ref. 15 to select a given SW as NI in the signal-independent noise case.Whereas this original constraint requires the signal-independent noise standard deviation to be estimated with a predefined accuracy from a single NI SW, Eq. ( 9) requires the same standard deviation estimation, but from the subset of NI SWs with mean intensities within the interval I k .According to Eq. ( 6), for an arbitrary intensity Īk ¼ I, equality in Eq. ( 9) holds for Therefore, ΔIðIÞ is just a scaled-down version of Δσ 2 σ n :rel ðIÞ.In the example considered in Fig. 2, we set σ n:rel: max ¼ 0.1.For this value, ΔIðIÞ varies from 1 to 10 in informative intensity intervals 1-4 and exceeds 200 in noninformative intervals [shown by the peaks in Fig. 2(a)].It follows that for informative intensities, noise variance can be accurately estimated from very narrow intensity intervals, enabling fine NLF analysis.
Based on Eq. ( 10), the set of intervals I k is obtained by the following procedure: ΔI is given by Eq. ( 10) for I ¼ Ī. 5. Add a new interval to the list of intervals found at previous iterations.Increase K by unit.6. Repeat this process until no new pair ð Ī; ΔIÞ can be found, and then terminate.
For each interval I k , σ2 n:k is estimated using either the NI þ fBm or the NI þ DCT estimator (under signal-independent hypothesis).At the same time, σ σ n :rel ð Īk ; ΔI k Þ is calculated for future use.The output of the proposed signaldependent noise estimator is the set of noise standard deviation estimates σ2 n:k obtained for each interval I k and the corresponding set of mean intensities Īk .Now, the maximum likelihood estimation (MLE) of coefficients vector c is obtained by where A is the Vandermonde matrix with elements , and is the potential accuracy of the estimation of σ 2 n:k .The correlation matrix of the estimate ĉ has the following form: where c 0 is the true value of c vector.In the first stage, noise is assumed to be signalindependent and an initial guess σn:SI:IG is calculated as specified in Ref.

Now let us comment on the algorithm behavior for the textured area of the test image with an intensity close to 60 (dark trees in the lower-right part of the test image).
With iterations, noise variance estimates are decreasing for this area.Consequently, the signal-to-noise ratio (measured locally) is constantly increasing.As a result, this area progressively becomes less and less noise informative [this is reflected by the increase of Δσ 2 σ n :rel ðIÞ function clearly seen in Fig. 4].Finally, in iteration 5 [Fig.4(b)], this area is excluded from the noise variance estimation process (it becomes related to the TI map).
The results obtained at final iteration confirm that, given a degraded image, Δσ 2 σ n :rel ðIÞ indicates that image areas that can provide accurate noise variance estimation and their localization with regard to image intensity.It can be clearly seen that the estimated curve σ 2 n ðI; ĉÞ is very close to noise variance σ 2 n ðI; c 0 Þ obtained by the calibration procedure.For the considered test image, there are enough NI intensities to provide a high estimation accuracy of signal-dependent noise variance.For images with not enough NI (homogeneous) areas for estimating noise parameters, σ 2 n ðI; ĉÞ may become inaccurate.Based on our experiments, we have found that this extreme case can always be detected by checking the accuracy of ĉ at the current iteration or the entries of the σ 2 c matrix [see Eq. ( 12) for details].
4 Experimental Results Using the NED2012 Database This section deals with the application of the two designed estimators, NI þ fBm and NI þ DCT, of signal-dependent noise parameters to the NED2012 database of images that have been corrupted by signal-dependent noise.Estimator performance is analyzed and compared to that of the recently developed Automatic Scatterplot Reference Points Fitting with Restrictions (ASRPFR) method (discussed in Ref. 12) and the ClipPoisGaus_stdEst2D method published in Ref. 9.
Fig. 4 Illustration of the signal-dependent noise estimation process by using the algorithm in Fig. 3 for the test image.
The primary goal of this study is to determine the potential accuracy of signal-dependent noise parameter estimation (variance of signal-independent and -dependent components) from real images of different types, and to compare the performance of existing estimators to this bound.

Image Database for Testing Signal-Dependent
Noise Estimation Algorithms: NED2012 A key item in testing noise variance estimators is the availability of images with known noise parameters.Artificial noise-free images with synthetic noise raise questions about the applicability of the obtained results to the real images corrupted by real noise.Another possibility lies in using real-life images with a low level of noise, with synthetic noise added for test purposes.In this area, the TID2008 44 database has been extensively used. 45However, we have identified four drawbacks of TID2008 that prevent us from exploiting it in our study: 1. Restricted image size.Indeed, the size of the images in the TID2008 database is 512 × 384 pixels.This corresponds to ≈0.2 Mpx.However, modern digital cameras typically have more than 10 Mpx sensors.2. Images from the TID2008 database are in 8-bit representation, while 12 or 14 bits is typical currently for images stored in raw format (for both digital cameras and remote sensing acquisition systems).3. TID2008 images are subject to the demosaicing procedure to convert them from a color filter array (CFA) to RGB representation.Demosaicing unfortunately affects the spectral properties of both image texture (due to smoothing) and inherent noise (because it becomes spatially correlated).It is highly preferable to deal with CFA representation to assess data directly from a camera sensor.4. TID2008 images contain inherent noise that, strictly speaking, does not allow to consider them as noisefree. 15This noise variance cannot be accurately estimated due to issues 1 and 3. Automatic analysis shows that its variance is about 4, 15 but manual analysis shows it can locally reach 4 to 10.Such values are critical for our situation, as the standard deviation of the estimation error of additive noise variance provided by the NI þ DCT estimator (as we will show next) can be as low as 0.2.
To overcome these drawbacks, we have decided to base our study on 12-bit raw images from the Nikon D80 DSLR camera with a 10.2-Mpx CCD sensor.No extra noise was generated; we only dealt with the parameter estimation for noise that was originally present in D80 images.Our main assumption is that the noise parameters remain the same with time and camera operational conditions.Absence of noise spatial correlation is also assumed.Our experiments have shown no violation of these assumptions at the attained level of accuracy.
To accurately estimate true noise parameters of Nikon D80 sensor for ISO100, we have used the following semiautomatic calibration procedure: 1.A series of 17 images of a white sheet of paper was taken in raw Nikon electronic file format.For these images, we had a fixed International Standardization Organization (ISO) value (equal to 100), as well as other camera settings (using a manual regime), except for shutter speed.By selecting different shutter speeds, a full image intensity range (in 12 bits) was covered.
To suppress image texture (due to paper surface), strongly defocused images were collected.Then paper texture was smoothed, leaving sensor noise unaffected.2. This series of images was partitioned into non overlapping 8 × 8 SWs.A 2-D DCT transform was then applied to each window.The highest 16 coefficients (with indices from 5 to 8 for both dimensions) from each window were stored for further processing.By relying on these high-frequency coefficients, we additionally diminished the influence of image texture.
For each such group of 16 coefficients, we calculated the image mean intensity in the corresponding SW. 3. The available intensity range from 0 to 4098 (12 bits)  was divided into narrow intervals.DCT coefficients were grouped according to their corresponding mean intensities.In this manner, for each kth intensity interval, N DCT:k DCT coefficients were collected.The sample variance of these N DCT:k coefficients, σ2 n:k , was an estimate of the signal-dependent noise variance in k'th intensity interval.4. The coefficient vector c was obtained by Eq. ( 11), with Visual analysis of noise variance dependence on image intensity shows notable deviation from the theoretic linear shape (first-order polynomial), especially for the blue component (Fig. 5).To take this into account, the order of the approximation polynomial was set to n p ¼ 2. The obtained estimates are given in Table 1 (we call them calibration lines).The noise parameters obtained via the calibration procedure will be marked with index "0" below.
It can be seen that the green channel is the least noisy one, followed by the red and blue channels.Quadratic terms for all channels are nonzero.These values are statistically significant (more than 0.704 • 10 −5 ∕0.04924 • 10 −5 or 14.3 sigma) and cannot be neglected.They probably appear due to the internal regulations of the camera.
For testing the two proposed estimators, we selected 25 D80 images taken from the same camera during a two-year interval and organized them into the NED2012 database.The database includes images with different content.Some of them have large homogeneous areas (e.g., sky), while others are quite textural, and defocused areas are present (Fig. 6).All images are presented as a CFA array of 2611× 3900 pixels in size.Red, green, and blue channel data were extracted from the CFA array by subsampling.Images have different ISO values (from 100 to 320), different shutter speeds (from 1∕1250 s to 1∕30 s), different apertures (from f/4 to f/14) and different focal lengths (from 18 mm to 135 mm).From this set of parameters, it is the ISO parameter that directly affects noise parameters for both signal-independent and -dependent components.In order to compensate for this influence and convert all images to the reference ISO100, we have simply normalized each image by a factor of 100/ISO before processing.
We will first prove that estimation results obtained by the NI þ DCT method agree with the calibration data shown above.For this goal, we applied the NI þ DCT method to all images from the NED2012 database simultaneously by processing 1,500 SWs of 9 × 9 pixels randomly selected from each NED2012 image (a total of 1; 500 × 25 ¼ 37; 500 windows).In this manner, very high estimation accuracy of the c 0 , c 1 , and c 2 coefficients can be reached.Estimation results for all three channels are shown in Table 1 (the NI þ DCT line).Figure 7 illustrates these results for the blue channel.
One can clearly see that there is no statistically significant difference between estimates of noise parameters obtained for two different datasets (the NED2012 database and the set of 17 calibration images) by the calibration procedure and the NI þ DCT method (they differ by less than 4σ).It is worth noting that the accuracy provided by the NI þ DCT method is only slightly worse than the one obtained by the calibration procedure.In this test, the NI þ DCT method shows its potential ability to deal with signaldependent noise that is more complex than mixtures of additive and Poisson/multiplicative noises.
In the next experiment, we restricted ourselves to the case of a mixture of signal-independent and Poisson-like signaldependent noises.The overall noise variance is thus signaldependent: σ 2 n ðIÞ ¼ σ 2 n:SI þ I ⋅ σ 2 n:SD .As was shown above, the noise in the Nikon D80 images does not strictly follow this linear model.Therefore, in order to nullify quadratic term c 2 before estimation, we normalized the image intensity in each SW I SW by where ĪSW is the mean of I SW and c 0 , c 1 , and c 2 are taken from Table 1.After such normalization, noise parameters become as specified in Table 1, with c 2 ¼ 0.

Accuracy Analysis of Signal-Dependent Noise Parameter Estimation
The considered set of four estimators, NI þ fBm, NI þ DCT, ASRPFR, and ClipPoisGaus_stdEst2D, were applied to each of the 25 images of the NED2012 database in a componentwise manner (red, green, and blue components were processed independently).A SW of 9 × 9 pixels in size was selected for both NI þ DCT and NI þ fBm.The choice of this particular SW size is justified in this subsection.For each component, the two noise variance components σ2 n:SI and σ2 n:SD were estimated.Overall, 25 estimates were obtained for each channel.Their empirical probability density functions (pdfs), pdf ( σ2 n:SI ) and pdf ( σ2 n:SD ), are shown in  Figs. 8 and 9, respectively.The main statistical characteristics of these estimates [mean Mð⋅Þ, bias and standard deviation STDð⋅Þ] are given in Table 2.The bias measure is calculated as bias ¼ 100%½Mð âÞ − a 0 ∕a 0 , where a 0 is the true value of a. Preliminary conclusions can be drawn from these results.Among these four estimators, ASRPFR and ClipPoisGaus_stdEst2D provide worse performance than the NI þ fBm and NI þ DCT with regard to both signalindependent and -dependent components.The main factor that degraded the performance of the ASRPFR and ClipPoisGaus_stdEst2D estimators is the significant number of outliers.However, they form a pronounced mode in the vicinity of the true value of signal-independent and -dependent noise component variances anyway.Therefore, we have decided to characterize the performance of these estimators by calling on extra robust measures.Specifically, median is used instead of mean value, and median absolute deviation (MAD) instead of standard deviation.
Estimates of signal-independent noise component variance are biased for all four estimators.The absolute value of the bias is the smallest for the NI þ DCT and NI þ fBm  methods (i.e., less than 6%).It increases to more than 30% for the ASRPFR method and to more than 12% for the ClipPoisGaus_stdEst2D.The estimates of the signal-dependent noise component variance are practically unbiased for the NI þ DCT method, negatively biased by about 6% for the NI+fBm method, and exhibit a positive bias less than 25% for the ASRPFR and ClipPoisGaus_stdEst2D methods.
It is worth highlighting that the NI þ DCT method provides the best estimation accuracy on both signal-independent and -dependent components.More precisely, with regard to the standard deviation of the noise variance estimates (specified in Table 2), it outperforms the NI þ fBm method by approximately 1.25 to 2.6 times, the ASRPFR method by 3.6 to 10 times, and by an even greater degree for the ClipPoisGaus_stdEst2D.We explain the reduced performance of the NI þ fBm estimator with regard to the NI þ DCT one by the sensitivity of the former to errors on Hurst exponent estimation.A second possible reason could be deviations of real image texture from the assumed fBm-model.
It is important to mention here that both ASRPFR and ClipPoisGaus_stdEst2D have been applied to NED2012 images without normalization [Eq.( 13)] for quadratic term c 2 compensation.The reason for this is that such normalization operates at the SW level and depends on image partitioning during processing.We had no opportunity to modify the original implementation of ASRPFR and ClipPoisGaus_stdEst2D to take this into account.Therefore, in an additional experiment, we quantified the noncompensated c 2 term influence on NI þ DCT (Table 2).Overall, it led to negative bias of signal-independent component variance and positive bias of signal-dependent component variance.For red and green channels, this additional bias was negligible and does not exceed 5% in magnitude.For the blue channel, this bias was more significant, with a magnitude of about 15%.We thus believe this is evidence that the noncompensated c 2 term cannot explain the decreased performance of ASRPFR and ClipPoisGaus_stdEst2D.
In Fig. 10, we detail the signal-independent and -dependent noise variance estimates obtained with the best NIþ DCT estimator on each component (red, green, and blue) of all images from the NED2012 database.For most of the images, very accurate estimates were obtained.For these images, the potential estimation accuracy on the signal-independent noise component variance is about 0.2-1, and it is about 0.8 − 2.2 • 10 −3 on the signal-independent noise component variance.But for some of the images, namely 13, 15, 16, 22, and 23, an increased estimation error was observed.This is reflected by a corresponding increase in the potential estimation accuracy for signal-independent and -dependent noise components to about 2 and 8 × 10 −3 , respectively.
Let us now assess the efficiency of the analyzed estimators with regard to the diagonal terms ðσ 2 σ n:SI ; σ 2 σ n:SD Þ of the correlation matrix σ 2 c of coefficient vector c defined above by Eq. (12).For this purpose, two normalized errors, one for σ2 n:SI and another one for σ2 n:SD , respectively, are to be considered: where XX is the noise component label (either SI or SD), and the sum for each estimator is calculated in Eq. ( 14) over all N e ¼ 75 available estimates.Figure 11 displays σ2 n:SI:norm and σ2 n:SD:norm pdfs, respectively, for the best NI þ DCT method (with N ¼ 9).The theoretical pdf Nð0; 1Þ is added for comparison purposes.Efficiencies ê for both noise components are shown in Table 3 for four different window sizes: N ¼ 7, 9, 11, and 13.
As is shown, the NI þ DCT estimator exhibits rather high efficiency for different SW sizes, with a value of about 10 for both noise components.Note that for the NI þ fBm estimator, the corresponding efficiency (not included in Table 3) is about 2%, and it is less than 0.1% for both the ASRPFR and ClipPoisGaus_stdEst2D estimators.More detailed analysis shows that the performance of the NI þ DCT estimator is highest for N ¼ 9 and 11.It is slightly degrading for N ¼ 13 and the degradation is much more significant for N ¼ 7.This can be explained by the influence of two factors compensating each other.On one side, the accuracy of texture parameter estimation reduces for smaller sized SWs.This factor is responsible for the decline in performance for N ¼ 7. On the other hand, the image texture becomes more heterogeneous for larger windows, making the fBm model less adequate.This factor is responsible for the slight  decrease in performance for N ¼ 13.Taking into account that the processing time is increasing fast with the SW size, we suggest N ¼ 9 as the best setting.
It is important to note that even for the NI þ DCT estimator, there exists an essential gap between the current level of accuracy and potential accuracy σ 2 c .This gap can be attributed equally to inherent shortcomings of the NI þ DCT estimator and to the bound σ 2 c itself.For example, the latter does not properly take into account errors associated with Hurst exponent interpolation from TI map to NI map.Further research needs to be undertaken to reduce this gap and to better assess the potential of blind noise parameter estimation.

Conclusions
The problem of parameter estimation of signal-dependent noise has been considered in this paper.It has been shown that distribution of FI with regard to noise standard deviation over the available image intensity range is mainly responsible for the accuracy of signal-dependent noise parameter estimation.This is in contrast to signal-independent noise, for which estimation accuracy is defined by overall FI.This feature has been utilized to extend the NI þ fBm and NI þ DCT estimators previously proposed by the authors to the case of signal-dependent noise.
The performance of the NI þ fBm and NI þ DCT estimators has been assessed using the newly developed NED2012 database, with real noise originating from the CCD sensor.True values of noise parameters in images from the NED2012 database have been found via the calibration procedure.
For images from the NED2012 database, the proposed NI þ DCT and NI þ fBm estimators are shown to notably outperform the recently published ASRPFR and ClipPoisGaus_stdEst2D estimators.Between these two estimators, NI þ DCT is considerably more effective than NI þ fBm in terms of bias and variance of noise parameter estimates.
Overall estimation accuracy of signal-dependent noise parameters for images of the NED2012 database is high: estimates are slightly biased most of the time, with a bias value of less than 2% and a standard deviation of about 10% to 25% with regard to true noise parameter value.However, for some images with complex content, component-wise processing fails to provide sufficiently accurate results.
The distinctive feature of our approach is its ability to calculate the potential estimation accuracy of signal-dependent noise parameters for a given noisy image.We have found that there exists an essential gap between the obtained accuracy with the best NI þ DCT estimator and the potential estimation accuracy thus provided (NI þ DCT statistical efficiency is about 10%).Such a gap indicates the necessity of further research in this area to design more efficient estimators and to obtain a more accurate lower bound on the performance of such estimators.
In general, the proposed estimation algorithms (preferably the NI þ DCT estimator) can be used for blind evaluation of an arbitrary polynomial dependency of the noise standard deviation on image intensity.Such experiments for second-order polynomials have been successfully carried out.In all cases, the potential estimation accuracy of polynomial coefficients can be obtained.

Acknowledgments
Here, we would like to thank S. Abramov for passing us results for the ASRPFR estimator and A. Foi for discussions, assistance and making implementation of his method available at http://www.cs.tut.fi/~foi/sensornoise.html.This work has been partly supported by the French-Ukrainian program Dnipro (PHC DNIPRO 2013, Projet No. 28370QL).

2. 2
Structure of NI þ fBm and NI þ DCT Signal-Independent Noise Variance Estimators At this point, let's briefly recall the main ideas suggested in the NI þ fBm and NI þ DCT signal-independent noise variance estimators (σ 2 n:SD ¼ 0) proposed by the authors in Ref. 15. Their generalized structure is recalled in Fig. 1.
σ x:N SW ; H N SW ; σ n Þ can be obtained in a similar way: I θ ðI; ΔIÞ ¼ I fBm:N SW I fBm:σ n :N SW I T fBm:σ n .1 : : : I T fBm:σ n :N SW I σ n σ n :N SW
3.1.Intensities of the test image cover range approximately from 0 to 1,200.In Fig. 2(b), four image intensity intervals with the lowest Δσ 2 σ n :rel ðIÞ can be seen, with image intensities around 10 (1), 200 (2), 400 (3), and 800 (4).Objects in the test image with intensities falling within intervals 1-4 are marked in Fig. 2(a) with the corresponding numbers.Interval 1 corresponds to the nonintensive dark tree texture, intervals 2 and 3 relate to homogeneous house fronts, and interval 4 relate to cloudy sky.One can see that Δσ 2 σ n :rel ðIÞ indicates image areas that provide accurate noise standard deviation estimation and their localization with regard to image intensity.3 NI FBm and NI DCT Estimators for Signal-Dependent Noise Parameters 3.1 Preliminaries of the Proposed Estimators of Polynomial SD Noise Variance While designing the signal-dependent noise parameter estimator, we assume piecewise-constant approximation to the σ 2

3. 2
Proposed Estimators of Polynomial SD Noise Variance Note that the signal-dependent noise variance estimate σ 2 n ðI; ĉÞ obtained by Eq. (11) requires availability of the values of Δσ 2 σ n :rel ðIÞ and σ σ 2 n:k , that are based on true noise variance σ 2 n ðI; c 0 Þ.Therefore, the signal-dependent noise variance estimation procedure should operate iteratively as described below.The structure of the proposed estimator for texture parameters and signal-dependent noise standard deviation is summarized in Fig. 3.In the following, we provide a description of this estimator and illustrate its behavior based on the test image [Fig.2(a)] for n p ¼ 1. True parameter vector c 0 ¼ ð5.0834; 0.1352Þ is obtained for this image in the experiment via a calibration procedure (see Sec. 4.1 of this article for details).
15.It is equal to the minimum of sample standard deviation estimates over all image nonoverlapping SWs.For the test image, the value σ2 n:SI:IG ≈ 170.60 was obtained.The initial guess for the vector c assumes the initial value ĉIG ¼ ðσ 2 n:SI:IG ; 0Þ.Texture and noise parameter estimates and NI and TI maps are updated in stage 2. In this stage, we fix noise parameters to be equal to either an initial guess ĉi¼1 ¼ ĉIG or the previously estimated value ĉi ¼ ĉi−1 (here, i defines the iteration index).The signal-dependent noise variance for each SW (both NI and TI) is calculated according to the retained polynomial noise model (a mixture of signal-independent and Poisson-like signal-dependent noises in this case) as σ 2 n ðI; ĉi Þ, substituting true image intensity by mean intensity over current SW.Then, fBm-model parameters for TI and NI SWs are estimated, and discrimination between texture/NI SWs can be refined as specified in Ref.15.Next, in stage 3, a current value of relative CRLB per unit image intensity (ΔI ¼ 1) Δσ 2 σ n :rel ðIÞ and the corresponding set of intervals I k are calculated as described above.For illustration purposes, the Δσ 2 σ n :rel ðIÞ obtained in the first iteration is shown in Fig.4(a) as a straight black line.The set of intervals I k allows the estimating of σ 2 n:k values using either the fBm-or DCT-based estimator and refining the ĉ estimate according to Eq. (11).The σ2 n:k estimates and σ 2 n ðI; ĉi Þ function are both shown in Fig.4(a) for the first iteration of the algorithm as black dots and a solid black line, respectively.The estimate σ 2 n ðI; ĉi Þ is iteratively refined by repeating stages 2-4 until convergence.The convergence of the algorithm can be observed from the comparison of Fig.4(a) with 4(b), where the first and the final (fifth) iteration estimates of Δσ 2 σ n :rel ðIÞ, σ2 n:k , and σ 2 n ðI; ĉi Þ for the test image are shown.Note that for each iteration, Fig. 4 shows σ 2 n ðI; ĉi Þ estimates at the beginning of the iteration (signal-independent noise for the first iteration), not at the end.It can be seen that σ 2 n:k estimates are concentrated at intensities where Δσ 2 σ n :rel ðIÞ takes on minimal values.Noise variance estimate σ 2 n ðI; ĉi Þ for I from 600 to 1200 does not change significantly with iterations.Conversely, for I from 0 to 300 that corresponds to areas (1) and (2) in Fig. 2(b), the noise variance estimate decreases from about 170 [the first iteration, Fig. 4(a)] to about 5 [the last iteration, Fig. 4(b)], and the number of σ2 n:k estimates decreases significantly (a smaller number of SWs is considered NI by the algorithm).

Fig. 3
Fig. 3 Generalized scheme of proposed NI þ fBm and NI þ DCT estimators for image texture and signal-dependent noise parameters.

Fig. 5
Fig.5Nikon D80 noise variance dependence on image intensity for red, green, and blue channels.

Fig. 7
Fig. 7 of NI þ DCT and calibration noise estimates for the blue channel of the NED2012 database.

Fig. 10 (
Fig. 10 (a) Signal-independent and (b) signal-dependent noise component variance according to image index i NED estimated by the NI þ DCT on red, green, and blue components of each image from the NED2012 database.The true noise parameters are shown as dashed horizontal lines.

Fig. 11
Fig. 11 Empirical pdfs of the normalized estimates of signal-independent and signal-dependent noise components variances by the NI þ DCT estimator N ¼ 9.The theoretical pdf Nð0; 1Þ is shown as a dashed black curve.

Table 2
Mean value Mð•Þ, bias and standard deviation STDð•Þ of noise variance estimates on the NED2012 database.
Bold values indicates values with the lowest bias magnitude and STD.Journal of Electronic Imaging 013019-13 Jan-Mar 2013/Vol.22(1) Uss et al.: Image informative maps for component-wise estimating parameters. . .Downloaded From: https://www.spiedigitallibrary.org/journals/Journal-of-Electronic-Imaging on 26 May 2019 Terms of Use: https://www.spiedigitallibrary.org/terms-of-use estimators with regard to these bounds can be estimated as ê ¼ 100% ⋅ N e ∕ X N e i¼1 σ2 norm:XX:i ;

Table 3
Statistical characteristics of normalized estimates for the NED2012 database (tree channels).