An observer study was run to determine the detection thresholds of several representative examples of column fixed
pattern noise, in the presence of varying levels of shot noise, which is known to mask structured noise. The data obtained
were fit well at relevant shot noise levels by a simple model based on signal detection theory. Individual metrics of fixed
pattern noise and shot noise, used in the masking equation, were computed from one dimensional integrations involving
the capture noise power spectra (mapped to CIELAB space); the modulation transfer function of the display; the display
pixel pitch; the viewing distance; and the S-CIELAB luminance contrast sensitivity function. The results of this work
can be used to predict detection thresholds that can be added to photon transfer curves for the purpose of determining
whether fixed pattern noise will be visible.
Sensor yield is directly related to the probability of defective pixel occurrence and the screening criteria.
Assuming a spatially independent distribution of single pixel defects, effective on-the-fly correction of singlepixel
defects in a color plane, and effective correction of two-pixel defects in a color plane (couplets) through
a defect map, sensor yield can be computed based on the occurrence of three adjacent defective pixels in a
color plane (triplets). Closed-form equations are derived for calculating the probability of occurrence of
couplets and triplets as a function of the probability of a single pixel being defective. If a maximum of one
triplet is allowed in a 5-megapixel sensor, to obtain a 98% yield, the probability of a pixel being defective (<i>p</i>)
must not exceed 1.3E-3 (6500 defective pixels). For an 8-megapixel sensor, the corresponding requirement
would be <i>p</i> < 1.1E-3 (8900 defective pixels). Numerical simulation experiments have confirmed the accuracy
of the derived equations.
A softcopy quality ruler study involving 12 scenes and 34 observers was performed to quantify the dependence of
quality on color saturation, in the absence of noise, with saturation measured using Imatest software. It was found that
quality falls off symmetrically with deviation of color saturation from the preferred value of about 110%, with a 20%
change in saturation reducing quality by about two just noticeable differences (JNDs). Optimization of noise versus color
saturation was investigated using (1) the aforementioned transform of color saturation to JNDs of quality; (2) a
previously published objective metric and JND transform for isotropic noise; and (3) the multivariate formalism, for
combining JNDs from independent attributes into an overall quality JNDs. As noise increases and signal to noise ratio
(SNR) decreases, the optimal color saturation decreases from the 110% position, so that there is less noise amplification
by the color correction matrix. A quality contour plot is presented, showing a region of plausible color saturation values,
as a function of SNR, for a representative use case. One example of a reasonable strategy is to provide 80% color
saturation at SNR = 5, 90% at SNR = 10, 100% at SNR = 20, and 110% at SNR 50 and above.
Proc. SPIE. 7867, Image Quality and System Performance VIII
KEYWORDS: Signal to noise ratio, Imaging systems, Image processing, Digital filtering, Denoising, Interference (communication), Gaussian filters, Image filtering, Modulation transfer functions, RGB color model
Noise Power Spectra (NPS) are traditionally measured using uniform areas of tone. Adaptive algorithms, such as noise
reduction, demosaicing, and sharpening, can modify their behavior based on underlying image structure. In particular,
noise reduction algorithms may suppress noise more strongly in perfectly uniform areas than they would in those with
modest variations, as found in actual pictorial images, and so yield unrepresentative NPS. This phenomenon would be
similar in nature to the susceptibility of high-contrast-edges to adaptive sharpening and the subsequent over-estimation
of effective pictorial modulation transfer function by some targets. Experimentation is described that examines the effect
of modern adaptive noise reduction algorithms on the NPS of images containing ramps of varying gradient. Gradients
are chosen based on a survey of consumer images from areas where noise is typically noticeable, such as blue sky, walls
and faces. Although loss in performance of adaptive noise reduction is observed as gradients increase, the effect is
perceptually small when weighted according to the frequency of occurrence of the gradients in pictorial imaging. The
significant additional complexity of measuring gradient-based NPS does not appear to be justified; measuring NPS from
uniform areas of tone should suffice for most perceptual work.
A system simulation model was used to create scene-dependent noise masks that reflect current performance of mobile
phone cameras. Stimuli with different overall magnitudes of noise and with varying mixtures of red, green, blue, and
luminance noises were included in the study. Eleven treatments in each of ten pictorial scenes were evaluated by twenty
observers using the softcopy ruler method. In addition to determining the quality loss function in just noticeable
differences (JNDs) for the average observer and scene, transformations for different combinations of observer sensitivity
and scene susceptibility were derived. The psychophysical results were used to optimize an objective metric of isotropic
noise based on system noise power spectra (NPS), which were integrated over a visual frequency weighting function to
yield perceptually relevant variances and covariances in CIE L*a*b* space. Because the frequency weighting function is
expressed in terms of cycles per degree at the retina, it accounts for display pixel size and viewing distance effects, so
application-specific predictions can be made. Excellent results were obtained using only L* and a* variances and L*a*
covariance, with relative weights of 100, 5, and 12, respectively. The positive a* weight suggests that the luminance
(photopic) weighting is slightly narrow on the long wavelength side for predicting perceived noisiness. The L*a*
covariance term, which is normally negative, reflects masking between L* and a* noise, as confirmed in informal
evaluations. Test targets in linear sRGB and rendered L*a*b* spaces for each treatment are available at
http://www.aptina.com/ImArch/ to enable other researchers to test metrics of their own design and calibrate them to
JNDs of quality loss without performing additional observer experiments. Such JND-calibrated noise metrics are
particularly valuable for comparing the impact of noise and other attributes, and for computing overall image quality.
The image structure quality resulting from several CMOS pixel structures (conventional, backside-illuminated, and
diagonally oriented) has been compared using three complimentary techniques: (1) objective measurements of noise
equivalent quanta (NEQ) as a function of spatial frequency; (2) perceptual modeling of the multivariate quality loss from
blur and noise in units of just noticeable differences (JNDs); and (3) subjective measurement with the softcopy quality
ruler, also producing results in JNDs. The results of the perceptual modeling and subjective measurement were in good
quantitative agreement. NEQ is not perceptually uniform and so could only be correlated qualitatively with the other
methods, but it was helpful in understanding how performance might vary by application, given the spatial frequencies at
which the curves crossed. The strengths and weaknesses of each approach are compared; all three have potential utility
in evaluating computational imaging systems.
ISO 20462 part 3 standardized the hardcopy quality ruler and a softcopy quality ruler based on a binary sort approach involving paired comparisons. The new softcopy ruler method described here utilizes a slider bar to match the quality of the ruler to that of the test image, which is found to substantially reduce the time required per assessment (30 to 15.5 s), with only a modest loss of precision (standard deviations of 2.5 to 2.9 just noticeable differences). In combination, these metrics implied a 20% improvement in the standard error of the mean achievable in a fixed amount of judging time. Ruler images calibrated against the standard quality scale of ISO 20462 are generated for 21 scenes, at 31 quality levels each, achieved through variation of sharpness, while other attributes are held near their preferred positions. The images are bundled with documentation and a MATLAB source code for a graphical user interface that administers softcopy ruler experiments, and these materials are donated to the International Imaging Industry Association for distribution. In conjunction with a specified large flat panel display, these materials should enable users to conduct softcopy quality ruler experiments with minimum effort, and should reduce the barriers to performing calibrated psychophysical measurements.
We describe a solution for image restoration in a computational
camera known as an extended depth of field
(EDOF) system. The specially-designed optics produce
point spread functions that are roughly invariant with object distance
in a range. However, this invariance involves a trade-off
with the peak sharpness of the lens. The lens blur
is a function of lens field-height, and the imaging sensor introduces signal-dependent noise. In this context, the principal contributions
of this paper are: a) the modeling of the EDOF focus recovery
problem; and b) the adaptive EDOF focus recovery approach, operating in signal-dependent noise.
The focus recovery solution is adaptive to complexities of an EDOF imaging system,
and performs a joint deblurring and noise
suppression. It also adapts to imaging conditions by accounting for the state of the sensor (e.g., low-light conditions).
Weighting of field heights is important in cases when a single numerical value needs to be calculated that characterizes
an attribute's overall impact on perceived image quality. In this paper we report an observer study to derive the
weighting of field heights for sharpness and noisiness.
One-hundred-forty images were selected to represent a typical
consumer photo space distribution. Fifty-three sample points were sampled per image, representing field heights of 0,
14, 32, 42, 51, 58, 71, 76, 86% and 100%. Six observers participated in this study. The field weights derived in this
report include both: the effect of area versus field height (which is a purely objective, geometric factor); and the effect of
the spatial distribution of image content that draws attention to or masks each of these image structure attributes. The
results show that relative to the geometrical area weights, sharpness weights were skewed to lower field heights, because
sharpness-critical subject matter was often positioned relatively near the center of an image. Conversely, because noise
can be masked by signal, noisiness-critical content (such as blue skies, skin tones, walls, etc.) tended to occur farther
from the center of an image, causing the weights to be skewed to higher field heights.
A softcopy quality ruler method was implemented for the International Imaging Industry Association (I3A) Camera
Phone Image Quality (CPIQ) Initiative. This work extends ISO 20462 Part 3 by virtue of creating reference digital
images of known subjective image quality, complimenting the hardcopy Standard Reference Stimuli (SRS). The
softcopy ruler method was developed using images from a Canon EOS 1Ds Mark II D-SLR digital still camera (DSC)
and a Kodak P880 point-and-shoot DSC. Images were viewed on an Apple 30in Cinema Display at a viewing distance of
34 inches. Ruler images were made for 16 scenes. Thirty ruler images were generated for each scene, representing ISO
20462 Standard Quality Scale (SQS) values of approximately 2 to 31 at an increment of one just noticeable difference
(JND) by adjusting the system modulation transfer function (MTF). A Matlab GUI was developed to display the ruler
and test images side-by-side with a user-adjustable ruler level controlled by a slider. A validation study was performed at
Kodak, Vista Point Technology, and Aptina Imaging in which all three companies set up a similar viewing lab to run the
softcopy ruler method. The results show that the three sets of data are in reasonable agreement with each other, with the
differences within the range expected from observer variability. Compared to previous implementations of the quality
ruler, the slider-based user interface allows approximately 2x faster assessments with 21.6% better precision.
The integration of novel optics designs, miniature CMOS sensors, and powerful digital processing into a single
imaging module package is driving progress in handset camera systems in terms of performance, size (thinness) and
cost. The miniature cameras incorporating high resolution sensors and fixed-focus Extended Depth of Field (EDOF)
optics allow close-range reading of printed material (barcode patterns, business cards), while providing high quality
imaging in more traditional applications. These cameras incorporate modified optics and digital processing to
recover the soft-focus images and restore sharpness over a wide range of object distances. The effects a variety of
parameters of the imaging module on the EDOF range were analyzed for a family of high resolution CMOS
modules. The parameters include various optical properties of the imaging lens, and the characteristics of the sensor.
The extension factors for the EDOF imaging module were defined in terms of an improved absolute resolution in
object space while maintaining focus at infinity. This definition was applied for the purpose of identifying the
minimally resolvable object details in mobile cameras with bar-code reading feature.
A flexible software tool was developed that combines predictive models for detector noise and blur with image
simulation and an improved human observer model to predict the clinical task performance of existing and future
radiographic systems. The model starts with high-fidelity images from a database and mathematical models of common
disease features, which may be added to the images at desired contrast levels. These images are processed through the
entire imaging chain including capture, the detector, image processing, and hardcopy or softcopy display. The simulated
images and the viewing conditions are passed to a human observer model, which calculates the detectability index d' of
the signal (disease or target feature). The visual model incorporates a channelized Hotelling observer with a luminance-dependent
contrast sensitivity function and two types of internal visual system noise (intrinsic and image background-induced).
It was optimized based on three independent human observer studies of target detection, and is able to predict
d' over a wide range of viewing conditions, background complexities, and target spatial frequency content. A more
intuitive metric of system performance, Task-Specific Detective Efficiency (TSDE), is defined to indicate how much
detector improvements would translate to better radiologist performance. The TSDE is calculated as the squared ratio of
d' for a system with the actual detector and a hypothetical system containing an ideal detector. A low TSDE, e.g., 5% for
the detection of 0.1 mm microcalcifications in typical mammography systems, indicates that improvements in the
detector characteristics are likely to translate to better detection performance. The TSDE of lung nodule detection is as
high as 75% even with the detective quantum efficiency (DQE) of the detector not exceeding 24%. Applications of the
model to system optimizations for flat-panel detectors, in mammography and dual energy digital radiography, are
A four-alternative forced-choice experiment was carried out to examine the effect of 8-bit versus 10-bit grayscale resolution on the detection of subtle lung nodules on a medical grayscale liquid crystal display (LCD). Sets of four independent backgrounds from each of three regions were derived from a very low-noise X-ray acquisition of a chest-phantom with an amorphous selenium radiographic detector. Simulated nodules of fixed diameter (10 mm) and varying contrast were digitally added to the centers of selected background images. Subsequently, multifrequency image processing was performed to enhance the image structures, followed by a tonescaling procedure that resulted in pixel values being specified as p-values, according to DICOM Part 14: The Grayscale Display Function. To investigate the effect that grayscale resolution may have upon softcopy detectability, each set of four images in the experiment was quantized to both 8-bit and 10-bit resolution. The resulting images were displayed on a DICOM-calibrated LCD display supporting up to 10 bits of grayscale input. Twenty observers with imaging expertise performed the nodule detection task for which the signal and location were known exactly. Results from all readers, chest regions, and backgrounds were pooled, and statistical significance between fractions of correct responses between 8-bit and 10-bit resolution was tested. Experimental results do not demonstrate a statistically significant difference in the fraction of correct answers between these two input grayscale resolutions.
Three ISO speeds for digital cameras, yielding the minimum, typical, and maximum exposures recommended for use, are defined in International Standard 12232, which is under revision. The typical and minimum acceptable exposures are based upon signal-to-noise criteria, described in ISO 12232, in which visual (perceptually relevant) noise is computed as a weighted sum of variances from a luminance (Y) and two chrominance (R-Y, B-Y) channels. The weights of the two chrominance variances, C1 (R-Y) and C2 (B-Y), are in need of reevaluation because of: (1) changes in linearization procedures being introduced in the revision of ISO 12232; (2) the limited nature of the original experiment to determine C1 and C2 and (3) suspicion that the initial C1 and C2 values were too high, overemphasizing the contributions of chrominance noise to perception. This paper describes the image simulations, psychophysical experiment, and analyses conducted to determine new values for the chrominance weights to be used in the revised standard. The values obtained, C1 = 0.279 (standard error = SE = 0.025) and C2 = 0.088 (SE = 0.017), are approximately one-half as large as those in the original version of ISO 12232. Systematic variation of the weights with the color of noise-sensitive uniform areas in the scenes is observed, but the effect is small and does not have a practical impact on the standard.
A four-alternative forced-choice experiment was conducted to investigate the relative impact of detector noise and anatomical structure on detection of subtle lung nodules. Sets of four independent backgrounds from each of three regions (heart, ribs, and lung field between the ribs) were derived from a very low-noise chest-phantom capture. Simulated nodules of varying contrast and fixed diameter (10 mm) were digitally added to the centers of selected background images. Subsequently, signal-dependent noise was introduced to simulate amorphous selenium radiographic detector performance at typical 80, 200, 400, 800, or higher speed class exposures. Series of four nodule contrasts each were empirically selected to yield comparable ranges of detectability index (d') for each background type and exposure level. Thirty-six observers with imaging expertise performed the nodule detection task, for which the signal and location were known exactly. Equally detectable nodule contrasts for each background type and exposure level were computed and their squares plotted against detector noise variance. The intercepts and slopes of the linear regressions increased in the order of lung, heart, and ribs, correlating with apparent anatomical structural complexity. The regression results imply that the effect of anatomical structure dominated that of capture device noise at clinically relevant exposures and beyond.
ISO 20462, a three-part standard entitled “Psychophysical experimental methods to estimate image quality,” is being developed by WG18 (Electronic Still Picture Imaging) of TC42 (Photography). As of late 2003, all three parts were in the Draft International Standard (DIS) ballot stage, with publication likely during 2004. This standard describes two novel perceptual methods, the triplet comparison technique and the quality ruler, that yield results calibrated in just noticeable differences (JNDs). Part 1, “Overview of psychophysical elements,” discusses specifications regarding observers, test stimuli, instructions, viewing conditions, data analysis, and reporting of results. Part 2, “Triplet comparison method,” describes a technique involving simultaneous five-point scaling of sets of three stimuli at a time, arranged so that all possible pairs of stimuli are compared exactly once. Part 3, “Quality ruler method,” describes a real-time technique optimized for obtaining assessments over a wider range of image quality. A single ruler is a series of ordered reference stimuli depicting a common scene but differing in a single perceptual attribute. Methods for generating quality ruler stimuli of known JND separation through modulation transfer function (MTF) variation are provided. Part 3 also defines a unique absolute Standard Quality Scale (SQS) of quality with one unit equal to one JND. Standard Reference Stimuli (SRS) prints calibrated against this new scale will be made available through the International Imaging Industry Association.
SC593: Characterization and Prediction of Image Quality
This course explains how to evaluate the quality of an image using numerical scales and physical standards; and how to predict the distribution of quality that would be produced by a pictorial imaging system under conditions of actual customer use. A framework is presented for conducting calibrated, extensible psychometric research so that results from different experiments can be rigorously integrated to construct predictive software using Monte Carlo simulations. Development of generalized objective metrics correlating with perceptual attributes based on psychometric data is discussed in detail and a number of examples of practical applications to product design are provided.