This paper presents secondary Standard Quality Scale (SQS2) rankings in overall quality JNDs for a subjective analysis of the 3 axes of noise, amplitude, spectral content, and noise type, based on the ISO 20462 softcopy ruler protocol. For the initial pilot study, a Python noise simulation model was created to generate the matrix of noise masks for the softcopy ruler base images with different levels of noise, different low pass filter noise bandwidths and different band pass filter center frequencies, and 3 different types of noise: luma only, chroma only, and luma and chroma combined. Based on the lessons learned, the full subjective experiment, involving 27 observers from Google, NVIDIA and STMicroelectronics was modified to incorporate a wider set of base image scenes, and the removal of band pass filtered noise masks to ease observer fatigue. Good correlation was observed with the Aptina subjective noise study. The absence of tone mapping in the noise simulation model visibly reduced the contrast at high levels of noise, due to the clipping of the high levels of noise near black and white. Under the 34-inch viewing distance, no significant difference was found between the luma only noise masks and the combined luma and chroma noise masks. This was not the intuitive expectation. Two of the base images with large uniform areas, ‘restaurant’ and ‘no parking’, were found to be consistently more sensitive to noise than the texture rich scenes. Two key conclusions are (1) there are fundamentally different sensitivities to noise on a flat patch versus noise in real images and (2) magnification of an image accentuates visual noise in a way that is non-representative of typical noise reduction algorithms generating the same output frequency. Analysis of our experimental noise masks applied to a synthetic Macbeth ColorChecker Chart confirmed the color-dependent nature of the visibility of luma and chroma noise.
In this paper we address the problem of Image Quality Assessment of no reference metrics,
focusing on JPEG
corrupted images. In general no reference metrics are not able to measure with the same
performance the distortions within their possible range and with respect to different image
contents. The crosstalk between content and distortion signals influences the human perception.
We here propose two strategies to improve the correlation between subjective and objective
quality data. The first strategy is based on grouping the images according to their spatial
complexity. The second one is based on a frequency analysis. Both the strategies are tested on
two databases available in the literature. The results show an improvement in the correlations
no reference metrics and psycho-visual data, evaluated in terms of the Pearson Correlation
The I3A Camera Phone Image Quality (CPIQ) initiative aims to provide a consumer-oriented
overall image quality metric for mobile phone cameras. In order to achieve this
goal, a set of subjectively correlated image quality metrics has been developed. This paper
describes the development of a specific group within this set of metrics, the spatial metrics.
Contained in this group are the edge acutance, visual noise and texture acutance metrics.
A common feature is that they are all dependent on the spatial content of the specific
scene being analyzed. Therefore, the measurement results of the metrics are weighted by
a contrast sensitivity function (CSF) and, thus, the conditions under which a particular
image is viewed must be specified. This leads to the establishment of a common framework
consisting of three components shared by all spatial metrics. First, the RGB image is transformed
to a color opponent space, separating the luminance channel from two chrominance
channels. Second, associated with this color space are three contrast sensitivity functions
for each individual opponent channel. Finally, the specific viewing conditions, comprising
both digital displays as well as printouts, are supported through two distinct MTFs.
Human observers are able to make fine discriminations of surface gloss. What cues are they using to perform this task? In
previous studies, we identified two reflection-related cues-the contrast of the reflected image (c, contrast gloss) and the sharpness of
reflected image (d, distinctness-of-image gloss)--but these were for objects rendered in standard dynamic range (SDR) images with
compressed highlights. In ongoing work, we are studying the effects of image dynamic range on perceived gloss, comparing high
dynamic range (HDR) images with accurate reflections and SDR images with compressed reflections. In this paper, we first present
the basic findings of this gloss discrimination study then present an analysis of eye movement recordings that show where observers
were looking during the gloss discrimination task. The results indicate that: 1) image dynamic range has significant influence on
perceived gloss, with surfaces presented in HDR images being seen as glossier and more discriminable than their SDR counterparts;
2) observers look at both light source highlights and environmental interreflections when judging gloss; and 3) both of these results
are modulated by surface geometry and scene illumination.
Imaging systems in camera phones have image quality limitations attributed to optics, size, and cost constraints. These
limitations generally result in unwanted system noise. In order to minimize the image quality degradation, nonlinear
noise cleaning algorithms are often applied to the images. However, as the strength of the noise cleaning increases, this
often leads to texture degradation. The Camera Phone Image Quality (CPIQ) initiative of the International Imaging
Industry Association (I3A) has been developing metrics to quantify texture appearance in camera phone images. Initial
research established high correlation levels between the metrics and psychophysical data from sets of images that had
noise cleaning filtering applied to simulate capture in actual camera phone systems. This paper describes the subsequent
work to develop a texture-based softcopy attribute ruler in order to assess the texture appearance of eight camera phone
units from four different manufacturers and to assess the efficacy of the texture metrics. Multiple companies
participating in the initiative have been using the softcopy ruler approach in order to pool observers and increase
statistical significance. Results and conclusions based on three captured scenes and two texture metrics will be
Because of the emergence of e-commerce and developments in print engines designed for economical output of very short runs, there are increased business opportunities and consumer options for print-on-demand books and photobooks. The current state of these printing modes allows for direct uploading of book files via the web, printing on nonoffset printers, and distributing by standard parcel or mail delivery services. The goal of this research is to assess the image quality of print-on-demand books and photobooks produced by various Web-based vendors and to identify correlations between psychophysical results and objective metrics. Six vendors were identified for one-off (single-copy) print-on-demand books, and seven vendors were identified for photobooks. Participants rank ordered overall quality of a subset of individual pages from each book, where the pages included text, photographs, or a combination of the two. Observers also reported overall quality ratings and price estimates for the bound books. Objective metrics of color gamut, color accuracy, accuracy of International Color Consortium profile usage, eye-weighted root mean square L*, and cascaded modulation transfer acutance were obtained and compared to the observer responses. We introduce some new methods for normalizing data as well as for strengthening the statistical significance of the results. Our approach includes the use of latent mixed-effect models. We found statistically significant correlation with overall image quality and some of the spatial metrics, but correlations between psychophysical results and other objective metrics were weak or nonexistent. Strong correlation was found between psychophysical results of overall quality assessment and estimated price associated with quality. The photobook set of vendors reached higher image-quality ratings than the set of print-on-demand vendors. However, the photobook set had higher image-quality variability.
Texture appearance is an important component of photographic image quality as well as object recognition. Noise
cleaning algorithms are used to decrease sensor noise of digital images, but can hinder texture elements in the process.
The Camera Phone Image Quality (CPIQ) initiative of the International Imaging Industry Association (I3A) is
developing metrics to quantify texture appearance. Objective and subjective experimental results of the texture metric
development are presented in this paper. Eight levels of noise cleaning were applied to ten photographic scenes that
included texture elements such as faces, landscapes, architecture, and foliage. Four companies (Aptina Imaging, LLC,
Hewlett-Packard, Eastman Kodak Company, and Vista Point Technologies) have performed psychophysical evaluations
of overall image quality using one of two methods of evaluation. Both methods presented paired comparisons of images
on thin film transistor liquid crystal displays (TFT-LCD), but the display pixel pitch and viewing distance differed. CPIQ
has also been developing objective texture metrics and targets that were used to analyze the same eight levels of noise
cleaning. The correlation of the subjective and objective test results indicates that texture perception can be modeled
with an objective metric. The two methods of psychophysical evaluation exhibited high correlation despite the
differences in methodology.
A softcopy quality ruler method was implemented for the International Imaging Industry Association (I3A) Camera
Phone Image Quality (CPIQ) Initiative. This work extends ISO 20462 Part 3 by virtue of creating reference digital
images of known subjective image quality, complimenting the hardcopy Standard Reference Stimuli (SRS). The
softcopy ruler method was developed using images from a Canon EOS 1Ds Mark II D-SLR digital still camera (DSC)
and a Kodak P880 point-and-shoot DSC. Images were viewed on an Apple 30in Cinema Display at a viewing distance of
34 inches. Ruler images were made for 16 scenes. Thirty ruler images were generated for each scene, representing ISO
20462 Standard Quality Scale (SQS) values of approximately 2 to 31 at an increment of one just noticeable difference
(JND) by adjusting the system modulation transfer function (MTF). A Matlab GUI was developed to display the ruler
and test images side-by-side with a user-adjustable ruler level controlled by a slider. A validation study was performed at
Kodak, Vista Point Technology, and Aptina Imaging in which all three companies set up a similar viewing lab to run the
softcopy ruler method. The results show that the three sets of data are in reasonable agreement with each other, with the
differences within the range expected from observer variability. Compared to previous implementations of the quality
ruler, the slider-based user interface allows approximately 2x faster assessments with 21.6% better precision.
SC1049: Benchmarking Image Quality of Still and Video Imaging Systems
Because image quality is multi-faceted, generating a concise and relevant evaluative summary of photographic systems can be challenging. Indeed, benchmarking the image quality of still and video imaging systems requires that the assessor understands not only the capture device itself, but also the imaging applications for the system.
This course explains how objective metrics and subjective methodologies are used to benchmark image quality of photographic still image and video capture devices. The course will go through key image quality attributes and the flaws that degrade those attributes, including causes and consequences of the flaws on perceived quality. Content will describe various subjective evaluation methodologies as well as objective measurement methodologies relying on existing standards from ISO, IEEE/CPIQ, ITU and beyond. Because imaging systems are intended for visual purposes, emphasis will be on the value of using objective metrics which are perceptually correlated and generating benchmark data from the combination of objective and subjective metrics.
The course "SC1157 Camera Characterization and Camera Models," describing camera models and objective measurements, complements the treatment of perceptual models and subjective measurements provided here.
Image Quality depends not only on the camera components, but also on lighting, photographer skills, picture content, viewing conditions and to some extent on the viewer. While measuring or predicting a camera's image quality as perceived by users can be an overwhelming task, many camera attributes can be accurately characterized with objective measurement methodologies. This course provides an insight on camera models, examining the mathematical models of the three main components of a camera (optics, sensor and ISP) and their interactions as a system (camera) or subsystem (camera at the raw level).
The course describes methodologies to characterize the camera as a system or subsystem (modeled from the individual component mathematical models), including lab equipment, lighting systems, measurements devices, charts, protocols and software algorithms. Attributes to be discussed include exposure, color response, sharpness, shading, chromatic aberrations, noise, dynamic range, exposure time, rolling shutter, focusing system, and image stabilization. The course will also address aspects that specifically affect video capture, such as video stabilization, video codec, and temporal noise.
The course "SC1049 Benchmarking Image Quality of Still and Video Imaging Systems," describing perceptual models and subjective measurements, complements the treatment of camera models and objective measurements provided here.