Balanced and representative test images are needed to study perceived visual quality in various application domains.
This study investigates naturalness and interestingness as image quality attributes in the context of test images. Taking a
top-down approach we aim to find the dimensions which constitute naturalness and interestingness in test images and the
relationship between these high-level quality attributes. We compare existing collections of test images (e.g. Sony sRGB
images, ISO 12640 images, Kodak images, Nokia images and test images developed within our group) in an experiment
combining quality sorting and structured interviews. Based on the data gathered we analyze the viewer-supplied criteria
for naturalness and interestingness across image types, quality levels and judges. This study advances our understanding
of subjective image quality criteria and enables the validation of current test images, furthering their development.
The goal of the study was to develop a method for quality computation of digitally printed images. We wanted to use
only the attributes which have a meaning for subjective visual quality experience of digitally printed images. Based on
the subjective data and our assessments the attributes for quality calculation were sharpness, graininess and color
contrast. The proposed graininess metric divides the fine detail image into blocks and used the low energy blocks for
graininess calculation. The proposed color contrast metric computes the contrast of dominant colors using the coarse
scale image. The proposed sharpness metric divides the coarse scale image into blocks and uses the high energy blocks
for sharpness calculation. The reduced reference features of sharpness and graininess metrics are the numbers of high or
low energy blocks. The reduced reference features of the color contrast metric are the directions of the dominant colors
in reference image. The overall image quality was calculated by combining the values. The performance of proposed
application specific image quality metric was high compared to the state of the art reduced reference applicationindependent
image quality metric. Linear correlation coefficients between subjective and predicted MOS were 0.88 for
electrophotography and 0.98 for ink-jet printed samples, for a sample set of 21 prints for electrophotography and for inkjet,
subjectively evaluated by 28 observers.
The aim of the study was to develop a test image for print quality evaluation to improve the current state of the art in
testing the quality of digital printing. The image presented by the authors in EI09 portrayed a breakfast scene, the content
of which could roughly be divided in four object categories: a woman, a table with objects, a landscape picture and a
gray wall. The image was considered to have four main areas of improvement: the busyness of the image, the control of
the color world, the salience of the object categories, and the naturalness of the event and the setting. To improve the first
image, another test image was developed. Whereas several aspects were improved, the shortcomings of the new image
found by visual testing and self-report were in the same four areas. To combine the insights of the two test images and to
avoid their pitfalls, a third image was developed. The goodness of the three test images was measured in subjective tests.
The third test image was found to address efficiently three of the four improvement areas, only the salience of the objects
left a bit to be desired.
A test image for color still image processes was developed. The image is based on general requirements on the content
and specific requirements arising from the quality attributes of interest. The quality attributes addressed in the study
include sharpness, noise, contrast, colorfulness and gloss. These were chosen based on visual relevance in studies of the
influence of paper in digital printing. Further requirements such as arising from the use cases of the image are discussed
based on eye tracking data and self-report of the usefulness of different objects for quality evaluation. From the
standpoint of being sufficiently sensitive to quality variations of the imaging systems to be measured the reference test
image needs to represent quality maxima in terms of the relevant quality parameters. As for different viewing times, no
object should be exceedingly salient. The paper presents the procedure of developing the test image and discusses its
merits and shortcomings from the standpoint of future development.
Subjective quality rating does not reflect the properties of the image directly, but it is the outcome of a quality decision
making process, which includes quantification of subjective quality experience. Such a rich subjective content is often
ignored. We conducted two experiments (with 28 and 20 observers), in order to study the effect of paper grade on image
quality experience of the ink-jet prints. Image quality experience was studied using a grouping task and a quality rating
task. Both tasks included an interview, but in the latter task we examined the relations of different subjective attributes in
this experience. We found out that the observers use an attribute hierarchy, where the high-level attributes are more
experiential, general and abstract, while low-level attributes are more detailed and concrete. This may reflect the
hierarchy of the human visual system. We also noticed that while the observers show variable subjective criteria for IQ,
the reliability of average subjective estimates is high: when two different observer groups estimated the same images in
the two experiments, correlations between the mean ratings were between .986 and .994, depending on the image
Due to the rise in performance of digital printing, image-based applications are gaining popularity. This creates needs for
specifying the quality potential of printers and materials in more detail than before. Both production and end-use
standpoints are relevant. This paper gives an overview of an
on-going study which has the goal of determining a
framework model for the visual quality potential of paper in color image printing. The approach is top-down and it is
founded on the concept of a layered network model. The model and its subjective, objective and instrumental
measurement layers are discussed. Some preliminary findings are presented. These are based on data from samples
obtained by printing natural image contents and simple test fields on a wide range of paper grades by ink-jet in a color
managed process. Color profiles were paper specific. Visual mean opinion score data by human observers could be
accounted for by two or three dimensions. In the first place these are related to brightness and color brightness. Image
content has a marked effect on the dimensions. This underlines the challenges in designing the test images.
The psychological complexity of multivariate image quality evaluation makes it difficult to develop general image quality metrics. Quality evaluation includes several mental processes and ignoring these processes and the use of a few test images can lead to biased results. By using a qualitative/quantitative (Interpretation Based Quality, IBQ) methodology, we examined the process of pair-wise comparison in a setting, where the quality of the images printed by laser printer on different paper grades was evaluated. Test image consisted of a picture of a table covered with several objects. Three other images were also used, photographs of a woman, cityscape and countryside. In addition to the pair-wise comparisons, observers (N=10) were interviewed about the subjective quality attributes they used in making their quality decisions. An examination of the individual pair-wise comparisons revealed serious inconsistencies in observers' evaluations on the test image content, but not on other contexts. The qualitative analysis showed that this inconsistency was due to the observers' focus of attention. The lack of easily recognizable context in the test image may have contributed to this inconsistency. To obtain reliable knowledge of the effect of image context or attention on subjective image quality, a qualitative methodology is needed.