14 March 2013 Can relative skill be determined from a photographic portfolio?
Author Affiliations +
In this study, our primary aim is to determine empirically the role that skill plays in determining image aesthetics, and whether it can be deciphered from the ratings given by a diverse group of judges. To this end, we have collected and analyzed data from a large number of subjects (total 168) on a set of 221 of images taken by 33 photographers having different photographic skill and experience. We also experimented with the rating scales used by previous studies in this domain by introducing a binary rating system for collecting judges’ opinions. The study also demonstrates the use of Amazon Mechanical Turk as a crowd-sourcing platform in collecting scientific data and evaluating the skill of the judges participating in the experiment. We use a variety of performance and correlation metrics to evaluate the consistency of ratings across different rating scales and compare our findings. A novel feature of our study is an attempt to define a threshold based on the consistency of ratings when judges rate duplicate images. Our conclusion deviates from earlier findings and our own expectations, with ratings not being able to determine skill levels of photographers to a statistically significant level.
© (2013) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Abhishek Agrawal, Abhishek Agrawal, Vittal Premachandran, Vittal Premachandran, Rajesh Somavarapu, Rajesh Somavarapu, Ramakrishna Kakarala, Ramakrishna Kakarala, "Can relative skill be determined from a photographic portfolio?", Proc. SPIE 8651, Human Vision and Electronic Imaging XVIII, 86510X (14 March 2013); doi: 10.1117/12.2008065; https://doi.org/10.1117/12.2008065

Back to Top