In this paper we will present an overview of our research into perception and biologically inspired modeling of illumination (flow) from 3D textures and the influence of roughness and illumination on material perception. Here 3D texture is defined as an image of an illuminated rough surface. In a series of theoretical and empirical papers we studied how we can estimate the illumination orientation (in the image plane) from 3D textures of globally flat samples. We found that the orientation can be estimated well by humans and computers using an approach based on second order statistics. This approach makes use of the dipole-like structures in 3D textures that are the results of illumination of bumps / throughs. For 3D objects, the local illumination direction varies over the object, resulting in surface illuminance flow. This again results in image illuminance flow in the image of a rough 3D object: the observable projection in the image of the field of local illumination orientations. Here we present results on image illuminance flow analysis for images from the Utrecht Oranges database, the Curet database and two vases. These results show that the image illuminance flow can be estimated robustly for various rough materials. In earlier studies we have shown that the image illuminance flow can be used to do shape and illumination inferences. Recently, in psychophysical experiments we found that adding 3D texture to a matte spherical object improves judgments of the direction and diffuseness of its illumination by human observers. This shows that human observers indeed use the illuminance flow as a cue for the illumination.
Whereas the 17th century painter Canaletto was a master in linear perspective of the architectural elements, he seems to have had considerable difficulty with linear perspective of shadows. A common trick to avoid shadow perspective problems is to set the (solar) illumination direction parallel to the projection screen. We investigated in one painting where Canaletto clearly used this trick, whether he followed this light direction choice consistently through in how he shades the persons. We approached this question with a perceptual experiment where we measured perceived light directions in isolated details of the paintings. Specifically, we controlled whether observers could only see the (cast) shadow, only shading or both. We found different trends in all three conditions. The results indicate that Canaletto probably used different shading than the parallel light direction would predict. We interpret the results as a form or artistic freedom that Canaletto used to shade the persons individually.
Whereas pictorial space plays an important role in art historic discussions, there is little research on the quantitative structure of pictorial spaces. Recently, a number of methods have been developed, one of which relies on size constancy: two spheres are rendered in the image while the observers adjusts the relative sizes such that they appear to have similar sizes in pictorial space. This method is based on pair-wise comparisons, resulting in n(n-1)/2 trials for n samples. Furthermore, it renders a probe in the image that does not conform to the style of the painting: it mixes computer graphics with a painting. The method proposed here uses probes that are already in the scene, not violating the paintings' style. An object is copied from the original painting and shown in a different location. The observer can adjust the scaling such that the two objects (one originally in the painting, and the other copy-pasted) appear to have equal sizes in pictorial space. Since the original object serves as a reference, the number of trials increases with n instead of <i>n</i><sup>2</sup> which is the case of the original method. We measured the pictorial spaces of two paintings using our method, one Canaletto and one Breughel. We found that observers typically agreed well with respect to each other, coefficients of determination as high as 0.9 were found when the probe was a human, while other probes scored somewhat (but significantly) lower. These initial findings appear very promising for the study of pictorial space.
We present a novel setup in which real objects made of different materials can be mixed optically. For the materials we
chose mutually very different materials, which we assume to represent canonical modes. The appearance of 3D objects
consisting of any material can be described as linear superposition of 3D objects of different canonical materials, as in
"painterly mixes". In this paper we studied mixtures of matte, glossy and velvety objects, representing diffuse, forward
and asperity scattering modes.
Observers rated optical mixtures on four scales: matte-glossy, hard-soft, cold-warm, light-heavy. The ratings were done
for the three combinations of glossy, matte, and velvety green birds. For each combination we tested 7 weightings.
Matte-glossy ratings varied most over the stimuli and showed highest (most glossy) scores for the rather glossy bird and
lowest (most matte) for the rather velvety bird. Hard-soft and cold-warm were rated highest (most soft and warm) for
rather velvety and lowest (most hard and cold) for rather glossy birds. Light-heavy was rated only somewhat higher
(heavier) for rather glossy birds. The ratings varied systematically with the weights of the contributions, corresponding to
gradually changing mixtures of material modes. We discuss a range of possibilities for our novel setup.
In this study we demonstrate that touch decreases the ambiguity in a visual image. It has been previously
found that visual perception of three-dimensional shape is subject to certain variations. These variations can
be described by the affine transformation. While the visual system thus seems unable to capture the Euclidean
structure of a shape, touch could potentially be a useful source to disambiguate the image. Participants performed
a so-called 'attitude task' from which the structure of the perceived three-dimensional shape was calculated. One
group performed the task with only vision and a second group could touch the stimulus while viewing it. We found
that the consistency within the haptics+vision group was higher than in the vision-only group. Thus, haptics
decreases the visual ambiguity. Furthermore, we found that the touched shape was consistently perceived as
having more relief than the untouched the shape. It was also found that the direction of affine shear differences
within the two groups was more consistent when touch was used. We thus show that haptics has a significant
influence on the perception of pictorial relief.