The output of most medical imaging systems is a display for interpretation by human observers. This paper provides a general summary of recent work on shape recognition by humans. Two broad modes of visual image processing executed by different cortical loci can be distinguished: a) a mode for motor interaction which is sensitive to quantitative variation in image parameters and b) a mode for basic-level object recognition which is based on a small set of qualitative contrasts in viewpoint invariant properties of images edges. Many medical image classifications pose inherently difficult problems for the recognition system in that they are based on quantitative and surface patch variations--rather than qualitative--variations. But when recognition can be achieved quickly and accurately it is possible that a small viewpoint invariant contrast has been discovered and is being exploited by the interpreter.