Human photointerpreters use expert knowledge and contextual information to help them analyze a scene. We have experimented with the Lockheed Expert System (LES) to see if contextual information can be useful in interpreting aerial photographs. First, the gray-scale image is segmented into uniform or slowly varying intensity regions or contiguous textured regions using an edge-based segmentation technique. Next, the system computes a set of attributes for each region. Some of these attributes are based on local properties of that region only (e.g., area, average intensity, texture strength, etc.); others are based on contextual or global information (e.g., adjacent regions and nearby regions). Finally, LES is given the task of classifying all the regions using the attribute values. It utilizes multiple goals and multiple rule sets to determine the best classification; regions that do not satisfy any of the rules are left unclassified. The authors obtained the rules by an introspection technique after studying many aerial photographs. Unlike programs that use only statistics in the region under consideration, LES can use contextual information such as the fact that cars are likely to be adjacent to roads, which significantly improves its performance on regions that are difficult to classify.