Automated information extraction from aerial imagery has proven to be a very difficult problem. Two decades of statistical pattern recognition research have indicated the need for more robust approaches. Image understanding, which integrates low level pattern recognition symbolizers and knowledge based artificial intelligence techniques, is the favored research approach for the future. Presently, however, the bulk of the image understanding efforts have not exploited the knowledge based aspects of information extraction from aerial imagery, but rather have been stymied in efforts devoted to pixel to symbol transformations that treat most landscape patterns as background noise. This paper outlines the physical structuring of landscapes which is believed to be a necessary ingredient for knowledge based aerial image understanding. A successful approach for manual interpretation of terrain information from aerial imagery is outlined. Two examples of computer terrain analysis that utilize terrain structural information are briefly introduced and sources of information on the physical structure of landscapes and digital terrain data are presented.