The application of knowledge-based processing in image understanding systems has always occurred at a high level. These approaches have generated great claims, but limited results as they are basically simple forward chaining rule systems (e.g., if object on a road, then object is a vehicle). The application of heuristic reasoning to the low-level processing (e.g., image enhancement, segmentation, feature extraction, and classification) is a requirement to provide more accurate object and region information for high-level analysis, and truly integrate artificial intelligence throughout the entire system rather than as a post processing afterthought. This paper addresses the design and development of an integrated knowledge-based vision system in three phases. First, the application of knowledge base system techniques to image understanding is analyzed in light of deficiencies and limitations. This analysis is then exploited to produce a synergistically integrated system design. Second, the application of heuristics to low-level processing is discussed with specific application in the areas of image enhancement and segmentation. Finally, conclusions drawn to date on system performance will be presented in association with a mapping of how they are directing future work in this area.