A bottleneck in building a knowledge base signal interpretation system is combining the appropriate problem-solving knowledge from the expert with physical observations the signal carry. The information and knowledge processing techniques available in the fields of pattern recognition, signal processing and heuristics can be used to automatically measure the parameters from the physical observations. In this paper, such methods are used to develop an intelligent signal interpretation system. This combined approach will not only automate and accelerate the knowledge acquisition and organization process, but will also formalize and structure the decision making process. The system is designed to interpret and classify signals emitted from a material source. The system consists of four basic components, namely; Fact Gathering, Knowledge Base, Knowledge Formalization, and Inference Engine. The fact gathering subsystem, 1) collects the transduced signals from materials and extracts a large feature set from them, and 2) collects the a priori real-world knowledge and the expert knowledge about the source material and testing conditions. The facts, a priori real-world knowledge, and the pattern measurements (features) are organized into a knowledge base. The next subsystem formalizes the knowledge into a tree structure using a class-association concept. The last subsystem is the Inference Engine which primarily classifies the signals using composite knowledge incorporated in the system. This paper presents the design of the proposed system and shows successful identification of unknown signals from several material defect sources.