In the modern sawmill industry automatic grading of the products is one of the key issues in increasing the production quality. The surface defects that determine the grading are identified according to the physiological origin of the defect, such as dry, encased or decayed knot. Variations within the classes are large since the knots can have different shapes, sizes and color, and each class has different discriminating features. Classification of the defects using pattern recognition techniques has turned out to be rather difficult, since it is difficult to determine the suitable features that would correlate with the physiological defect types. In this paper we describe a wood defect classification system that is based on self-organizing feature construction and neural network classification. Due to the automatic, unsupervised learning of the features, the system is easily adaptable to different tasks, such as inspection of lumber or veneer, with different tree species and different cutting processes. Performance of the classification system was evaluated with a set of over 400 samples from spruce boards. The knot recognition rate was about 85% with only gray level images, giving about 90% accuracy for the final board grading. Compared to 75 - 80% accuracy that can be maintained by a human inspector, the result can be considered good.