A general class of distortion invariant quadratic filter is developed for optical pattern recognition. It produces an equal response to a set of training images while maximizing the normalized response. Different forms of the filter are obtained by constraining the filter to operate on different linear subspaces of the input signal. These subspaces can be used to represent constraints due to system components or they can represent feature spaces which model the internal structure of certain types of signals. The proposed quadratic filter can be mapped onto a variety of different optical processor architecwres providing a general framework for developing optical processors for pattern classification.