Mathematical morphology with gray scale structuring elements has attracted much attention, since combinations of the operations in this class can realize almost all noise-removing filters. However, the optimization method for the combination is still uncertain. In this paper, an optimization method for a mathematical morphological filter with gray scale structuring elements is proposed. This method is based on the concept of a neural network with morphological operations and on learning using simulated annealing. The method is also applied to gray scale bipolar morphological filters for image differentiation.