General stereo image matching provides an adequate but hard problem with sufficient complexity, with which the potential of wavelets may be exploited to a full extend. An ideal stereo image matching algorithm is supposed to be invariant to the scale, translation, rotation, and partial correspondence between two given stereo images. While the multi-resolution of wavelets is good at scale adaptivity, we also require the wavelet transform and pyramids to be translation- and rotation-invariant. This paper is intended to serve for three purposes: (1) To present the general problem of stereo image matching in a sufficient depth and extent, so that pure wavelet mathematicians could think on adequate and efficient solutions, (2) To present a complete algorithm for top-down image matching including surface reconstruction by using wavelet pyramids, (3) To search for a wavelet family optimal for image matching. It is expected that a family of adequately designed wavelets could provide a generic and robust solution to the stereo image matching problem, which could be an important breakthrough in computer vision, photogrammetry, and pattern recognition.