We address the problem of automatically learning object models for recognition and pose estimation. In contrast to the traditional approach, we formulate the recognition problem as one of matching visual appearance rather than shape. The appearance of an object in a two- dimensional image depends on its shape, reflectance properties, pose in the scene, and the illumination conditions. While shape and reflectance are intrinsic properties of an object and are constant, pose and illumination vary from scene to scene. We present a new compact representation of object appearance that is parametrized by pose and illumination. For each object of interest, a large set of images is obtained by automatically varying pose and illumination. This large image set is compressed to obtain a low-dimensional subspace, called the eigenspace, in which the object is represented as a hypersurface. Given an unknown input image, the recognition system projects the image onto the eigenspace. The object is recognized based on the hypersurface it lies on. The exact position of the projection on the hypersurface determines the object's pose in the image. We have conducted experiments using several objects with complex appearance characteristics. We conclude with a discussion on various issues related to the learning and recognition techniques proposed in the paper.