Steps are taken toward the automatic, intensity-based recognition of human faces by constructing a vision system to automatically detect frontally-viewed human eyes in real data. The eye is modeled using a deformable template that specifies a parameterized geometry and an intensity model. The fit of the template is measured by a cost-functional employing robust estimators, i.e., (alpha) -trimmed means and variances, to overcome highlights, shadows, nonrigid boundaries, noise, and other such difficulties. Recognition proceeds in three stages. First, candidate eyes are located by matching a simplified eye model against the responses of a robust, general purpose detector of intensity valleys and peaks. Second, the best fit of each candidate eye is found by minimizing the energy of a cost functional. Third, each candidate is accepted or rejected based on the amount of variance in the image data it explains.