We describe simultaneous determination of surface height profile and material distribution in the surface by imaging polarization detection. Starting from the single-beam case, the measurement principle based on the Stokes-Mueller formalism is deduced. This algorithm implies waviness detection of smooth as well as of rough surfaces. The basic concept of an imaging ellipsometer and its system configuration are discussed in the Stokes-Mueller formalism. Measuring the local slopes of the surface applying Fresnel's equations, the height image is deduced. Furthermore the imaging ellipsometer delivers the local complex refractive index from which the image of material distribution can be derived by a look-up table.