A composite surface model is a data structure in which a stream of surface information from different sources and different viewing positions is integrated. A composite surface model may be used for learning about objects, planning actions and monitoring the execution of actions. This paper describes a representation for a composite surface model based on small planar patches. The primitive symbol of this representation, the generalized surface patch, embodies mutual constraints between uncertainties in position, spatial extent and orientation. Techniques for integrating surface information from different sources, and for dynamically updating a composite surface model are described.