This work addresses the issue of Terrain Classification that can be applied for path planning for an Unmanned Ground Vehicle (UGV) platform. We are interested in classification of features such as rocks, bushes, trees and dirt roads. Currently, the data is acquired from a color camera mounted on the UGV as we can add range data from a second sensor in the future. The classification is accomplished by first, coarse segmenting a frame and then refining the initial segmentations through a convenient user interface. After the first frame, temporal information is exploited to improve the quality of the image segmentation and help classification adapt to changes due to ambient lighting, shadows, and scene changes as the platform moves. The Mean Shift Classifier algorithm provides segmentation of the current frame data. We have tested the above algorithms with four sequence of frames acquired in an environment with terrain representative of the type we expect to see in the field. A comparison of the results from this algorithm was done with accurate manually-segmented (ground-truth) data, for each frame in the sequence.