Hollow village is a special phenomenon in the process of urbanization in China, which causes the waste of land resources. Therefore, it's imminent to carry out the hollow village recognition and renovation. However, there are few researches on the remote sensing identification of hollow village. In this context, in order to recognize the abandoned homesteads by remote sensing technique, the experiment was carried out as follows. Firstly, Gram-Schmidt transform method was utilized to complete the image fusion between multi-spectral images and panchromatic image of WorldView-2. Then the fusion images were made edge enhanced by high pass filtering. The multi-resolution segmentation and spectral difference segmentation were carried out to obtain the image objects. Secondly, spectral characteristic parameters were calculated, such as the normalized difference vegetation index (NDVI), the normalized difference water index (NDWI), the normalized difference Soil index (NDSI) etc. The shape feature parameters were extracted, such as Area, Length/Width Ratio and Rectangular Fit etc.. Thirdly, the SEaTH algorithm was used to determine the thresholds and optimize the feature space. Furthermore, the threshold classification method and the random forest classifier were combined, and the appropriate amount of samples were selected to train the classifier in order to determine the important feature parameters and the best classifier parameters involved in classification. Finally, the classification results was verified by computing the confusion matrix. The classification results were continuous and the phenomenon of salt and pepper using pixel classification was avoided effectively. In addition, the results showed that the extracted Abandoned Homesteads were in complete shapes, which could be distinguished from those confusing classes such as Homestead in Use and Roads.