This paper proposes a novel method for extracting facades structure from real-world pictures by using local geometric moment. Compared with existing methods, the proposed method has advantages of easy-to-implement, low computational cost, and robustness to noises, such as uneven illumination, shadow, and shade from other objects. Besides, our method is faster and has a lower space complexity, making it feasible for mobile devices and the situation where real-time data processing is required. Specifically, a facades structure modal is first proposed to support the use of our special noise reduction method, which is based on a self-adapt local threshold with Gaussian weighted average for image binarization processing and the feature of the facades structure. Next, we divide the picture of the building into many individual areas, each of which represents a door or a window in the picture. Subsequently we calculate the geometric moment and centroid for each individual area, for identifying those collinear ones based on the feature vectors, each of which is thereafter replaced with a line. Finally, we comprehensively analyze all the geometric moment and centroid to find out the facades structure of the building. We compare our result with other methods and especially report the result from the pictures taken in bad environmental conditions. Our system is designed for two application, i.e, the reconstruction of facades based on higher resolution ground-based on imagery, and the positional system based on recognize the urban building.