Utilizing linear feature is now widely used in building detection. These linear feature-based methods are simple but low accuracy and time-consuming. This paper proposes a novel and efficient method of automatically detecting buildings based on multi-characteristic fusion from remote sensing images. The method firstly adopts Canny algorithm to detect edges lines from images. Then utilizing the feature of building distribution and the Hough transform, it employs ISODATA clustering algorithms to detect the main orientations of buildings. This clustering analysis method could filter edge lines and help to get latent edges of building objects. After that, the edges were linked to get the buildings' shape according to some linking rules. However there exit large amounts of false detection objects. In order to reduce them, a series of geometrical characteristics (such as the corner characteristic, the shadow characteristic, etc) and gray characteristic of buildings as criteria were brought up as the building judgments to eliminate them. We put forward the corresponding algorithm to extract each characteristic, later the fusion method based on the maximum membership principle in fuzzy pattern recognition was introduced to combine all these algorithm results together, and at last successfully detect buildings. The large number of experiment results show that this new method in this paper, compared with common linear feature-based building detection methods, is of high speed, more accurate and has good robustness. This new method is especially fit for practical applications in relatively complicated environments.