Because of the variety and complexity of defects in the fabric texture image, fabric defect detection is a challenging issue in the fields of machine vision. In this paper, a novel fabric defect detection method is proposed based on wavelet transform and background estimating. Firstly, the feature map of the fabric image is generated according to wavelet transform. Secondly, the multi-backgrounds are estimated by averaging the divided blocks of the feature map, and the saliency maps are generated by comparing the map blocks with the estimating backgrounds. Thirdly, an integrated saliency map is generated by a fusing method. Finally, the contrast between foreground and background is enhanced by estimating the probability density function of the saliency map, and the threshold segmentation algorithm is adopted to locate the defect area. Experiment results show that the proposed algorithm is superior to the state of the art detection methods.