Text detection in natural scenes holds great importance in the field of research and still remains a challenge and an important task because of size, various fonts, line orientation, different illumination conditions, weak characters and complex backgrounds in image. The contribution of our proposed method is to filtering out complex backgrounds by combining three strategies. These are enhancing the edge candidate detection in HSV space color, then using MSER candidate detection to get different masks applied in HSV space color as well as gray color. After that, we opt for the Stroke Width Transform (SWT) and heuristic filtering. Such strategies are followed so as to maximize the capacity of zones text pixels candidates and distinguish between text boxes and the rest of the image. The non-text components are filtered by classifying the characters candidates based on Support Vector Machines (SVM) using Histogram of Oriented Gradients (HOG) features. Finally we apply boundary box localization after a stage of word grouping where false positives are eliminated by geometrical properties of text blocks. The proposed method has been evaluated on ICDAR 2013 scene text detection competition dataset and the encouraging experiments results demonstrate the robustness of our method.