Reading text or searching for key words within a historical document is a very challenging task. one of the ﬁrst steps of the complete task is binarization, where we separate foreground such as text, ﬁgures and drawings from the background. Successful results of this important step in many cases can determine next steps to success or failure, therefore it is very vital to the success of the complete task of reading and analyzing the content of a document image. Generally, historical documents images are of poor quality due to their storage condition and degradation over time, which mostly cause to varying contrasts, stains, dirt and seeping ink from reverse side. In this paper, we use banks of anisotropic predeﬁned ﬁlters in diﬀerent scales and orientations to develop a binarization method for degraded documents and manuscripts. Using the fact, that handwritten strokes may follow diﬀerent scales and orientations, we use predeﬁned sets of ﬁlter banks having various scales, weights, and orientations to seek a compact set of ﬁlters and weights in order to generate diﬀerent layers of foregrounds and background. Results of convolving these ﬁlters on the gray level image locally, weighted and accumulated to enhance the original image. Based on the diﬀerent layers, seeds of components in the gray level image and a learning process, we present an improved binarization algorithm to separate the background from layers of foreground. Diﬀerent layers of foreground which may be caused by seeping ink, degradation or other factors are also separated from the real foreground in a second phase. Promising experimental results were obtained on the DIBCO2011 , DIBCO2013 and H-DIBCO2016 data sets and a collection of images taken from real historical documents.