This paper proposes a new tree-like fuzzy binary support vector machines multi-class classifier (FBSVM) for the optical character recognition task. We construct this tree-like classifier by fusing of fuzzy clustering technique and support vector machine (SVM). In k-class task, the new classifier contains k-1 SVM sub-classifiers, but the "one-against-one" method which is usually used contains k(k-1)/2 sub-classifiers. This method also overcomes the drawback such as unclassifiable region that the "one-against-one" method has, and has a good classification performance. Furthermore, it needs less memory. By applying the new classifier to the real mail zipcode digits recognition task, the experimental results indicate that the FBSVM has a better recognition performance.