The proposed approach addresses the problem of recognition of touching characters by forming a closed loop system between segmentation and isolated character classification for mutually beneficial feedback. Multiple hypotheses are generated and ranked on the basis of various constraints such as classifier confidence, geometric or structural requirements and contextual information at every intermediate step. The method uses a variable width window sliding throughout the word and results in a tree structure with intermediate nodes representing validated characters. Each partial path is assigned a confidence value on the basis of segmentation confidence, recognition confidence and first order transition probability, if applicable. Final candidates for the field truth are ranked according to the value of path confidence. The proposed system is more robust since it uses all the available knowledge sources, including context, run time at every intermediate step.