This paper presents a system to extract the logical structure of handwritten mail documents. It consists in two
joined tasks: the segmentation of documents into blocks and the labeling of such blocks. The main considered
label classes are: addressee details, sender details, date, subject, text body, signature. This work has to face
with difficulties of unconstrained handwritten documents: variable structure and writing.
We propose a method based on a geometric analysis of the arrangement of elements in the document. We
give a description of the document using a two-dimension grammatical formalism, which makes it possible to
easily introduce knowledge on mail into a generic parser. Our grammatical parser is LL(k), which means several
combinations are tried before extracting the good one. The main interest of this approach is that we can deal
with low structured documents. Moreover, as the segmentation into blocks often depends on the associated
classes, our method is able to retry a different segmentation until labeling succeeds.
We validated this method in the context of the French national project RIMES, which proposed a contest on
a large base of documents. We obtain a recognition rate of 91.7% on 1150 images.