The Rosetta phone: a hand-held device for automatic translation of signs in natural images

Syed Ali Raza Jafri; Aravind K. Mikkilineni; Mireille Boutin; Edward J. Delp

doi:10.1117/12.766648

Translator Disclaimer

28 February 2008 The Rosetta phone: a hand-held device for automatic translation of signs in natural images

Syed Ali Raza Jafri; Aravind K. Mikkilineni; Mireille Boutin; Edward J. Delp

Author Affiliations +

Proceedings Volume 6821, Multimedia on Mobile Devices 2008; 68210E (2008) https://doi.org/10.1117/12.766648
Event: Electronic Imaging, 2008, San Jose, California, United States

Abstract

When traveling in a region where the local language is not written using the Roman alphabet, translating written text (e.g., documents, road signs, or placards) is a particularly difficult problem since the text cannot be easily entered into a translation device or searched using a dictionary. To address this problem, we are developing the "Rosetta Phone," a handheld device (e.g., PDA or mobile telephone) capable of acquiring a picture of the text, identifying the text within the image, and producing both an audible and a visual English interpretation of the text. We started with English, as a developement language, for which we achieved close to 100% accuracy in identifying and reading text. We then modified the system to be able to read and translate words written using the Arabic character set. We currently achieve approximately 95% accuracy in reading words from a small directory of town names.

Citation Download Citation

Syed Ali Raza Jafri, Aravind K. Mikkilineni, Mireille Boutin, and Edward J. Delp "The Rosetta phone: a hand-held device for automatic translation of signs in natural images", Proc. SPIE 6821, Multimedia on Mobile Devices 2008, 68210E (28 February 2008); https://doi.org/10.1117/12.766648

ACCESS THE FULL ARTICLE