Skip to content

afarajian/PersianTokenizer

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 

Repository files navigation

PersianTokenizer

An open source text normalizer and tokenizer for Persian (Version 1.0)

Written by M. Amin Farajian at HLT-MT group in FBK, Italy.

The text normalization rules are mostly borrowed from the "Persian Pre-processor" script written by Mojgan Seraji (http://stp.lingfil.uu.se/~mojgan/preper.html).

If you use this software, please cite: Nicola Bertoldi, M. Amin Farajian, Prashant Mathur, Nicholas Ruiz, Marcello Federico. "FBK’s Machine Translation Systems for the IWSLT 2013 Evaluation Campaign" In Proceedings of the International Workshop on Spoken Language Translation 2013, Heidelberg, Germany, December 2013.

About

An open source text normalizer and tokenizer for Persian

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages