Preprocessing steps for English-Arabic translation. (c2007)

Abstract

Parallel corpus is an essential resource in any statistical machine translation system. It constitutes the basic block for training the system and creating translation and language models that acts as the knowledge base for translation. In this thesis, we present preprocessing steps for English-Arabic Translation. These steps will help in improving the word alignment in the machine learning phase of statistical machine translation. The aim is to make the frequency of Arabic words increase in the text and to minimize the number of words in English and Arabic sentences by splitting them. The preprocessing steps include filtering the Arabic texts from diacritizations and other unnecessary characters, separating the prefixes and suffixes from Arabic words, and splitting English-Arabic sentence pairs according to predetermined stopwords. We apply our technique on a parallel corpus taken from the United Nation's documents. Our results show that it is essential to preprocess English-Arabic text. We obtained an error rate of7.7% on splitting the sentence by stopwords and around 5% when splitting on the comma.