Kazakh and Tatar

From Apertium

Contents 1 General information 1.1 Demonstration 2 Installation 3 Developers 3.1 Development workflow 3.1.1 Adding words 3.1.2 Adding language-pair-specific stems to the lexc files

This is a language pair translating between Kazakh and Tatar. The pair is currently located in staging.

[edit] General information

• The Kazakh transducer has 10,672 stems and ~89% coverage over random corpora
• The Tatar transducer has 9,011 stems and ~87.8% coverage over random corpora

[edit] Demonstration

• $ echo "бұл аударушымен татарша жазылған мәтіндер қазақша аударып оқыса болады" | apertium -d . kaz-tat

бу аударучы белән татарча язылган текстләр казакъча аударып укыша була

[edit] Installation

You will need:

• hfst (svn ≥r1916)
  • foma
    • flex
• apertium
  • lttoolbox
• CG
  • ICU
• apertium-lex-tools

[edit] Developers

Information on what remains to be done for this pair can be found at the /TODO list.

[edit] Development workflow

We work on the transducers (apertium-kaz and apertium-tat) individually, and use a special process to import to the pair transducers that contain only the words found in the bidix. The following documents this process.

[edit] Adding words

In order to add a new word (and its translation equivalent) to the Kazakh-Tatar translator, you have to do the following:

1. add an entry in the bilingual dictionary — apertium-kaz-tat.kaz-tat.dix file in staging/apertium-kaz-tat directory,
2. add an entry in the Kazakh monolingual dictionary — apertium-kaz.kaz.lexc file, which, as the name indicates, is in the incubator/apertium-kaz directory,
3. add an entry in the Tatar monolingual dictionary — apertium-tat.tat.lexc file in incubator/apertium-tat,
4. cd to the staging/apertium-kaz-tat directory in terminal,
5. run update-morphs.bash script and recompile.

This script runs the trim-lexc.py script, which itself lies in /trunk/apertium-tools, and copies its output over to the kaz-tat directory, renaming the files to fit the conventions.

In addition, it copies .twol and .rlx files from the apertium-kaz and apertium-tat directories.

The same workflow applies for any other pair involving Kazakh or Tatar — if you need to change something in .lexc, .twol or .rlx files of these languages, you do so in apertium-kaz and apertium-tat directories respectively, and then copy these files to the "bilingual" directory you are working on (either using update-morphs.bash script or manually).

[edit] Adding language-pair-specific stems to the lexc files

Sometimes we have to translate a word into Kazakh or Tatar with two or more words, e.g.:

<e><p><l>everywhere<s n="adv"/></l><r>барлық<b/>жерде<s n="adv"/></r></p></e>

In order to make it work, we will need to add барлық жерде as a single adverb in kaz.lexc, like this:

барлық% жерде:барлық% жерде ADV ; ! ""

But, certainly, these are two words in Kazakh — a determiner and a noun — and we don't want that the standalone Kazakh morphological analyzer would contain them as one word. In order to know that this entry was added for a specific language pair, we mark it with Use/MT at the end of the line:

барлық% жерде:барлық% жерде ADV ; ! "" Use/MT

That way, we can collaborate on one single file across many pairs, but in the same time we are able to "clean up" the lexicon for standalone use if needed.