Turkic languages

From Apertium
Revision as of 23:08, 1 January 2014 by Sushain (talk | contribs)
Jump to navigation Jump to search

The Turkic languages include Turkish, Azeri, Uzbek, Uyghur, Kazakh, Kyrgyz, Tatar, Chuvash, Turkmen, Bashkir, Sakha, and several dozen other languages. The languages are related with varying levels of mutual intelligibility. A team called Apertium Turkic aims to enhance MT coverage of these languages with apertium.

The ultimate master plan involves generating independent finite-state transducers for each language, and then making individual dictionaries and transfer rules for every pair. The current status of these goals is listed below.

Status

The ultimate goal is to have multi-purposable transducers for a variety of Turkic languages. These can then be paired for X→Y translation with the addition of a CG for language X and transfer rules / dictionary for the pair X→Y. Below is listed development progress for each language's transducers and dictionary pairs.

Transducers

See also: Turkic lexicon

Once a transducer has ~80% coverage on a range of medium-large corpora we can say it is "working". Over 90% and it can be considered to be "production".

name Language native name ISO 639 formalism state stems coverage location primary authors
-2 -3
apertium-kaz Kazakh қазақ тілі kk kaz HFST (lexc+twol) production 36,595 ~94.5% apertium-kaz (languages) Ilnar, Jonathan, Fran, Nathan
apertium-tat Tatar татар теле tt tat HFST (lexc+twol) working 55,702 ~91% apertium-tat (languages) Ilnar, Fran, Jonathan, Milli
apertium-kir Kyrgyz кыргыз тили ky kir HFST (lexc+twol) working 8,564 ~90.4% apertium-kir (languages) Jonathan, Mirlan, Fran
apertium-tur Turkish Türk dili tr tur HFST (lexc+twol) working 17,221 ~87.3% apertium-tur (incubator) Fran, Gianluca
apertium-chv Chuvash чӑваш чӗлхи cv chv HFST (lexc+twol) development 8,579 ~85% apertium-chv (languages) Hèctor
apertium-kum Kumyk къумукъ тил - kum HFST (lexc+twol) development 4,918 ~90.2% apertium-kum (languages) Fran, Jonathan
apertium-uzb Uzbek o'zbek tili uz uzb HFST (lexc+twol) development 34,470 ~82.9% apertium-uzb (languages)
apertium-bak Bashkir башҡорт теле ba bak HFST (lexc+twol) development 2,827 ~66% apertium-bak (languages) Fran, Jonathan, Ilnar, Milli
apertium-nog Nogay Ногай тили - nog HFST (lexc+twol) development 1,385 ~81.4% apertium-nog (languages) Fran, Jonathan
apertium-tuk Turkmen Türkmençe tk tuk HFST (lexc+twol) development 2,988 ~70.7% apertium-tuk (languages) Fran
apertium-kaa Karakalpak Qaraqalpaq tili - kaa HFST (lexc+twol) prototype 25,545 ~86.1% apertium-kaa (incubator) Fran
apertium-tyv Tuvan Тыва дыл - tyv HFST (lexc+twol) prototype 11,695 ~92.7% apertium-tyv (incubator) Fran
apertium-aze Azerbaijani Azərbaycan dili az aze SFST not known to work apertium-tur-aze (staging) Gianluca

Turkic languages by subgroup

Pairs

Some Turkic languages that are particularly similar to one another (and hence have high levels of mutual intelligibility) include those in the following list:

Chuvash is very distant from other Turkic languages and is not even partially mutually intelligible with any of them.

Table of existing pairs

See also: Turkic-Turkic translator

Text in italic denotes language pairs under development / in the incubator. Regular text denotes a functioning language pair in staging, while text in bold denotes a stable well-working language pair in trunk. As counted with dixcounter

tur aze tuk uzb kir kaz tat chv bak uig kum nog kaa
tur - tur-aze
tuk-tur
tur-uzb
tur-kir
tur-tat
cv-tr
aze tur-aze
-
tuk tuk-tur
-
uzb tur-uzb
- kir-uzb
kir tur-kir
kir-uzb
- kaz-kir
tat-kir
kaz kaz-kir
- 'kaz-tat
'
nog-kaz
kaz-kaa
tat tur-tat
tat-kir
'kaz-tat
'
- cv-tt
tat-bak
chv cv-tr
cv-tt
-
bak tat-bak
-
uig -
kum -
nog nog-kaz
-
kaa kaz-kaa
-
eng tr-en
ky-en
eng-kaz
epo eo-tr
khk khk-kaz
rus tt-ru
cv-ru

Roadmap

  • Stable release of apertium-kaz
  • Stable release of apertium-tat
  • Stable release of apertium-kaz-tat
  • Rework apertium-kir to match new standards
  • Bring apertium-bak up to date (based on apertium-tat)
  • Expand apertium-tat-bak
  • Beta release of apertium-kaz-kir
  • Expand apertium-tuk
  • Expand apertium-chv
  • Basic transducers for:
    • Khakas
    • Tuvan
    • Sakha
    • Shor
    • Qaralpaq (based on Tatar Kazakh, probably, no?)
    • Uzbek
    • Uyghur
    • Nogay
    • Kumyk

Getting involved

We have a work plan for developing Turkic-Turkic translators and are working on a how-to for building a Turkic lexicon. Please come talk to us on IRC or contact us on the apertium-turkic mailing list.

Samples

Article 1 of the Universal Declaration of Human Rights:

All human beings are born free and equal in dignity and rights. They are endowed with reason and conscience and should act towards one another in a spirit of brotherhood.

Language Text
Kazakh Барлық адамдар тумысынан азат және қадір‐қасиеті мен кұқықтары тең болып дүниеге келеді. Адамдарға ақыл‐парасат, ар‐ождан берілген, сондықтан олар бір‐бірімен туыстық, бауырмалдық қарым‐қатынас жасаулары тиіс.
Tatar Барлык кешеләр дә азат һәм үз абруйлары һәм хокуклары ягыннан тиң булып туалар. Аларга акыл һәм вөҗдан бирелгән һәм бер-берсенә карата туганнарча мөнәсәбәттә булырга тиешләр.
Kirghiz Бардык адамдар өз беделинде жана укуктарында эркин жана тең укуктуу болуп жаралат. Алардын аң‐сезими менен абийири бар жана бири‐бирине бир туугандык мамилекылууга тийиш.
Turkish Bütün insanlar hür, haysiyet ve haklar bakımından eşit doğarlar. Akıl ve vicdana sahiptirler ve birbirlerine karşı kardeşlik zihniyeti ile hareket etmelidirler.
Uzbek (Latin) Barcha odamlar erkin, qadr-qimmat ba huquqlarda tang bo'lib tug'iladilar. Ular aql va vijdon sohibidilar va bir-birla iga birodarlarcha muomala qilishlari zarur.
Tuva Бүгү кижилер хостуг база мөзүзү болгаш эргелери дең кылдыр төрүттүнер. Оларга угаансарыыл болгаш арын-нүүр бердинген болур болгаш олар бот-боттарынга акы-дуңмалышкы хамаарылганы көргүзер ужурлуг.

Vulnerability

This table summarizes the vulnerability of various Turkic languages. Vulnerability data is derived from the ‘Atlas of the World’s Languages in Danger, © UNESCO, http://www.unesco.org/culture/languages-atlas’.

Language ISO639-3 Areas Vulnerability
Karaim (Crimea) kdr Ukraine 5 - Extinct
Kamas Turk kjh Russian Federation 5 - Extinct
Chulym Turk clw Russian Federation 4 - Critically endangered
Tofa kim Russian Federation 4 - Critically endangered
Shor cjs Russian Federation 3 - Severely endangered
Northern Altay atv Russian Federation 3 - Severely endangered
Saryg Yugur ybe China 3 - Severely endangered
Ili Turk ili China 3 - Severely endangered
Siberian Tatar tat Russian Federation 2 - Definitely endangered
Nogay (Caucasus) nog Russian Federation 2 - Definitely endangered
Dolgan dlg Russian Federation 2 - Definitely endangered
Southern Altay alt Russian Federation 2 - Definitely endangered
Urum uum Georgia, Russian Federation, Ukraine 2 - Definitely endangered
Bashkir bak Russian Federation 1 - Vulnerable
Kumyk kum Russian Federation 1 - Vulnerable
Karachay-Balkar krc Russian Federation 1 - Vulnerable
Khorasani Turk kmz Iran (Islamic Republic of) 1 - Vulnerable
Salar slr China 1 - Vulnerable
Khalaj klj Iran (Islamic Republic of) 1 - Vulnerable
Yakut sah Russian Federation 1 - Vulnerable
Tuvan tyv China, Mongolia, Russian Federation 1 - Vulnerable
Chuvash chv Russian Federation 1 - Vulnerable

Tagset

Rough guide to tagsets in various Turkic language transducers, with an eye to keeping stuff that is basically the same tagged the same. In the following table, A stands for Apertium and T stands for TRmorph (See also the general tagset list).

Phenomenon Morphology Description Tag(s) Language(s) Notes
Part of speech
Noun <n>
Proper noun <np>
Determiner <det>
Numeral <num>
Adjective <adj> incl. var/yok
Adverb <adv>
Pronoun <prn>
Verb <v>
Auxiliary verb <vaux>
Copula <cop>
Adverb <adv>
Postadverb <postadv>
Postposition <post>
Particle[1] <part>
Coordinating conjunction <cnjcoo>
Subordinating conjunction <cnjsub>
Adverbial conjunction <cnjadv>
Abbreviation <abbr>
Personal Title <title>
Interjection <ij> Әлбетте(kaz), жок(kir) (cf. Adj)
Proper noun types
Toponym <top>
Anthroponym <ant>
Patronym <pat>
Cognomen (Surname) <cog>
Acronym <acr>
Other <al>
Pronoun types
Personal <pers>
Ordinal <ord>
Demonstrative <dem>
Indefinite <ind>
Interrogative <itg>
Reflexive <ref>
Quantifier <qnt>
Positive <pst>
Negative <neg>
Comparative <comp>
Reciprocal <recip>
First person <p1>
Second person <p2>
Third person <p3>
Numeral types
Substantive Substantive form of numerals (when they are used as the head of the noun phrase) <subst>
Ordinal <ord> Chuvash: -мĂш
Distributive <dist> Chuvash: -шĂр
Collective <coll> Chuvash: -Ăн
Case
Nominative case (unmarked) <nom>
Genitive case <gen>
Dative case <dat>
Locative case <loc>
Ablative case -DAn Case indicating movement away <abl> Pan-turkic
Comitative case <com>
Terminative case <ter> Chuvash: -ччен
Benefactive (Purposive) case <ben> Chuvash: -шĂн
Allative (Directive) case Case indicating motion towards something <all> Chuvash: -АллА
Posession
1st pers sg <px1sg>
1st pers pl <px1pl>
2nd pers sg <px2sg>
2nd pers pl <px2pl>
3rd pers sg <px3sg>
3rd pers pl <px3pl>
3rd pers sg or pl <px3sp>
Gender
Masculine <m>
Feminine <f>
Masculine / feminine <mf>
Number
Singular <sg>
Plural <pl>
Tense, aspect, mood
Present tense <pres>
Present continuous tense <cont> Turkish: -{bI}yor
Evidential tenseless/past tense <evid> Turkish: -m{I}ş (<past><evid>)
Past tense <past> Kyrgyz: -{G}{A}н
Definite past tense <ifi> Turkish: -{D}{I}
Imperfect <pii> Turkish: Aorist + -m{A}kt{A}
Past habitual tense <pih> Turkish: Aorist + -{D}{I}
Future tense <fut> Turkish: -{bY}{A}c{A}{k}
Imperative Mood for giving orders <imp>A, <t_imp>T Pan-turkic Turkish: -ø
Conditional <cond> Turkish: -s{A}
Aorist <aor> Turkish: -{A}r or -{bI}r
Optative <opt> Turkish: -{bY}{A}, Kirghiz: -мAк>чI
Obligative <oblig> Turkish: -m{A}l{I}
Potential <pot> Kirghiz: -чUдAй
Not-yet tense <notyet> Kirghiz: -E элек
Non-finite verb forms
Gerund makes verbs usable as nouns <ger>, <vn>?
Verbal adjective makes verbs usable as adjectives <vadj>
Participle makes verb a matrix verb usable auxiliaries and modals <part>, <vadv>?
??? makes verb usable as first of a dual-predicate construction ??
Infinitive citation form of verb and use in certain constructions <inf>?
Gerund #1 <ger1> Turkish: -m{A}
Gerund #2 <ger2> Turkish: -m{A}{K}
Gerund #3 <ger3> Turkish: -{D}{I}{k}
Gerund #4 <ger4> Turkish: -{bY}{I}ş
Gerund #5 <ger5> Turkish: -{bY}{A}n
Gerund #6 <ger6> Turkish: -{bY}{A}r{A}k
Gerund #7 <ger10> Turkish: -{bY}{I}p
Future gerund #1 <fger> Turkish: -{bY}{A}c{A}{k}
Imperfect participle #1 <fger> Turkish: -{bY}{A}r{A}{k}
Productive verbal derivation
Passive <pass>
Causative <caus>
Cooperative <coop> -{I}ш(kir), -{I}с(kaz)
Transitivity
Transitive, переходный <tv>
Intransitive, непереходный <iv>
Modal/question/etc. "particles"
Question used with yes/no, focus, etc. question morphemes <qst> most-all {М}{А}(kaz), {B}{I}(kir), m{I}(tur); +ше(kaz), ч{I}(kir)
Emphatic used with imperative/optative and other coercive verb forms <emph> most +ш{I}(kaz), +ч{I}(kir),(tat), s{A}n{A}(tur)

Official poem

  • Kovayla bira içerim, ama sen bilmezsin. Yarın gelir misin?
  • Vedrəyle pivə içirəm, ama sen bilməzsən. Yarın gələrmisən?
  • Чиләк белән сыра эчәм, әмма син белмисең. Иртәгә киләсеңме?
  • Шелекпен сыра ішемін, бірақ сен білмейсің. Ертең келесің бе?
  • Чака менен сыра ичем, бирок сен билбейсиң. Эртең келесиңби?
  • Челек булан пиво ичемен амма сен билмейсен. Эртен гелемисен?
  • Chelak bilan pivo ichaman lekin sen bilmaysan. Ertaga kelasanmi?

Footnotes

  1. Warning: The use of the particle tag is highly discouraged.