Difference between revisions of "Turkic languages"
Firespeaker (talk | contribs) |
Firespeaker (talk | contribs) (→Transducers: Tatar update) |
||
Line 41: | Line 41: | ||
|| <code>[[apertium-tat]]</code> |
|| <code>[[apertium-tat]]</code> |
||
|| [[Tatar]] |
|| [[Tatar]] |
||
|| {{#lst:apertium-tat/stats|nativename}} |
|||
|| татар теле |
|||
|| <code>tt</code> |
|| <code>tt</code> |
||
|| <code>tat</code> |
|| <code>tat</code> |
||
|| HFST (lexc+twol) |
|| HFST (lexc+twol) |
||
|| {{#lst:apertium-tat/stats|state}} |
|||
|| production |
|||
|align="right"| {{#lst:apertium-tat/stats|stems}} |
|align="right"| {{#lst:apertium-tat/stats|stems}} |
||
|align="center"| [[apertium-tat#Current_State|~{{:apertium-tat/stats/average}}%]] |
|align="center"| [[apertium-tat#Current_State|~{{:apertium-tat/stats/average}}%]] |
||
|| |
|| {{#lst:apertium-tat/stats|location}} |
||
|| {{#lst:apertium-tat/stats|authors}} |
|||
|| [[User:Ilnar.salimzyan|Ilnar]], [[User:Francis Tyers|Fran]], [[User:Firespeaker|Jonathan]], Milli |
|||
|- |
|- |
||
| <code>[[apertium-kir]]</code> |
| <code>[[apertium-kir]]</code> |
Revision as of 22:26, 2 March 2014
The Turkic languages include Turkish, Azeri, Uzbek, Uyghur, Kazakh, Kyrgyz, Tatar, Chuvash, Turkmen, Bashkir, Sakha, and several dozen other languages. The languages are related with varying levels of mutual intelligibility. A team called Apertium Turkic aims to enhance MT coverage of these languages with apertium.
The ultimate master plan involves generating independent finite-state transducers for each language, and then making individual dictionaries and transfer rules for every pair. The current status of these goals is listed below.
Status
The ultimate goal is to have multi-purposable transducers for a variety of Turkic languages. These can then be paired for X→Y translation with the addition of a CG for language X and transfer rules / dictionary for the pair X→Y. Below is listed development progress for each language's transducers and dictionary pairs.
Transducers
- See also: Turkic lexicon and Languages
Once a transducer has ~80% coverage on a range of medium-large corpora we can say it is "working". Over 90% and it can be considered to be "production".
name | Language | native name | ISO 639 | formalism | state | stems | coverage | location | primary authors | |
---|---|---|---|---|---|---|---|---|---|---|
-2 | -3 | |||||||||
apertium-kaz
|
Kazakh | қазақ тілі | kk
|
kaz
|
HFST (lexc+twol) | production | 36,595 | ~94.5% | apertium-kaz (languages) | Ilnar, Jonathan, Fran, Nathan |
apertium-tat
|
Tatar | татар теле | tt
|
tat
|
HFST (lexc+twol) | production | 55,702 | ~91% | apertium-tat (languages) | Ilnar, Fran, Jonathan, Röstäm |
apertium-kir
|
Kyrgyz | кыргыз тили | ky
|
kir
|
HFST (lexc+twol) | working | 14,424 | ~90.4% | apertium-kir (languages) | Jonathan, Mirlan, Fran |
apertium-tur
|
Turkish | Türk dili | tr
|
tur
|
HFST (lexc+twol) | working | 17,221 | ~87.3% | apertium-tur (incubator) | Fran, Gianluca |
apertium-chv |
Chuvash | чӑваш чӗлхи | cv |
chv |
HFST (lexc+twol) | development | 8,579 | ~85% | apertium-chv (languages) | Hèctor |
apertium-kum |
Kumyk | къумукъ тил | - |
kum |
HFST (lexc+twol) | development | 4,918 | ~90.2% | apertium-kum (languages) | Fran, Jonathan |
apertium-uzb |
Uzbek | o'zbek tili | uz |
uzb |
HFST (lexc+twol) | development | 34,470 | ~82.9% | apertium-uzb (languages) | |
apertium-bak
|
Bashkir | башҡорт теле | ba
|
bak
|
HFST (lexc+twol) | development | 2,827 | ~66% | apertium-bak (languages) | Fran, Jonathan, Ilnar, Milli |
apertium-nog
|
Nogay | Ногай тили | -
|
nog
|
HFST (lexc+twol) | development | 1,385 | ~81.4% | apertium-nog (languages) | Fran, Jonathan |
apertium-tuk
|
Turkmen | Türkmençe | tk
|
tuk
|
HFST (lexc+twol) | development | 2,988 | ~70.7% | apertium-tuk (languages) | Fran |
apertium-kaa
|
Karakalpak | Qaraqalpaq tili | -
|
kaa
|
HFST (lexc+twol) | prototype | 25,545 | ~86.1% | apertium-kaa (incubator) | Fran |
apertium-sah
|
Yakut | Саха тыла | -
|
sah
|
HFST (lexc+twol) | prototype | 11,531 | ~89.6% | apertium-sah (incubator) | Fran |
apertium-tyv
|
Tuvan | Тыва дыл | -
|
tyv
|
HFST (lexc+twol) | prototype | 11,695 | ~92.7% | apertium-tyv (incubator) | Fran |
apertium-aze
|
Azerbaijani | Azərbaycan dili | az
|
aze
|
SFST | not known to work | apertium-tur-aze (staging) | Gianluca |
Turkic languages by subgroup
- Oğuz: Turkish, Azeri, Turkmen, Gagauz, Trukhmen, Bojnur, etc.
- Qarluq: Uzbek, Uyghur
- Qıpçaq
- Aralo-Caspian: Kazakh, Karakalpak, Nogay (, Kyrgyz)
- North: Tatar, Bashqort
- West: Kumyk, Karachay-Balkar, Karaim, Crimean Tatar
- Qıpçaq-Yenisei: South Altay (Yenisei), Kyrgyz (Qıpçaq)
- Yeniseian: Khakas, Shor, Chulym, Baraba Tatar, North Altay (, South Altay)
- Sayan: Tuvan, Tofa, Tozhu, Dukha, Soyot
- Lena: Sakha, Dolgan
- Arğu: Halaj
- Oğur: Chuvash
Pairs
Some Turkic languages that are particularly similar to one another (and hence have high levels of mutual intelligibility) include those in the following list:
- Turkish, Azerbaycani, (Turkmen)
- Qazaq, Qaraqalpaq, Noğay, (Qırğız)
- Kumyk, Karachay-Balkar
- Tatar, Bashqort
- Uzbek, Uyğur
- Shor, Khakas(, Altay)
- Sakha, Dolgan
- Tuvan, Tofa
Chuvash is very distant from other Turkic languages and is not even partially mutually intelligible with any of them.
Table of existing pairs
- See also: Turkic-Turkic translator
Text in italics denotes language pairs in the incubator. Regular text denotes a developing language pair in nursery, while text in bold denotes a stable well-working language pair in trunk and text in bold and italics denotes a pair in staging. Bidix stems as counted with dixcounter are displayed below.
tur | aze | tuk | uzb | kir | kaz | tat | bak | chv | nog | kaa | kum | uig | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
tur | - | tur-aze |
tuk-tur |
tur-uzb |
tur-kir |
tur-tat |
cv-tr |
||||||
aze | tur-aze |
- | |||||||||||
tuk | tuk-tur |
- | |||||||||||
uzb | tur-uzb |
- | kir-uzb |
||||||||||
kir | tur-kir |
kir-uzb |
- | kaz-kir |
tat-kir |
||||||||
kaz | kaz-kir |
- | 'kaz-tat ' |
nog-kaz |
kaz-kaa |
||||||||
tat | tur-tat |
tat-kir |
'kaz-tat ' |
- | tat-bak |
cv-tt |
|||||||
bak | tat-bak |
- | |||||||||||
chv | cv-tr |
cv-tt |
- | ||||||||||
nog | nog-kaz |
- | |||||||||||
kaa | kaz-kaa |
- | |||||||||||
kum | - | ||||||||||||
uig | - | ||||||||||||
eng | tr-en |
ky-en |
eng-kaz |
||||||||||
epo | eo-tr |
||||||||||||
khk | khk-kaz |
||||||||||||
rus | tt-ru |
cv-ru |
Roadmap
- Stable release of apertium-kaz
- Stable release of apertium-tat
- Stable release of apertium-kaz-tat
- Rework apertium-kir to match new standards
- Bring apertium-bak up to date (based on apertium-tat)
- Expand apertium-tat-bak
- Beta release of apertium-kaz-kir
- Expand apertium-tuk
- Expand apertium-chv
- Basic transducers for:
Getting involved
We have a work plan for developing Turkic-Turkic translators and are working on a how-to for building a Turkic lexicon. Please come talk to us on IRC or contact us on the apertium-turkic mailing list.
Samples
Article 1 of the Universal Declaration of Human Rights:
All human beings are born free and equal in dignity and rights. They are endowed with reason and conscience and should act towards one another in a spirit of brotherhood.
Language | Text |
---|---|
Kazakh | Барлық адамдар тумысынан азат және қадір‐қасиеті мен кұқықтары тең болып дүниеге келеді. Адамдарға ақыл‐парасат, ар‐ождан берілген, сондықтан олар бір‐бірімен туыстық, бауырмалдық қарым‐қатынас жасаулары тиіс. |
Tatar | Барлык кешеләр дә азат һәм үз абруйлары һәм хокуклары ягыннан тиң булып туалар. Аларга акыл һәм вөҗдан бирелгән һәм бер-берсенә карата туганнарча мөнәсәбәттә булырга тиешләр. |
Kirghiz | Бардык адамдар өз беделинде жана укуктарында эркин жана тең укуктуу болуп жаралат. Алардын аң‐сезими менен абийири бар жана бири‐бирине бир туугандык мамилекылууга тийиш. |
Turkish | Bütün insanlar hür, haysiyet ve haklar bakımından eşit doğarlar. Akıl ve vicdana sahiptirler ve birbirlerine karşı kardeşlik zihniyeti ile hareket etmelidirler. |
Uzbek (Latin) | Barcha odamlar erkin, qadr-qimmat ba huquqlarda tang bo'lib tug'iladilar. Ular aql va vijdon sohibidilar va bir-birla iga birodarlarcha muomala qilishlari zarur. |
Tuva | Бүгү кижилер хостуг база мөзүзү болгаш эргелери дең кылдыр төрүттүнер. Оларга угаансарыыл болгаш арын-нүүр бердинген болур болгаш олар бот-боттарынга акы-дуңмалышкы хамаарылганы көргүзер ужурлуг. |
Vulnerability
This table summarizes the vulnerability of various Turkic languages. Vulnerability data is derived from the ‘Atlas of the World’s Languages in Danger, © UNESCO, http://www.unesco.org/culture/languages-atlas’ and Ethnologue.
Language | ISO639-3 | Location | Speakers | Status | |
---|---|---|---|---|---|
Ethnologue | UNESCO | ||||
Chagatai | chg
|
Turkmenistan | 0 | 10 (Extinct) | - |
Salchuq | slq
|
Iran | 0 | 10 (Extinct) | - |
Chulym | clw
|
Russian Federation | 44 | 8b (Nearly extinct) | 4 (Critically endangered) |
Karaim | kdr
|
Lithuania & Ukraine | 56 | 8b (Nearly extinct) | 5 (Extinct) |
Tofa (Karagas) | kim
|
Russian Federation | 90 | 8b (Nearly extinct) | 4 (Critically endangered) |
Krimchak | jct
|
Ukraine | 200 | 8b (Nearly extinct) | - |
Ili Turki | ili
|
China | 120 | 8a (Moribund) | 3 (Severely endangered) |
Yugur (Western/Saryg) | ybe
|
China | 4,600 | 8a (Moribund) | 3 (Severely endangered) |
Balkan Gagauz Turkish | bgx
|
Turkey | 331,000 | 7 (Shifting) | - |
Dolgan | dlg
|
Russian Federation | 1,050 | 6b (Threatened) | 2 (Definitely endangered) |
Shor | cjs
|
Russian Federation | 2,840 | 6b (Threatened) | 3 (Severely endangered) |
Ainu | aib
|
China | 6,570 | 6b (Threatened) | - |
Altay, Northern | atv
|
Russian Federation | 57,400 | 6b (Threatened) | 3 (Severely endangered) |
Salar | slr
|
China | 70,000 | 6b (Threatened) | 1 (Vulnerable) |
Urum | uum
|
Georgia, Russian Federation, Ukraine | 192,700 | 6b (Threatened) | 2 (Definitely endangered) |
Khalaj | klj
|
Iran | 42,100 | 6a (Vigorous) | 1 (Vulnerable) |
Khorasani | kmz
|
Iran | 400,000 | 6a (Vigorous) | 1 (Vulnerable) |
Kashkay | qxq
|
Iran | 1,500,000 | 6a (Vigorous) | - |
Khakas | kjh
|
Russian Federation | 42,610 | 5 (Developing) | 5 (Extinct) |
Nogay | nog
|
Russian Federation | 87,410 | 5 (Developing) | 2 (Definitely endangered) |
Gagauz | gag
|
Moldova | 173,920 | 5 (Developing) | - |
Karakalpak | kaa
|
Uzbekistan | 410,410 | 5 (Developing) | - |
Kumyk | kum
|
Russian Federation | 426,550 | 5 (Developing) | 1 (Vulnerable) |
Tuva | tyv
|
China, Mongolia, Russian Federation | 283,400 | 4 (Educational) | 1 (Vulnerable) |
Karachay-Balkar | krc
|
Russian Federation | 310,730 | 4 (Educational) | 1 (Vulnerable) |
Crimean Tatar | crh
|
Ukraine | 475,540 | 4 (Educational) | - |
Chuvash | chv
|
Russian Federation | 1,077,420 | 4 (Educational) | 1 (Vulnerable) |
Bashkort | bak
|
Russian Federation | 1,221,340 | 4 (Educational) | 1 (Vulnerable) |
Uzbek, Southern | uzs
|
Afghanistan | 2,961,980 | 3 (Wider communication) | - |
Azerbaijani, Southern | azb
|
Iran | 16,913,280 | 3 (Wider communication) | - |
Altay, Southern | alt
|
Russian Federation | 57,400 | 2 (Provincial) | 2 (Definitely endangered) |
Yakut | sah
|
Russian Federation | 450,000 | 2 (Provincial) | 1 (Vulnerable) |
Tatar | tat
|
Russian Federation | 5,407,550 | 2 (Provincial) | 2 (Definitely endangered) |
Uyghur | uig
|
China | 8,791,610 | 2 (Provincial) | - |
Kyrgyz | kir
|
Kyrgyzstan | 2,941,930 | 1 (National) | - |
Azerbaijani, Northern | azj
|
Azerbaijan | 7,324,270 | 1 (National) | - |
Turkmen | tuk
|
Turkmenistan | 7,560,560 | 1 (National) | - |
Kazakh | kaz
|
Kazakhstan | 8,077,770 | 1 (National) | - |
Uzbek, Northern | uzn
|
Uzbekistan | 18,968,250 | 1 (National) | - |
Turkish | tur
|
Turkey | 50,733,420 | 1 (National) | - |
Tagset
Rough guide to tagsets in various Turkic language transducers, with an eye to keeping stuff that is basically the same tagged the same. In the following table, A stands for Apertium and T stands for TRmorph (See also the general tagset list).
Phenomenon | Morphology | Description | Tag(s) | Language(s) | Notes |
---|---|---|---|---|---|
Part of speech | |||||
Noun | <n> |
||||
Proper noun | <np> |
||||
Determiner | <det> |
||||
Numeral | <num> |
||||
Adjective | <adj> |
incl. var/yok | |||
Adverb | <adv> |
||||
Pronoun | <prn> |
||||
Verb | <v> |
||||
Auxiliary verb | <vaux> |
||||
Copula | <cop> |
||||
Adverb | <adv> |
||||
Postadverb | <postadv> |
||||
Postposition | <post> |
||||
Particle[1] | <part> |
||||
Coordinating conjunction | <cnjcoo> |
||||
Subordinating conjunction | <cnjsub> |
||||
Adverbial conjunction | <cnjadv> |
||||
Abbreviation | <abbr> |
||||
Personal Title | <title> |
||||
Interjection | <ij> |
Әлбетте(kaz ), жок(kir ) (cf. Adj)
| |||
Proper noun types | |||||
Toponym | <top> |
||||
Anthroponym | <ant> |
||||
Patronym | <pat> |
||||
Cognomen (Surname) | <cog> |
||||
Acronym | <acr> |
||||
Other | <al> |
||||
Pronoun types | |||||
Personal | <pers> |
||||
Ordinal | <ord> |
||||
Demonstrative | <dem> |
||||
Indefinite | <ind> |
||||
Interrogative | <itg> |
||||
Reflexive | <ref> |
||||
Quantifier | <qnt> |
||||
Positive | <pst> |
||||
Negative | <neg> |
||||
Comparative | <comp> |
||||
Reciprocal | <recip> |
||||
First person | <p1> |
||||
Second person | <p2> |
||||
Third person | <p3> |
||||
Numeral types | |||||
Substantive | Substantive form of numerals (when they are used as the head of the noun phrase) | <subst> |
|||
Ordinal | <ord> |
Chuvash: -мĂш | |||
Distributive | <dist> |
Chuvash: -шĂр | |||
Collective | <coll> |
Chuvash: -Ăн | |||
Case | |||||
Nominative case (unmarked) | <nom> |
||||
Genitive case | <gen> |
||||
Dative case | <dat> |
||||
Locative case | <loc> |
||||
Ablative case | -DAn | Case indicating movement away | <abl> |
Pan-turkic | |
Comitative case | <com> |
||||
Terminative case | <ter> |
Chuvash: -ччен | |||
Benefactive (Purposive) case | <ben> |
Chuvash: -шĂн | |||
Allative (Directive) case | Case indicating motion towards something | <all> |
Chuvash: -АллА | ||
Posession | |||||
1st pers sg | <px1sg> |
||||
1st pers pl | <px1pl> |
||||
2nd pers sg | <px2sg> |
||||
2nd pers pl | <px2pl> |
||||
3rd pers sg | <px3sg> |
||||
3rd pers pl | <px3pl> |
||||
3rd pers sg or pl | <px3sp> |
||||
Gender | |||||
Masculine | <m> |
||||
Feminine | <f> |
||||
Masculine / feminine | <mf> |
||||
Number | |||||
Singular | <sg> |
||||
Plural | <pl> |
||||
Tense, aspect, mood | |||||
Present tense | <pres> |
||||
Present continuous tense | <cont> |
Turkish: -{bI}yor | |||
Evidential tenseless/past tense | <evid> |
Turkish: -m{I}ş (<past><evid>) | |||
Past tense | <past> |
Kyrgyz: -{G}{A}н | |||
Definite past tense | <ifi> |
Turkish: -{D}{I} | |||
Imperfect | <pii> |
Turkish: Aorist + -m{A}kt{A} | |||
Past habitual tense | <pih> |
Turkish: Aorist + -{D}{I} | |||
Future tense | <fut> |
Turkish: -{bY}{A}c{A}{k} | |||
Imperative | -ø | Mood for giving orders | <imp> A, <t_imp> T |
Pan-turkic | Turkish: -ø |
Conditional | <cond> |
Turkish: -s{A} | |||
Aorist | <aor> |
Turkish: -{A}r or -{bI}r | |||
Optative | <opt> |
Turkish: -{bY}{A}, Kirghiz: -мAк>чI | |||
Obligative | <oblig> |
Turkish: -m{A}l{I} | |||
Potential | <pot> |
Kirghiz: -чUдAй | |||
Not-yet tense | <notyet> |
Kirghiz: -E элек | |||
Non-finite verb forms | |||||
Gerund | makes verbs usable as nouns | <ger> , <vn> ? |
|||
Verbal adjective | makes verbs usable as adjectives | <vadj> |
|||
Participle | makes verb a matrix verb usable auxiliaries and modals | <part> , <vadv> ? |
|||
??? | makes verb usable as first of a dual-predicate construction | ?? | |||
Infinitive | citation form of verb and use in certain constructions | <inf> ? |
|||
Gerund #1 | <ger1> |
Turkish: -m{A} | |||
Gerund #2 | <ger2> |
Turkish: -m{A}{K} | |||
Gerund #3 | <ger3> |
Turkish: -{D}{I}{k} | |||
Gerund #4 | <ger4> |
Turkish: -{bY}{I}ş | |||
Gerund #5 | <ger5> |
Turkish: -{bY}{A}n | |||
Gerund #6 | <ger6> |
Turkish: -{bY}{A}r{A}k | |||
Gerund #7 | <ger10> |
Turkish: -{bY}{I}p | |||
Future gerund #1 | <fger> |
Turkish: -{bY}{A}c{A}{k} | |||
Imperfect participle #1 | <fger> |
Turkish: -{bY}{A}r{A}{k} | |||
Productive verbal derivation | |||||
Passive | <pass> |
||||
Causative | <caus> |
||||
Cooperative | <coop> |
-{I}ш(kir ), -{I}с(kaz )
| |||
Transitivity | |||||
Transitive, переходный | <tv> |
||||
Intransitive, непереходный | <iv> |
||||
Modal/question/etc. "particles" | |||||
Question | used with yes/no, focus, etc. question morphemes | <qst> |
most-all | {М}{А}(kaz ), {B}{I}(kir ), m{I}(tur ); +ше(kaz ), ч{I}(kir )
| |
Emphatic | used with imperative/optative and other coercive verb forms | <emph> |
most | +ш{I}(kaz ), +ч{I}(kir ),(tat ), s{A}n{A}(tur )
|
Official poem
- Kovayla bira içerim, ama sen bilmezsin. Yarın gelir misin?
- Vedrəyle pivə içirəm, ama sen bilməzsən. Yarın gələrmisən?
- Чиләк белән сыра эчәм, әмма син белмисең. Иртәгә киләсеңме?
- Шелекпен сыра ішемін, бірақ сен білмейсің. Ертең келесің бе?
- Чака менен сыра ичем, бирок сен билбейсиң. Эртең келесиңби?
- Челек булан пиво ичемен амма сен билмейсен. Эртен гелемисен?
- Chelak bilan pivo ichaman lekin sen bilmaysan. Ertaga kelasanmi?
Footnotes
- ↑ Warning: The use of the particle tag is highly discouraged.