Turkic languages

The Turkic languages include Turkish, Azeri, Uzbek, Uyghur, Kazakh, Kyrgyz, Tatar, Chuvash, Turkmen, Bashkir, Sakha, and several dozen other languages. The languages are related with varying levels of mutual intelligibility. A team called Apertium Turkic aims to enhance MT coverage of these languages with apertium.

The ultimate master plan involves generating independent finite-state transducers for each language, and then making individual dictionaries and transfer rules for every pair. The current status of these goals is listed below.

Status

The ultimate goal is to have multi-purpose transducers for a variety of Turkic languages. These can then be paired for X→Y translation with the addition of a CG for language X and transfer rules / dictionary for the pair X→Y. Below is listed development progress for each language's transducers and dictionary pairs.

Transducers

See also: Turkic lexicon and Languages

Once a transducer has ~80% coverage on a range of medium-large corpora we can say it is "working". Over 90% and it can be considered to be "production".

name	Language	native name	ISO 639		formalism	state	stems	coverage	location	primary authors
			-2	-3
apertium-kaz	Kazakh	қазақ тілі	kk	kaz	HFST (lexc+twol)	production	36,595	~94.5%	apertium-kaz (languages)	Ilnar, Jonathan, Fran, Nathan
apertium-tat	Tatar	татар теле	tt	tat	HFST (lexc+twol)	production	55,702	~91%	apertium-tat (languages)	Ilnar, Fran, Jonathan, Röstäm
apertium-kir	Kyrgyz	кыргыз тили	ky	kir	HFST (lexc+twol)	working	14,424	~90.4%	apertium-kir (languages)	Jonathan, Mirlan, Fran
apertium-tur	Turkish	Türk dili	tr	tur	HFST (lexc+twol)	working	17,221	~87.3%	apertium-tur (incubator)	Fran, Gianluca
apertium-chv	Chuvash	чӑваш чӗлхи	cv	chv	HFST (lexc+twol)	development	8,579	~85%	apertium-chv (languages)	Hèctor
apertium-kum	Kumyk	къумукъ тил	-	kum	HFST (lexc+twol)	development	4,918	~90.2%	apertium-kum (languages)	Fran, Jonathan
apertium-uzb	Uzbek	o'zbek tili	uz	uzb	HFST (lexc+twol)	development	34,470	~82.9%	apertium-uzb (languages)
apertium-bak	Bashkir	башҡорт теле	ba	bak	HFST (lexc+twol)	development	2,827	~66%	apertium-bak (languages)	Fran, Jonathan, Ilnar, Milli
apertium-nog	Nogay	Ногай тили	-	nog	HFST (lexc+twol)	development	1,385	~81.4%	apertium-nog (languages)	Fran, Jonathan
apertium-tuk	Turkmen	Türkmençe	tk	tuk	HFST (lexc+twol)	development	2,988	~70.7%	apertium-tuk (languages)	Fran
apertium-kaa	Karakalpak	Qaraqalpaq tili	-	kaa	HFST (lexc+twol)	prototype	25,545	~86.1%	apertium-kaa (incubator)	Beknazar, Fran, Jonathan
apertium-uig	Uyghur	ئۇيغۇر تىلى	ug	uig	HFST (lexc+twol)	prototype	17,585	~54.2%	apertium-uig (incubator)	Jonathan, Märdan, Fran
apertium-crh	Crimean Tatar	Qırımtatar tili	-	crh	HFST (lexc+twol)	prototype	11,757	~85.4%	apertium-crh (incubator)	Fran
apertium-sah	Yakut	Саха тыла	-	sah	HFST (lexc+twol)	prototype	11,531	~89.6%	apertium-sah (incubator)	Fran
apertium-tyv	Tuvan	Тыва дыл	-	tyv	HFST (lexc+twol)	prototype	11,695	~92.7%	apertium-tyv (incubator)	Fran
apertium-kjh	Khakas	Хакас тілі	-	kjh	HFST (lexc+twol)	prototype	293	~39.3%	apertium-kjh (incubator)	Fran
apertium-aze	Azerbaijani	Azərbaycan dili	az	aze	SFST	not known to work			apertium-tur-aze (staging)	Gianluca

Turkic languages by subgroup

• Oğuz: Turkish, Azeri, Turkmen, Gagauz, Trukhmen, Bojnur, etc.
• Qarluq: Uzbek, Uyghur
• Qıpçaq
  • Aralo-Caspian: Kazakh, Karakalpak, Nogay (, Kyrgyz)
  • North: Tatar, Bashqort
  • West: Kumyk, Karachay-Balkar, Karaim, Crimean Tatar
• Qıpçaq-Yenisei: South Altay (Yenisei), Kyrgyz (Qıpçaq)
• Yeniseian: Khakas, Shor, Chulym, Baraba Tatar, North Altay (, South Altay)
• Sayan: Tuvan, Tofa, Tozhu, Dukha, Soyot
• Lena: Sakha, Dolgan
• Arğu: Halaj
• Oğur: Chuvash

Pairs

Some Turkic languages that are particularly similar to one another (and hence have high levels of mutual intelligibility) include those in the following list:

• Turkish, Azerbaycani, (Turkmen)
• Qazaq, Qaraqalpaq, Noğay, (Qırğız)
• Kumyk, Karachay-Balkar
• Tatar, Bashqort
• Uzbek, Uyğur
• Shor, Khakas(, Altay)
• Sakha, Dolgan
• Tuvan, Tofa

Chuvash is very distant from other Turkic languages and is not even partially mutually intelligible with any of them.

Table of existing pairs

See also: Turkic-Turkic translator

Text in italics denotes language pairs in the incubator. Regular text denotes a developing language pair in nursery, while text in bold denotes a stable well-working language pair in trunk and text in bold and italics denotes a pair in staging. Bidix stems as counted with dixcounter are displayed below.

	tur	aze	tuk	uzb	kir	kaz	tat	bak	chv	nog	kaa	kum	uig
tur	-	tur-aze	tuk-tur	tur-uzb	tur-kir		tur-tat		cv-tr
aze	tur-aze	-
tuk	tuk-tur		-
uzb	tur-uzb			-	kir-uzb
kir	tur-kir			kir-uzb	-	kaz-kir	tat-kir
kaz					kaz-kir	-	'kaz-tat '			nog-kaz	kaz-kaa	kaz-kum	kaz-uig
tat	tur-tat				tat-kir	'kaz-tat '	-	tat-bak	cv-tt
bak							tat-bak	-
chv	cv-tr						cv-tt		-
nog						nog-kaz				-
kaa						kaz-kaa					-
kum						kaz-kum						-
uig						kaz-uig							-

Other pairs

	tur	aze	tuk	uzb	kir	kaz	tat	bak	chv	nog	kaa	kum	uig
eng	tr-en				ky-en	eng-kaz
epo	eo-tr
khk						khk-kaz
rus							tat-rus		cv-ru

Roadmap

• Stable release of apertium-kaz
• Stable release of apertium-tat
• Stable release of apertium-kaz-tat
• Rework apertium-kir to match new standards
• Bring apertium-bak up to date (based on apertium-tat)
• Expand apertium-tat-bak
• Beta release of apertium-kaz-kir
• Beta release of apertium-tur-kir
• Expand apertium-tuk
• Expand apertium-chv
• Basic transducers for:
  • Qaraqalpaq (based on Tatar Kazakh, probably, no?)
  • Uzbek
  • Uyghur
  • Nogay
  • Kumyk
  • Khakas
  • Tuvan
  • Sakha
  • Shor
  • Crimean Tatar

Getting involved

We have a work plan for developing Turkic-Turkic translators and are working on a how-to for building a Turkic lexicon. Please come talk to us on IRC or contact us on the apertium-turkic mailing list.

Samples

Article 1 of the Universal Declaration of Human Rights:

All human beings are born free and equal in dignity and rights. They are endowed with reason and conscience and should act towards one another in a spirit of brotherhood.

Language	Text
Kazakh	Барлық адамдар тумысынан азат және қадір‐қасиеті мен кұқықтары тең болып дүниеге келеді. Адамдарға ақыл‐парасат, ар‐ождан берілген, сондықтан олар бір‐бірімен туыстық, бауырмалдық қарым‐қатынас жасаулары тиіс.
Tatar	Барлык кешеләр дә азат һәм үз абруйлары һәм хокуклары ягыннан тиң булып туалар. Аларга акыл һәм вөҗдан бирелгән һәм бер-берсенә карата туганнарча мөнәсәбәттә булырга тиешләр.
Kirghiz	Бардык адамдар өз беделинде жана укуктарында эркин жана тең укуктуу болуп жаралат. Алардын аң‐сезими менен абийири бар жана бири‐бирине бир туугандык мамилекылууга тийиш.
Turkish	Bütün insanlar hür, haysiyet ve haklar bakımından eşit doğarlar. Akıl ve vicdana sahiptirler ve birbirlerine karşı kardeşlik zihniyeti ile hareket etmelidirler.
Uzbek (Latin)	Barcha odamlar erkin, qadr-qimmat ba huquqlarda tang bo'lib tug'iladilar. Ular aql va vijdon sohibidilar va bir-birla iga birodarlarcha muomala qilishlari zarur.
Tuva	Бүгү кижилер хостуг база мөзүзү болгаш эргелери дең кылдыр төрүттүнер. Оларга угаансарыыл болгаш арын-нүүр бердинген болур болгаш олар бот-боттарынга акы-дуңмалышкы хамаарылганы көргүзер ужурлуг.

Vulnerability

This table summarizes the vulnerability of various Turkic languages. Vulnerability data is derived from the ‘Atlas of the World’s Languages in Danger, © UNESCO, http://www.unesco.org/culture/languages-atlas’ and Ethnologue.

Language	ISO639-3	Location	Speakers	Status
				Ethnologue	UNESCO
Chagatai	chg	Turkmenistan	0	10 (Extinct)	-
Salchuq	slq	Iran	0	10 (Extinct)	-
Chulym	clw	Russian Federation	44	8b (Nearly extinct)	4 (Critically endangered)
Karaim	kdr	Lithuania & Ukraine	56	8b (Nearly extinct)	5 (Extinct)
Tofa (Karagas)	kim	Russian Federation	90	8b (Nearly extinct)	4 (Critically endangered)
Krimchak	jct	Ukraine	200	8b (Nearly extinct)	-
Ili Turki	ili	China	120	8a (Moribund)	3 (Severely endangered)
Yugur (Western/Saryg)	ybe	China	4,600	8a (Moribund)	3 (Severely endangered)
Balkan Gagauz Turkish	bgx	Turkey	331,000	7 (Shifting)	-
Dolgan	dlg	Russian Federation	1,050	6b (Threatened)	2 (Definitely endangered)
Shor	cjs	Russian Federation	2,840	6b (Threatened)	3 (Severely endangered)
Ainu	aib	China	6,570	6b (Threatened)	-
Altay, Northern	atv	Russian Federation	57,400	6b (Threatened)	3 (Severely endangered)
Salar	slr	China	70,000	6b (Threatened)	1 (Vulnerable)
Urum	uum	Georgia, Russian Federation, Ukraine	192,700	6b (Threatened)	2 (Definitely endangered)
Khalaj	klj	Iran	42,100	6a (Vigorous)	1 (Vulnerable)
Khorasani	kmz	Iran	400,000	6a (Vigorous)	1 (Vulnerable)
Kashkay	qxq	Iran	1,500,000	6a (Vigorous)	-
Khakas	kjh	Russian Federation	42,610	5 (Developing)	5 (Extinct)
Nogay	nog	Russian Federation	87,410	5 (Developing)	2 (Definitely endangered)
Gagauz	gag	Moldova	173,920	5 (Developing)	-
Karakalpak	kaa	Uzbekistan	410,410	5 (Developing)	-
Kumyk	kum	Russian Federation	426,550	5 (Developing)	1 (Vulnerable)
Tuva	tyv	China, Mongolia, Russian Federation	283,400	4 (Educational)	1 (Vulnerable)
Karachay-Balkar	krc	Russian Federation	310,730	4 (Educational)	1 (Vulnerable)
Crimean Tatar	crh	Ukraine	475,540	4 (Educational)	-
Chuvash	chv	Russian Federation	1,077,420	4 (Educational)	1 (Vulnerable)
Bashkort	bak	Russian Federation	1,221,340	4 (Educational)	1 (Vulnerable)
Uzbek, Southern	uzs	Afghanistan	2,961,980	3 (Wider communication)	-
Azerbaijani, Southern	azb	Iran	16,913,280	3 (Wider communication)	-
Altay, Southern	alt	Russian Federation	57,400	2 (Provincial)	2 (Definitely endangered)
Yakut	sah	Russian Federation	450,000	2 (Provincial)	1 (Vulnerable)
Tatar	tat	Russian Federation	5,407,550	2 (Provincial)	2 (Definitely endangered)
Uyghur	uig	China	8,791,610	2 (Provincial)	-
Kyrgyz	kir	Kyrgyzstan	2,941,930	1 (National)	-
Azerbaijani, Northern	azj	Azerbaijan	7,324,270	1 (National)	-
Turkmen	tuk	Turkmenistan	7,560,560	1 (National)	-
Kazakh	kaz	Kazakhstan	8,077,770	1 (National)	-
Uzbek, Northern	uzn	Uzbekistan	18,968,250	1 (National)	-
Turkish	tur	Turkey	50,733,420	1 (National)	-

Official poem

• (tur) Kovayla bira içerim, ama sen bilmezsin. Yarın gelir misin?
• (aze) Vedrəyle pivə içirəm, ama sen bilməzsən. Yarın gələrmisən?
• (tat) Чиләк белән сыра эчәм, әмма син белмисең. Иртәгә киләсеңме?
• (kaz) Шелекпен сыра ішемін, бірақ сен білмейсің. Ертең келесің бе?
• (kir) Чака менен сыра ичем, бирок сен билбейсиң. Эртең келесиңби?
• Челек булан пиво ичемен амма сен билмейсен. Эртен гелемисен?
• (uzb) Chelak bilan pivo ichaman lekin sen bilmaysan. Ertaga kelasanmi?
• (kaa) Shelek penen pivo ishemen, biraq sen bilmeysen'. Erten' kelesen'be?

Bibliography

• Washington, J. N., Salimzyanov, I., and Tyers, F. M. (2014) "Finite-state morphological transducers for three Kypchak languages". Proceedings of the 9th Conference on Language Resources and Evaluation, LREC2014
• Salimzyanov, I., Washington, J. N., and Tyers, F. M. (2013) "A free/open-source Kazakh-Tatar machine translation system". Proceedings of MT Summit XIV
• Washington, J. N., Ipasov, M. and Tyers, F. M. (2012) "A finite-state morphological analyser for Kyrgyz". Proceedings of the 8th Conference on Language Resources and Evaluation, LREC2012
• Tyers, F. M., Washington, J. N., Salimzyanov, I., and Batalov, R. (2012) "A prototype machine translation system for Tatar and Bashkir based on free/open-source components". Proceedings of the Turkic Languages Workshop at the Language Resources and Evaluation Conference, LREC2012