Difference between revisions of "Indic languages"
(→Samples: added Santali sentence and corrected Panjabi to Punjabi) |
|||
(15 intermediate revisions by 4 users not shown) | |||
Line 18: | Line 18: | ||
!rowspan=2| state |
!rowspan=2| state |
||
!rowspan=2| stems |
!rowspan=2| stems |
||
!rowspan=2| paradigms |
|||
!rowspan=2| coverage |
!rowspan=2| coverage |
||
!rowspan=2| location |
!rowspan=2| location |
||
Line 32: | Line 33: | ||
|| working |
|| working |
||
|align="right"| {{#lst:Apertium-san/stats|stems}} |
|align="right"| {{#lst:Apertium-san/stats|stems}} |
||
|align="right"| {{#lst:Apertium-san/stats|paradigms}} |
|||
|align="center"| - |
|align="center"| - |
||
|| [[apertium-san]] ([[languages]]) |
|| [[apertium-san]] ([[languages]]) |
||
Line 44: | Line 46: | ||
|| working |
|| working |
||
|align="right"| {{#lst:apertium-hin/stats|stems}} |
|align="right"| {{#lst:apertium-hin/stats|stems}} |
||
|align="right"| {{#lst:apertium-hin/stats|paradigms}} |
|||
|align="center"| ~{{#lst:apertium-hin/stats/average}}% |
|align="center"| ~{{#lst:apertium-hin/stats/average}}% |
||
|| [[apertium-hin]] ([[languages]]) |
|| [[apertium-hin]] ([[languages]]) |
||
Line 55: | Line 58: | ||
|| development |
|| development |
||
|align="right"| {{#lst:apertium-ben/stats|stems}} |
|align="right"| {{#lst:apertium-ben/stats|stems}} |
||
|align="right"| {{#lst:apertium-ben/stats|paradigms}} |
|||
|align="center"| ~{{#lst:apertium-ben/stats/average}}% |
|align="center"| ~{{#lst:apertium-ben/stats/average}}% |
||
|| [[apertium-ben]] ([[languages]]) |
|| [[apertium-ben]] ([[languages]]) |
||
Line 66: | Line 70: | ||
|| development |
|| development |
||
|align="right"| {{#lst:apertium-urd/stats|stems}} |
|align="right"| {{#lst:apertium-urd/stats|stems}} |
||
|align="right"| {{#lst:apertium-urd/stats|paradigms}} |
|||
|align="center"| ~{{#lst:apertium-urd/stats/average}}% |
|align="center"| ~{{#lst:apertium-urd/stats/average}}% |
||
|| [[apertium-urd]] ([[languages]]) |
|| [[apertium-urd]] ([[languages]]) |
||
Line 77: | Line 82: | ||
|| prototype |
|| prototype |
||
|align="right"| {{#lst:apertium-nep/stats|stems}} |
|align="right"| {{#lst:apertium-nep/stats|stems}} |
||
|align="right"| {{#lst:apertium-nep/stats|paradigms}} |
|||
|align="center"| - |
|align="center"| - |
||
|| [[apertium-ne-en]] ([[incubator]])<br />[[apertium-eo-ne]] ([[incubator]]) |
|| [[apertium-ne-en]] ([[incubator]])<br />[[apertium-eo-ne]] ([[incubator]]) |
||
Line 88: | Line 94: | ||
|| prototype |
|| prototype |
||
|align="right"| {{#lst:apertium-mar/stats|stems}} |
|align="right"| {{#lst:apertium-mar/stats|stems}} |
||
|align="right"| {{#lst:apertium-mar/stats|paradigms}} |
|||
|align="center"| - |
|align="center"| - |
||
|| [[apertium-mar-eng]] ([[incubator]])<br />[[apertium-mr-hi]] ([[incubator]]) |
|| [[apertium-mar-eng]] ([[incubator]])<br />[[apertium-mr-hi]] ([[incubator]]) |
||
|| |
|| |
||
⚫ | |||
|| <code>[[apertium-sin]]</code> |
|||
|| [[Sinhala]] |
|||
|| <code>si</code> |
|||
|| <code>sin</code> |
|||
|| [[lttoolbox]] |
|||
|| {{#lst:apertium-sin/stats|state}} |
|||
|align="right"| {{#lst:apertium-sin/stats|stems}} |
|||
|align="right"| {{#lst:apertium-sin/stats|paradigms}} |
|||
|align="center"| - |
|||
|| {{#lst:apertium-sin/stats|location}} |
|||
|| {{#lst:apertium-sin/stats|authors}} |
|||
|- |
|- |
||
|- |
|- |
||
Line 99: | Line 118: | ||
|| — |
|| — |
||
|| possibly non-existant |
|| possibly non-existant |
||
|align="right"| - |
|||
|align="right"| - |
|align="right"| - |
||
|align="center"| - |
|align="center"| - |
||
|| [[apertium-pa-hi]] ([[incubator]])<br />[[apertium-ur-pa]] ([[incubator]]) |
|| [[apertium-pa-hi]] ([[incubator]])<br />[[apertium-ur-pa]] ([[incubator]]) |
||
| |
|- |
||
|- |
|||
|| <code>[[apertium-sat]]</code> |
|||
|| [[Santali]] |
|||
|| <code>-</code> |
|||
|| <code>sat</code> |
|||
|| [[lttoolbox]] |
|||
|| development |
|||
|align="right"| - |
|||
|align="right"| - |
|||
|align="center"| - |
|||
|| [[apertium-eng-sat]] |
|||
|| [[User:Rocky 734|Prasanta Hembram]] |
|||
|} |
|} |
||
=== Indic Language Classification === |
=== Indic Language Classification === |
||
* Dardic: [[Pahayi]], [[Khowar]], [[Kohistani]], [[Shina language]], [[Kashiri]] |
* Dardic: [[Pahayi]], [[Khowar]], [[Kohistani]], [[Shina language]], [[Kashiri]] |
||
* Austroasiatic: [[Santali]] |
|||
* Northern Zone: |
* Northern Zone: |
||
**Central Pahari: [[Garhwali]], [[Kumauni]] |
**Central Pahari: [[Garhwali]], [[Kumauni]] |
||
Line 117: | Line 150: | ||
* [[Sanskrit]] |
* [[Sanskrit]] |
||
* Southern Zone: [[Marathi]], [[Konkani]], [[Urdu]] |
* Southern Zone: [[Marathi]], [[Konkani]], [[Urdu]] |
||
** Insular Indic: [[ |
** Insular Indic: [[Sinhala]], [[Maldivian]] |
||
* Eastern Zone: [[Bengali]], [[Oriya]], [[Tharu]] |
* Eastern Zone: [[Bengali]], [[Oriya]], [[Tharu]], [[Santali]] |
||
** Bihari: [[Bhojpuri]], [[Maithili]], etc. |
** Bihari: [[Bhojpuri]], [[Maithili]], etc. |
||
=== Existing language pairs === |
=== Existing language pairs === |
||
Text in ''italics'' denotes language pairs in the incubator. Regular text denotes a developing language pair in nursery, while text in '''bold''' denotes a stable well-working language pair in trunk and text in '''''bold and italics''''' denotes a pair in staging. Bidix stems as counted with [[dixcounter]] are displayed below. |
Text in ''italics'' denotes language pairs in the incubator. Regular text denotes a developing language pair in nursery, while text in '''bold''' denotes a stable well-working language pair in trunk and text in '''''bold and italics''''' denotes a pair in staging. Bidix stems as counted with [[dixcounter]] are displayed below. |
||
{| style="text-align: center;" class="wikitable" |
{| style="text-align: center;" class="wikitable dixtable" |
||
|- style="background: #ececec" |
|- style="background: #ececec" |
||
! !! hin !! ben !! urd !! san !! nep !! mar !! pan |
! !! hin !! ben !! urd !! san !! nep !! mar !! pan !! sin !! asm !! eng !! epo !! fas !! sat |
||
|- |
|- |
||
| '''hin''' || - || ''[[Apertium-bn-hi|bn-hi]]''<br>{{#lst:Apertium-bn-hi/stats|bn- |
| '''hin''' || - || ''[[Apertium-bn-hi|bn-hi]]''<br>{{#lst:Apertium-bn-hi/stats|bn-hi_stems}} || '''[[Apertium-urd-hin|urd-hin]]'''<br>'''{{#lst:Apertium-urd-hin/stats|urd-hin_stems}}''' || || || ''[[Apertium-mar-hin|mar-hin]]''<br>{{#lst:Apertium-mar-hin/stats|mar-hin_stems}} || ''[[Apertium-pa-hi|pa-hi]]''<br>{{#lst:Apertium-pa-hi/stats|pa-hi_stems}} || || || || || || |
||
|- |
|- |
||
| '''ben''' || ''[[Apertium-bn-hi|bn-hi]]''<br>{{#lst:Apertium-bn-hi/stats|bn- |
| '''ben''' || ''[[Apertium-bn-hi|bn-hi]]''<br>{{#lst:Apertium-bn-hi/stats|bn-hi_stems}} || - || || || || || || || || || || || |
||
|- |
|- |
||
| '''urd''' || [[Apertium- |
| '''urd''' || '''[[Apertium-urd-hin|urd-hin]]'''<br>'''{{#lst:Apertium-urd-hin/stats|urd-hin_stems}}''' || || - || || || || ''[[Apertium-ur-pa|ur-pa]]''<br>{{#lst:Apertium-ur-pa/stats|ur-pa_stems}} || || || || || || |
||
|- |
|- |
||
| '''san''' || || || || - || || || |
| '''san''' || || || || - || || || || || || || || || |
||
|- |
|- |
||
| '''nep''' || || || || || - || || |
| '''nep''' || || || || || - || || || || || || || || |
||
|- |
|- |
||
| '''mar''' || ''[[Apertium- |
| '''mar''' || ''[[Apertium-mar-hin|mar-hin]]''<br>{{#lst:Apertium-mar-hin/stats|mar-hin_stems}} || || || || || - || || || || || || || |
||
|- |
|- |
||
| '''pan''' || ''[[Apertium-pa-hi|pa-hi]]''<br>{{#lst:Apertium-pa-hi/stats|pa- |
| '''pan''' || ''[[Apertium-pa-hi|pa-hi]]''<br>{{#lst:Apertium-pa-hi/stats|pa-hi_stems}} || || ''[[Apertium-ur-pa|ur-pa]]''<br>{{#lst:Apertium-ur-pa/stats|ur-pa_stems}} || || || || - || || || || || || |
||
|- |
|- |
||
| || || || || || || || |
| '''sin''' || || || || || || || || - || || || || || |
||
|- |
|- |
||
| '''asm''' || ''[[Apertium-as-hi|as-hi]]''<br>{{#lst:Apertium-as-hi/stats|as- |
| '''asm''' || ''[[Apertium-as-hi|as-hi]]''<br>{{#lst:Apertium-as-hi/stats|as-hi_stems}} || ''[[Apertium-asm-ben|asm-ben]]''<br>{{#lst:Apertium-asm-ben/stats|asm-ben_stems}} || || || || || || || || || || || |
||
|- |
|- |
||
| '''eng''' || [[Apertium-eng-hin|eng-hin]]<br>{{#lst:Apertium-eng-hin/stats|eng- |
| '''eng''' || [[Apertium-eng-hin|eng-hin]]<br>{{#lst:Apertium-eng-hin/stats|eng-hin_stems}} || ''[[Apertium-bn-en|bn-en]]''<br>{{#lst:Apertium-bn-en/stats|bn-en_stems}} || || || ''[[Apertium-ne-en|ne-en]]''<br>{{#lst:Apertium-ne-en/stats|ne-en_stems}} || ''[[Apertium-mar-eng|mar-eng]]''<br>{{#lst:Apertium-mar-eng/stats|mar-eng_stems}} || || ''[[Apertium-si-en|si-en]]''<br>{{#lst:Apertium-si-en/stats|si-en_stems}} || || || || || ''[[Apertium-eng-sat|eng-sat]]'' |
||
|- |
|- |
||
| '''epo''' || || || || || ''[[Apertium-eo-ne|eo-ne]]''<br>{{#lst:Apertium-eo-ne/stats|eo- |
| '''epo''' || || || || || ''[[Apertium-eo-ne|eo-ne]]''<br>{{#lst:Apertium-eo-ne/stats|eo-ne_stems}} || || || || || || || || |
||
|- |
|- |
||
| '''fas''' || || || ''[[Apertium-ur-fa|ur-fa]]''<br>{{#lst:Apertium-ur-fa/stats|ur- |
| '''fas''' || || || ''[[Apertium-ur-fa|ur-fa]]''<br>{{#lst:Apertium-ur-fa/stats|ur-fa_stems}} || || || || || || || || || || |
||
|- |
|||
| '''sat''' || || || || || || || || || || ''[[Apertium-eng-sat|sat-eng]]'' || || || |
|||
|} |
|} |
||
Line 162: | Line 196: | ||
! Language !! Text |
! Language !! Text |
||
|- |
|- |
||
⚫ | |||
⚫ | |||
|- |
|- |
||
|| Hindi || सभी मनुष्यों को गौरव और अधिकारों के मामले में जन्मजात स्वतन्त्रता और समानता प्राप्त है । उन्हें बुद्घि और अन्तरात्मा की देन प्राप्त है और परस्पर उन्हें भाईचारे के भाव से बर्ताव करना चाहिए । |
|| Hindi || सभी मनुष्यों को गौरव और अधिकारों के मामले में जन्मजात स्वतन्त्रता और समानता प्राप्त है । उन्हें बुद्घि और अन्तरात्मा की देन प्राप्त है और परस्पर उन्हें भाईचारे के भाव से बर्ताव करना चाहिए । |
||
|- |
|- |
||
⚫ | |||
⚫ | |||
⚫ | |||
⚫ | |||
|- |
|- |
||
|| Nepali || सबै व्यक्ति हरू जन्मजात स्वतन्त्र हुन ती सबैको समान अधिकार र महत्व छ। निजहरूमा विचार शक्ति र सद्धिचार भएकोले निजहरूले आपसमा भातृत्वको भावना बाट व्यवहार गर्नु पर्छ। |
|| Nepali || सबै व्यक्ति हरू जन्मजात स्वतन्त्र हुन ती सबैको समान अधिकार र महत्व छ। निजहरूमा विचार शक्ति र सद्धिचार भएकोले निजहरूले आपसमा भातृत्वको भावना बाट व्यवहार गर्नु पर्छ। |
||
|- |
|- |
||
⚫ | |||
⚫ | |||
|- |
|- |
||
⚫ | |||
⚫ | |||
|- |
|||
|| Santali (Ol Chiki) || ᱡᱷᱚᱛᱚ ᱦᱚᱲ ᱠᱚ ᱜᱚᱨᱚᱡᱽ ᱟᱨ ᱚᱫᱷᱤᱠᱟᱹᱨ ᱢᱟᱹᱢᱞᱟᱹ ᱨᱮ ᱡᱟᱱᱟᱢ ᱡᱟᱛ ᱥᱟᱢᱮᱞᱟᱭ ᱧᱟᱢ ᱠᱟᱱᱟ ᱾ ᱩᱱᱤᱠᱩ ᱠᱚ ᱛᱚᱨᱠ ᱟᱨ ᱣᱤᱣᱮᱠ ᱛᱮ ᱯᱩᱨᱟᱹᱣ ᱠᱟᱱᱟ ᱠᱚ ᱟᱨ ᱩᱱᱠᱩ ᱫᱚ ᱵᱚᱭᱦᱟ ᱦᱤᱥᱟᱹᱵ ᱛᱮ ᱱᱤᱡᱚᱨ ᱦᱤᱥᱟᱹᱵ ᱛᱮ ᱠᱟᱹᱢᱤ ᱠᱟᱛᱷᱟ ᱾ |
|||
|- |
|||
⚫ | |||
|} |
|} |
||
== Also see == |
|||
*[[Languages of India]] |
|||
== Helpful Pages == |
|||
*[[List_of_symbols|The General Tagset List]] |
|||
<!-- |
|||
==Tagset== |
==Tagset== |
||
Rough guide to tagsets in various Indic language transducers, with an eye to keeping stuff that is basically the same tagged the same (see also [[List_of_symbols| |
Rough guide to tagsets in various Indic language transducers, with an eye to keeping stuff that is basically the same tagged the same (see also [[List_of_symbols|The General Tagset List]]). |
||
{|class="wikitable" |
{|class="wikitable" |
||
Line 188: | Line 230: | ||
| Noun || || || {{tag|n}} || || |
| Noun || || || {{tag|n}} || || |
||
|} |
|} |
||
--> |
|||
[[Category:Indic languages]] |
Latest revision as of 09:06, 18 January 2022
The Indic languages include Hindi, Urdu, Bengali, Sanskrit, and a number of other languages. These languages are the dominant language family of the Indian subcontinent. The number of people that speak an Indic language is upwards of 900,000,000.
The master plan involves generating independent finite-state transducers for each language, and then making individual dictionaries and transfer rules for every pair. The current status of these goals is listed below.
Status[edit]
The ultimate goal is to have multi-purposable transducers for a variety of Indic languages. These can then be paired for X→Y translation with the addition of a CG for language X and transfer rules / dictionary for the pair X→Y. Below is listed development progress for each language's transducers and dictionary pairs.
Transducers[edit]
Once a transducer has ~80% coverage on a range of medium-large corpora we can say it is "working". Over 90% and it can be considered to be "production".
Indic Language Classification[edit]
- Dardic: Pahayi, Khowar, Kohistani, Shina language, Kashiri
- Austroasiatic: Santali
- Northern Zone:
- North-Western Zone: Panjabi, Lahnda, Sindhi
- Western Zone: Gujarati, Bhil, Khandeshi, Domari-Romani
- Rajasthani: Marwari, Rajasthani
- Hindi
- Sanskrit
- Southern Zone: Marathi, Konkani, Urdu
- Eastern Zone: Bengali, Oriya, Tharu, Santali
Existing language pairs[edit]
Text in italics denotes language pairs in the incubator. Regular text denotes a developing language pair in nursery, while text in bold denotes a stable well-working language pair in trunk and text in bold and italics denotes a pair in staging. Bidix stems as counted with dixcounter are displayed below.
hin | ben | urd | san | nep | mar | pan | sin | asm | eng | epo | fas | sat | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
hin | - | bn-hi 219 |
urd-hin 4,943 |
mar-hin 15,857 |
pa-hi 51,577 |
||||||||
ben | bn-hi 219 |
- | |||||||||||
urd | urd-hin 4,943 |
- | ur-pa 0 |
||||||||||
san | - | ||||||||||||
nep | - | ||||||||||||
mar | mar-hin 15,857 |
- | |||||||||||
pan | pa-hi 51,577 |
ur-pa 0 |
- | ||||||||||
sin | - | ||||||||||||
asm | as-hi 833 |
asm-ben 969 |
|||||||||||
eng | eng-hin 39,242 |
bn-en 7,497 |
ne-en 163 |
mar-eng 102 |
si-en 80 |
eng-sat | |||||||
epo | eo-ne 9,306 |
||||||||||||
fas | ur-fa ? |
||||||||||||
sat | sat-eng |
Samples[edit]
Article 1 of the Universal Declaration of Human Rights:
All human beings are born free and equal in dignity and rights. They are endowed with reason and conscience and should act towards one another in a spirit of brotherhood.
Language | Text |
---|---|
Bengali | সমস্ত মানুষ স্বাধীনভাবে সমান মর্যাদা এবং অধিকার নিয়ে জন্মগ্রহণ করে। তাঁদের বিবেক এবং বুদ্ধি আছে ; সুতরাং সকলেরই একে অপরের প্রতি ভ্রাতৃত্বসুলভ মনোভাব নিয়ে আচরণ করা উচিত্। |
Hindi | सभी मनुष्यों को गौरव और अधिकारों के मामले में जन्मजात स्वतन्त्रता और समानता प्राप्त है । उन्हें बुद्घि और अन्तरात्मा की देन प्राप्त है और परस्पर उन्हें भाईचारे के भाव से बर्ताव करना चाहिए । |
Marathi | सर्व मानवी व्यक्ति जन्मतःच स्वतंत्र आहेत व त्यांना समान प्रतिष्ठा व समान अधिकार आहेत. त्यांना विचारशक्ति व सदसविद्वेकबुद्धि लाभलेली आहे. व त्यांनी एकमेकांशी बंधुत्याच्या भावनेने आचरण करावे. |
Nepali | सबै व्यक्ति हरू जन्मजात स्वतन्त्र हुन ती सबैको समान अधिकार र महत्व छ। निजहरूमा विचार शक्ति र सद्धिचार भएकोले निजहरूले आपसमा भातृत्वको भावना बाट व्यवहार गर्नु पर्छ। |
Punjabi, Eastern | ਸਾਰਾ ਮਨੁੱਖੀ ਪਰਿਵਾਰ ਆਪਣੀ ਮਹਿਮਾ, ਸ਼ਾਨ ਅਤੇ ਹੱਕਾਂ ਦੇ ਪੱਖੋਂ ਜਨਮ ਤੋਂ ਹੀ ਆਜ਼ਾਦ ਹੈ ਅਤੇ ਸੁਤੇ ਸਿੱਧ ਸਾਰੇ ਲੋਕ ਬਰਾਬਰ ਹਨ । ਉਨ੍ਹਾਂ ਸਭਨਾ ਨੂੰ ਤਰਕ ਅਤੇ ਜ਼ਮੀਰ ਦੀ ਸੌਗਾਤ ਮਿਲੀ ਹੋਈ ਹੈ ਅਤੇ ਉਨ੍ਹਾਂ ਨੂੰ ਭਰਾਤਰੀਭਾਵ ਦੀ ਭਾਵਨਾ ਰਖਦਿਆਂ ਆਪਸ ਵਿਚ ਵਿਚਰਣਾ ਚਾਹੀਦਾ ਹੈ । |
Sanskrit | सर्वे मानवा: स्वतन्त्रा: समुत्पन्ना: वर्तन्ते अपि च, गौरवदृशा अधिकारदृशा च समाना: एव वर्तन्ते। एते सर्वे चेतना-तर्क-शक्तिभ्यां सुसम्पन्ना: सन्ति। अपि च, सर्वेऽपि बन्धुत्व-भावनया परस्परं व्यवहरन्तु। |
Santali (Ol Chiki) | ᱡᱷᱚᱛᱚ ᱦᱚᱲ ᱠᱚ ᱜᱚᱨᱚᱡᱽ ᱟᱨ ᱚᱫᱷᱤᱠᱟᱹᱨ ᱢᱟᱹᱢᱞᱟᱹ ᱨᱮ ᱡᱟᱱᱟᱢ ᱡᱟᱛ ᱥᱟᱢᱮᱞᱟᱭ ᱧᱟᱢ ᱠᱟᱱᱟ ᱾ ᱩᱱᱤᱠᱩ ᱠᱚ ᱛᱚᱨᱠ ᱟᱨ ᱣᱤᱣᱮᱠ ᱛᱮ ᱯᱩᱨᱟᱹᱣ ᱠᱟᱱᱟ ᱠᱚ ᱟᱨ ᱩᱱᱠᱩ ᱫᱚ ᱵᱚᱭᱦᱟ ᱦᱤᱥᱟᱹᱵ ᱛᱮ ᱱᱤᱡᱚᱨ ᱦᱤᱥᱟᱹᱵ ᱛᱮ ᱠᱟᱹᱢᱤ ᱠᱟᱛᱷᱟ ᱾ |
Urdu | تمام انسان آزادی اور حقوق و عزت کے اعتبار سے برابر پیدا ہویٔے ہیں۔ انہیں ضمیر اور عقل و دیعت ہویٔی ہے۔ اس لیٔے انہیں ایک دوسرے کے ساتھ بھایٔی چارے کا سلوک کرنا چاہیء۔ |