Indic languages
The Indic languages include Hindi, Urdu, Bengali, Sanskrit, and a number of other languages. These languages are the dominant language family of the Indian subcontinent. The number of people that speak an Indic language is upwards of 900,000,000.
The master plan involves generating independent finite-state transducers for each language, and then making individual dictionaries and transfer rules for every pair. The current status of these goals is listed below.
Status
The ultimate goal is to have multi-purposable transducers for a variety of Indic languages. These can then be paired for X→Y translation with the addition of a CG for language X and transfer rules / dictionary for the pair X→Y. Below is listed development progress for each language's transducers and dictionary pairs.
Transducers
Once a transducer has ~80% coverage on a range of medium-large corpora we can say it is "working". Over 90% and it can be considered to be "production".
| name | Language | ISO 639 | formalism | state | stems | coverage | location | primary authors | |
|---|---|---|---|---|---|---|---|---|---|
| -2 | -3 | ||||||||
apertium-san
|
Sanskrit | sa
|
san
|
lttoolbox | working | 123,373 | - | apertium-san (languages) | Amba Kulkarni |
apertium-hin
|
Hindi | hi
|
hin
|
lttoolbox | working | 37,833 | ~83.1% | apertium-hin (languages) | Nikant, Abu Zaher Md. Faridee, Fran |
apertium-ben
|
Bengali | bn
|
ben
|
lttoolbox | development | 8,230 | ~74% | apertium-ben (languages) | Abu Zaher Md. Faridee |
apertium-urd
|
Urdu | ur
|
urd
|
lttoolbox | development | 14,943 | ~64.6% | apertium-urd (languages) | Muhammad Humayoun |
apertium-nep
|
Nepali | ne
|
nep
|
lttoolbox | prototype | 215 | - | apertium-ne-en (incubator) apertium-eo-ne (incubator) |
|
apertium-mar
|
Marathi | mr
|
mar
|
lttoolbox | prototype | 14,886 | - | apertium-mar-eng (incubator) apertium-mr-hi (incubator) |
|
apertium-pan
|
Punjabi | pa
|
pan
|
— | possibly non-existant | - | - | apertium-pa-hi (incubator) apertium-ur-pa (incubator) |
|
Indic Language Classification
- Dardic: Pahayi, Khowar, Kohistani, Shina language, Kashiri
- Northern Zone:
- North-Western Zone: Panjabi, Lahnda, Sindhi
- Western Zone: Gujarati, Bhil, Khandeshi, Domari-Romani
- Rajasthani: Marwari, Rajasthani
- Hindi
- Sanskrit
- Southern Zone: Marathi, Konkani, Urdu
- Eastern Zone: Bengali, Oriya, Tharu
Existing language pairs
Text in italics denotes language pairs in the incubator. Regular text denotes a developing language pair in nursery, while text in bold denotes a stable well-working language pair in trunk and text in bold and italics denotes a pair in staging. Bidix stems as counted with dixcounter are displayed below.
| hin | ben | urd | san | nep | mar | pan | |
|---|---|---|---|---|---|---|---|
| hin | - | bn-hi |
ur-hi |
mr-hi |
pa-hi | ||
| ben | bn-hi |
- | |||||
| urd | ur-hi |
- | ur-pa | ||||
| san | - | ||||||
| nep | - | ||||||
| mar | mr-hi |
- | |||||
| pan | pa-hi |
ur-pa |
- | ||||
| asm | as-hi |
||||||
| eng | eng-hin |
bn-en |
ne-en |
mar-eng |
|||
| epo | eo-ne |
||||||
| fas | ur-fa |
Samples
Article 1 of the Universal Declaration of Human Rights:
All human beings are born free and equal in dignity and rights. They are endowed with reason and conscience and should act towards one another in a spirit of brotherhood.
| Language | Text |
|---|---|
| Sanskrit | सर्वे मानवा: स्वतन्त्रा: समुत्पन्ना: वर्तन्ते अपि च, गौरवदृशा अधिकारदृशा च समाना: एव वर्तन्ते। एते सर्वे चेतना-तर्क-शक्तिभ्यां सुसम्पन्ना: सन्ति। अपि च, सर्वेऽपि बन्धुत्व-भावनया परस्परं व्यवहरन्तु। |
| Hindi | सभी मनुष्यों को गौरव और अधिकारों के मामले में जन्मजात स्वतन्त्रता और समानता प्राप्त है । उन्हें बुद्घि और अन्तरात्मा की देन प्राप्त है और परस्पर उन्हें भाईचारे के भाव से बर्ताव करना चाहिए । |
| Bengali | সমস্ত মানুষ স্বাধীনভাবে সমান মর্যাদা এবং অধিকার নিয়ে জন্মগ্রহণ করে। তাঁদের বিবেক এবং বুদ্ধি আছে ; সুতরাং সকলেরই একে অপরের প্রতি ভ্রাতৃত্বসুলভ মনোভাব নিয়ে আচরণ করা উচিত্। |
| Urdu | تمام انسان آزادی اور حقوق و عزت کے اعتبار سے برابر پیدا ہویٔے ہیں۔ انہیں ضمیر اور عقل و دیعت ہویٔی ہے۔ اس لیٔے انہیں ایک دوسرے کے ساتھ بھایٔی چارے کا سلوک کرنا چاہیء۔ |
| Nepali | सबै व्यक्ति हरू जन्मजात स्वतन्त्र हुन ती सबैको समान अधिकार र महत्व छ। निजहरूमा विचार शक्ति र सद्धिचार भएकोले निजहरूले आपसमा भातृत्वको भावना बाट व्यवहार गर्नु पर्छ। |
| Marathi | सर्व मानवी व्यक्ति जन्मतःच स्वतंत्र आहेत व त्यांना समान प्रतिष्ठा व समान अधिकार आहेत. त्यांना विचारशक्ति व सदसविद्वेकबुद्धि लाभलेली आहे. व त्यांनी एकमेकांशी बंधुत्याच्या भावनेने आचरण करावे. |
| Panjabi, Eastern | ਸਾਰਾ ਮਨੁੱਖੀ ਪਰਿਵਾਰ ਆਪਣੀ ਮਹਿਮਾ, ਸ਼ਾਨ ਅਤੇ ਹੱਕਾਂ ਦੇ ਪੱਖੋਂ ਜਨਮ ਤੋਂ ਹੀ ਆਜ਼ਾਦ ਹੈ ਅਤੇ ਸੁਤੇ ਸਿੱਧ ਸਾਰੇ ਲੋਕ ਬਰਾਬਰ ਹਨ । ਉਨ੍ਹਾਂ ਸਭਨਾ ਨੂੰ ਤਰਕ ਅਤੇ ਜ਼ਮੀਰ ਦੀ ਸੌਗਾਤ ਮਿਲੀ ਹੋਈ ਹੈ ਅਤੇ ਉਨ੍ਹਾਂ ਨੂੰ ਭਰਾਤਰੀਭਾਵ ਦੀ ਭਾਵਨਾ ਰਖਦਿਆਂ ਆਪਸ ਵਿਚ ਵਿਚਰਣਾ ਚਾਹੀਦਾ ਹੈ । |
Tagset
Rough guide to tagsets in various Indic language transducers, with an eye to keeping stuff that is basically the same tagged the same (see also the general tagset list).
| Phenomenon | Morphology | Description | Tag(s) | Language(s) | Notes |
|---|---|---|---|---|---|
| Part of speech | |||||
| Noun | <n> |
||||
