Indic languages

The Indic languages include Hindi, Urdu, Bengali, Sanskrit, and a number of other languages. These languages are the dominant language family of the Indian subcontinent. The number of people that speak an Indic language is upwards of 900,000,000.

The master plan involves generating independent finite-state transducers for each language, and then making individual dictionaries and transfer rules for every pair. The current status of these goals is listed below.

Status

The ultimate goal is to have multi-purposable transducers for a variety of Indic languages. These can then be paired for X→Y translation with the addition of a CG for language X and transfer rules / dictionary for the pair X→Y. Below is listed development progress for each language's transducers and dictionary pairs.

Transducers

Once a transducer has ~80% coverage on a range of medium-large corpora we can say it is "working". Over 90% and it can be considered to be "production".

name	Language	ISO 639		formalism	state	stems	paradigms	coverage	location	primary authors
		-2	-3
apertium-san	Sanskrit	sa	san	lttoolbox	working	123,373	279	-	apertium-san (languages)	Amba Kulkarni
apertium-hin	Hindi	hi	hin	lttoolbox	working	37,833	101	~83.1%	apertium-hin (languages)	Nikant, Abu Zaher Md. Faridee, Fran
apertium-ben	Bengali	bn	ben	lttoolbox	development	8,230	137	~74%	apertium-ben (languages)	Abu Zaher Md. Faridee
apertium-urd	Urdu	ur	urd	lttoolbox	development	14,943	123	~64.6%	apertium-urd (languages)	Muhammad Humayoun
apertium-nep	Nepali	ne	nep	lttoolbox	prototype	215		-	apertium-ne-en (incubator) apertium-eo-ne (incubator)
apertium-mar	Marathi	mr	mar	lttoolbox	prototype	14,886	111	-	apertium-mar-eng (incubator) apertium-mr-hi (incubator)
apertium-sin	Sinhala	si	sin	lttoolbox	prototype	73	22	-	apertium-si-en (incubator)
apertium-pan	Punjabi	pa	pan	—	possibly non-existant	-	-	-	apertium-pa-hi (incubator) apertium-ur-pa (incubator)
apertium-sat	Santali	-	sat	lttoolbox	development	-	-	-	apertium-eng-sat	Prasanta Hembram

Indic Language Classification

• Dardic: Pahayi, Khowar, Kohistani, Shina language, Kashiri
• Austroasiatic: Santali
• Northern Zone:
  • Central Pahari: Garhwali, Kumauni
  • Eastern Pahari: Nepali
• North-Western Zone: Panjabi, Lahnda, Sindhi
  • Dogri-Kangri: Dogri, Kangri, Mandeali, etc.
• Western Zone: Gujarati, Bhil, Khandeshi, Domari-Romani
  • Rajasthani: Marwari, Rajasthani
• Hindi
• Sanskrit
• Southern Zone: Marathi, Konkani, Urdu
  • Insular Indic: Sinhala, Maldivian
• Eastern Zone: Bengali, Oriya, Tharu, Santali
  • Bihari: Bhojpuri, Maithili, etc.

Existing language pairs

Text in italics denotes language pairs in the incubator. Regular text denotes a developing language pair in nursery, while text in bold denotes a stable well-working language pair in trunk and text in bold and italics denotes a pair in staging. Bidix stems as counted with dixcounter are displayed below.

	hin	ben	urd	san	nep	mar	pan	sin	asm	eng	epo	fas	sat
hin	-	bn-hi 219	urd-hin 4,943			mar-hin 15,857	pa-hi 51,577
ben	bn-hi 219	-
urd	urd-hin 4,943		-				ur-pa 0
san				-
nep					-
mar	mar-hin 15,857					-
pan	pa-hi 51,577		ur-pa 0				-
sin								-
asm	as-hi 833	asm-ben 969
eng	eng-hin 39,242	bn-en 7,497			ne-en 163	mar-eng 102		si-en 80					eng-sat
epo					eo-ne 9,306
fas			ur-fa ?
sat										sat-eng

Samples

Article 1 of the Universal Declaration of Human Rights:

All human beings are born free and equal in dignity and rights. They are endowed with reason and conscience and should act towards one another in a spirit of brotherhood.

Language	Text
Sanskrit	सर्वे मानवा: स्वतन्त्रा: समुत्पन्ना: वर्तन्ते अपि च, गौरवदृशा अधिकारदृशा च समाना: एव वर्तन्ते। एते सर्वे चेतना-तर्क-शक्तिभ्यां सुसम्पन्ना: सन्ति। अपि च, सर्वेऽपि बन्धुत्व-भावनया परस्परं व्यवहरन्तु।
Hindi	सभी मनुष्यों को गौरव और अधिकारों के मामले में जन्मजात स्वतन्त्रता और समानता प्राप्त है । उन्हें बुद्घि और अन्तरात्मा की देन प्राप्त है और परस्पर उन्हें भाईचारे के भाव से बर्ताव करना चाहिए ।
Bengali	সমস্ত মানুষ স্বাধীনভাবে সমান মর্যাদা এবং অধিকার নিয়ে জন্মগ্রহণ করে। তাঁদের বিবেক এবং বুদ্ধি আছে ; সুতরাং সকলেরই একে অপরের প্রতি ভ্রাতৃত্বসুলভ মনোভাব নিয়ে আচরণ করা উচিত্‍।
Urdu	تمام انسان آزادی اور حقوق و عزت کے اعتبار سے برابر پیدا ہویٔے ہیں۔ انہیں ضمیر اور عقل و دیعت ہویٔی ہے۔ اس لیٔے انہیں ایک دوسرے کے ساتھ بھایٔی چارے کا سلوک کرنا چاہیء۔
Nepali	सबै व्यक्ति हरू जन्मजात स्वतन्त्र हुन ती सबैको समान अधिकार र महत्व छ। निजहरूमा विचार शक्ति र सद्धिचार भएकोले निजहरूले आपसमा भातृत्वको भावना बाट व्यवहार गर्नु पर्छ।
Marathi	सर्व मानवी व्यक्ति जन्मतःच स्वतंत्र आहेत व त्यांना समान प्रतिष्ठा व समान अधिकार आहेत. त्यांना विचारशक्ति व सदसविद्वेकबुद्धि लाभलेली आहे. व त्यांनी एकमेकांशी बंधुत्याच्या भावनेने आचरण करावे.
Panjabi, Eastern	ਸਾਰਾ ਮਨੁੱਖੀ ਪਰਿਵਾਰ ਆਪਣੀ ਮਹਿਮਾ, ਸ਼ਾਨ ਅਤੇ ਹੱਕਾਂ ਦੇ ਪੱਖੋਂ ਜਨਮ ਤੋਂ ਹੀ ਆਜ਼ਾਦ ਹੈ ਅਤੇ ਸੁਤੇ ਸਿੱਧ ਸਾਰੇ ਲੋਕ ਬਰਾਬਰ ਹਨ । ਉਨ੍ਹਾਂ ਸਭਨਾ ਨੂੰ ਤਰਕ ਅਤੇ ਜ਼ਮੀਰ ਦੀ ਸੌਗਾਤ ਮਿਲੀ ਹੋਈ ਹੈ ਅਤੇ ਉਨ੍ਹਾਂ ਨੂੰ ਭਰਾਤਰੀਭਾਵ ਦੀ ਭਾਵਨਾ ਰਖਦਿਆਂ ਆਪਸ ਵਿਚ ਵਿਚਰਣਾ ਚਾਹੀਦਾ ਹੈ ।

Tagset

Rough guide to tagsets in various Indic language transducers, with an eye to keeping stuff that is basically the same tagged the same (see also the general tagset list).

Phenomenon	Morphology	Description	Tag(s)	Language(s)	Notes
Part of speech
Noun			<n>