Indic languages

The Indic languages include Hindi, Urdu, Bengali, Sanskrit, and several other languages. These languages are the dominant language family of the Indian subcontinent. The number of people that speak an Indic language is upwards of 900,000,000.

The master plan involves generating independent finite-state transducers for each language, and then making individual dictionaries and transfer rules for every pair. The current status of these goals is listed below.

Status

The ultimate goal is to have multi-purposable transducers for a variety of Indic languages. These can then be paired for X→Y translation with the addition of a CG for language X and transfer rules / dictionary for the pair X→Y. Below is listed development progress for each language's transducers and dictionary pairs.

Transducers

Once a transducer has ~80% coverage on a range of medium-large corpora we can say it is "working". Over 90% and it can be considered to be "production".

name	Language	ISO 639		formalism	state	stems	coverage	location	primary authors
		-2	-3
apertium-san	Sanskrit	sa	san	lttoolbox	production	123,373	-	apertium-san (languages)	Amba Kulkarni
apertium-hin	Hindi	hi	hin	lttoolbox	production	37,833	-	apertium-hin (languages)	Nikant, Abu Zaher Md. Faridee, Fran
apertium-ben	Bengali	bn	ben	lttoolbox	development	8,230	-	apertium-ben (languages)	Abu Zaher Md. Faridee
apertium-urd	Urdu	ur	urd	lttoolbox	development	14,943	-	apertium-urd (languages)	Muhammad Humayoun
	Nepali	ne	nep			-	-	apertium-ne-en (incubator) apertium-eo-ne (incubator)
	Marathi	mr	mar			-	-	apertium-mr-hi (incubator) apertium-mar-eng (incubator)
	Punjabi	pa	pan			-	-	apertium-pa-hi (incubator) apertium-ur-pa (incubator)

Indic Language Classification

• Dardic: Pahayi, Khowar, Kohistani, Shina language, Kashiri
• Northern Zone:
  • Central Pahari: Garhwali, Kumauni
  • Eastern Pahari: Nepali
• North-Western Zone: Panjabi, Lahnda, Sindhi
  • Dogri-Kangri: Dogri, Kangri, Mandeali, etc.
• Western Zone: Gujarati, Bhil, Khandeshi, Domari-Romani
  • Rajasthani: Marwari, Rajasthani
• Hindi
• Sanskrit
• Southern Zone: Marathi, Konkani, Urdu
  • Insular Indic: Sinhalese, Maldivian
• Eastern Zone: Bengali, Oriya, Tharu
  • Bihari: Bhojpuri, Maithili, etc.

Existing language pairs

Indic-Indic pairs

	hin	ben	urd	san	nep	mar	pan
hin	-
ben	bn-hi	-
urd	ur-hi		-				ur-pa
san				-
nep					-
mar	mr-hi					-
pan	pa-hi						-

Pairs with non-Indic languages

	eng	asm	epo	pes
hin	eng-hin	as-hi
ben	bn-en
urd				ur-fa
san
nep	ne-en		eo-ne
mar	mar-eng
pan

Tagset

Rough guide to tagsets in various Indic language transducers, with an eye to keeping stuff that is basically the same tagged the same (see also the general tagset list).

Phenomenon	Morphology	Description	Tag(s)	Language(s)	Notes
Part of speech
Noun			<n>