Indic languages

From Apertium
Revision as of 18:15, 22 November 2013 by Jsome2 (talk | contribs)
Jump to navigation Jump to search

The Indic languages include Hindi, Urdu, Bengali, Sanskrit, and several other languages. These languages are the dominant language family of the Indian subcontinent. The number of people that speak an Indic language is upwards of 900,000,000.

The master plan involves generating independent finite-state transducers for each language, and then making individual dictionaries and transfer rules for every pair. The current status of these goals is listed below.

Status

The ultimate goal is to have multi-purposable transducers for a variety of Indic languages. These can then be paired for X→Y translation with the addition of a CG for language X and transfer rules / dictionary for the pair X→Y. Below is listed development progress for each language's transducers and dictionary pairs.

Transducers

Once a transducer has ~80% coverage on a range of medium-large corpora we can say it is "working". Over 90% and it can be considered to be "production".

name Language ISO 639 formalism state stems coverage location primary authors
-2 -3
apertium-hin Hindi hi hin HFST (lexc+twol) production 37,833 - apertium-hin (languages) Nikant, Abu Zaher Md. Faridee, Fran
apertium-urd Urdu ur urd HFST (lexc+twol) production 14,943 - apertium-urd (languages) -
apertium-ben Bengali bn ben HFST (lexc+twol) production 8,230 - apertium-ben (languages) Abu Zaher Md. Faridee
apertium-san Sanskrit sa san HFST (lexc+twol) production 123,373 - apertium-san (languages) Amba Kulkarni


Indic Language Classification


Indic-Indic pairs

hin ben urd san
hin -
ben bn-hi -
urd ur-hi -
san -

Pairs with non-Indic languages

eng as mr pa fa
hin eng-hin as-hi mr-hi pa-hi
ben bn-en
urd ur-pa ur-fa
san

Tagset

Rough guide to tagsets in various Indic language transducers, with an eye to keeping stuff that is basically the same tagged the same. In the following table, A stands for Apertium and T stands for TRmorph (See also the general tagset list).

Phenomenon Morphology Description Tag(s) Language(s) Notes
Part of speech
Noun <n>