Language identification

From Apertium

Revision as of 08:55, 13 February 2015 by Unhammer (talk | contribs) (Created page with "'''Language identification''' or '''language recognition''' is the process of identifying what language a text (document/paragraph/sentence/word/…) is in. Apertium-apy uses...")

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Jump to navigation Jump to search

Language identification or language recognition is the process of identifying what language a text (document/paragraph/sentence/word/…) is in.

Apertium-apy uses the CLD2 library for language identification (optionally it can use coverage of analysers, but this is really slow)

See Apertium-apy/Language identification for some accuracy experiments of CLD2

See also

http://odur.let.rug.nl/~vannoord/TextCat/ the original TextCat library (perl, there's also a C port)
- https://github.com/unhammer/gt-CorpusTools/blob/master/corpustools/text_cat.py python2 reimplementation of textcat
http://www.ling.upenn.edu/Events/PLC/plc37/abstracts/Szymanski_PLC37_CT.pdf for identifying parallel text in another language within the same document.

Retrieved from "https://wiki.apertium.org/w/index.php?title=Language_identification&oldid=52815"