Ideas for Google Summer of Code/Automatic diacritic restoration

From Apertium
< Ideas for Google Summer of Code
Revision as of 19:50, 24 March 2020 by Popcorndude (talk | contribs) (categorize)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Kevin Scannell has a Perl implementation of various statistical restoration algorithms called charlifter, which has been trained for more than 100 languages using web crawled data. Details are in his paper linked below. You can try the system here (or install the Firefox extension here).

A port of the algorithm to C++ should be easy. The more subtle issue is to optimize smoothing of the statistical models on a language-by-language basis.