Task ideas for Google Code-in/Add words

1. Select a language pair, ideally such that the source language is a language you know (L₂) and the target language a language you use every day (L₁).
2. Install Apertium locally from the Subversion repository; install the language pair; make sure that it works AND/OR get Apertium VirtualBox and update, check out & compile the language pair.
3. Using a large enough corpus of representative text in the source language (e.g. plain text taken from Wikipedia, newspapers, literature, etc.) detect the 50 most frequent unknown words (source words which are not in the dictionaries of the language pair).
4. add these words to the source dictionary (so that they are not unknown anymore), add the correspondence to the bilingual dictionary, and add the word to the target dictionary if not already there.
5. Compile and test again
6. Submit a patch to your mentor (or commit it if you have already gained developer access)

How to find the most frequent unknowns

<Unhammer> translate your corpus, make it one word per line, grab only the
           ones with * at the start, sort, count number of hits per word, sort
           again
<Unhammer> e.g.                                                         [11:25]
<Unhammer> zcat corpus.txt.gz | apertium -d . ron-fra | tr ' ' '\n' | grep
           '^\*' | sort |uniq -d |sort -n >hitlist
<asusAndrei> awesome!                                                   [11:26]
<Unhammer> hitlist will be unknowns sorted by frequency, but you might have to
           skip a couple that are "strange" or difficult to add
<Unhammer> and that's ok, as long as you start from the most frequent and work
           your way down