Maltese and Hebrew/Final report
Writing the Maltese morphological analyser was the hardest task in the project and required most of the time. That being said, I am very pleased with the results we got.
We then used Malteses frequency lists generated from the various corpora, and added them slowly using all kinds of translation tools, dictionaries and/or learning/guestimating by context and usage. This was a headache but got very good results; within about 2 weeks (and during my exams period) we got to a ~80% coverage of the Maltese corpora.
Documentation about Maltese morphology & grammar is very sparse and unsatisfying. This presented a huge challenge throughout the whole time. Luckily, my ninja mentors were able to figure out ways to learn what's needed. Additionally, we contacted people who previously researched and worked on Maltese and they all were very nice and glad to help out - we were able to use their works and knowledge in a few critical points in the project.
- The verbs
Analysing Maltese verbs has proved to be the biggest issue and I don't feel we got it right yet. We ended up having two scripts that generate verb forms from given stems lists: one I initially wrote using examples from Teach Yourself Maltese and the web, that has a lot of problems and errors in it, and a better one written by Fran who did a much more careful and thorough work. One of the most important things that remains to be done is merging this into one script that's written intelligently using the information laid out in the new grammar book we found.
In comparison, writing he.dix and handling Hebrew generation was fairly easy. Other than my own Hebrew knowledge, this was mostly due to the research I've done before GSoC started (for my application).
We have tweaked some code from the hspell Hebrew spellchecker project, to get most of the open-category terms. This way we easily got good enough coverage of nouns, verbs, adjectives, etc.
For closed-category terms, I added a lot of them at the beginning of the project, and then fixed what was needed as we went alone with the bidix.
The mt→he bidix work was very quick and automatic. A lot of the terms we previously added to mt.dix came with gloss so we were able to use it for the translations. For the rest,
- Maltese Wikipedia ( , std. dev.: )
- Maltese news sites ( , std. dev.: )
- Maltese Scannel corpus ( , std. dev.: )
- Error rate