Hectoralos/GSOC 2019 proposal: Catalan-Italian and Catalan-Portuguese
Contents
Contact Information
Name: Hèctor Alòs i Font
Location: Shupashkar, Chuvashia, Russia
E-mail: hectoralos@gmail.com
IRC: hector2
GitHub: hectoralos
Telegram: hectoralos
Skype: hectoralos
Why is it you are interested in machine translation?
I’m a sociolinguist. I'm very interested in creating resources for minoritised languages.
Why is it that you are interested in Apertium?
- Because Apertium is free/open-source software.
- Because its community is strongly committed to under-resourced and minoritised/marginalised languages.
- Because there is lot of good work done and being done in it.
- Because it is not only machine translation, but also free resources than can be used for other purposes: e.g. dictionaries, morphological analysers, spellcheckers, etc.
Which of the published tasks are you interested in? What do you plan to do?
1.2 Bring a released language pair up to state-of-the-art quality: I'd like to improve the pairs Catalan-Italian and Catalan-Portuguese. In the first case, only the side from Italian to Catalan is available. I plan to create the other side.
My proposal
Title
Improving the Catalan-Italian and Catalan-Portuguese language pairs
Major goals
- Improving existing translators, obtaining a WER below 15% and a Wikipedia coverage above 91% (probably, above 92%):
- Italian to Catalan
- Portuguese to Catalan
- Catalan to Portuguese
- Developing a translator from Catalan to Italian with a WER below 15% and a Wikipedia coverage above 91% (probably, above 92%)
Unlike it happens with French, compared to other Romance languages, there are not big structural (syntax) differences between Catalan, Italian and Portuguese. If we improve the morphological disambiguation, add several thousands of words in the dictionaries, introduce lexical selection rules and create some more transfer rules, a low WER can be reached.
Reasons why Google and Apertium should sponsor it
The pairs Italian-Catalan, Portuguese-Catalan give unsatisfactory results (WER ≈ 30%). Both were published in 2009 and, apparently, no one has worked on them since then. In addition, the translation from Catalan to Italian is not developed.
Current situation of the language pairs
Next are a few data on the three existing directions, in its current state of development on GitHub. It must be taken into account that the versions on the web are still those of the apertium-ca-it and apertium-pt-ca pairs, while I have made the calculations from apertium-cat-ita and apertium-por-cat.
Italian to Catalan
- bidix: 9091 pairs (excluding proper names)
- Word error rate (WER): 29.6% (calculated using random Wikipedia texts with a total of 933 words)
Test file: 'it_wikipedia_ca_ini.txt'
Reference file 'it_wikipedia_ca_fin.txt'
Statistics about input files
Number of words in reference: 1009
Number of words in test: 998
Number of unknown words (marked with a star) in test: 176
Percentage of unknown words: 17.64 %
Results when removing unknown-word marks (stars)
Edit distance: 299
Word error rate (WER): 29.63 %
Number of position-independent correct words: 775
Position-independent word error rate (PER): 23.19 %
Portuguese to Catalan
- bidix: 7576 pairs (excluding proper names)
- Word error rate (WER): 26.9% (calculated using random Wikipedia texts with a total of 1648 words)
Test file: 'pt_wikipedia_ca_ini.txt'
Reference file 'pt_wikipedia_ca_fin.txt'
Statistics about input files
Number of words in reference: 1780
Number of words in test: 1780
Number of unknown words (marked with a star) in test: 294
Percentage of unknown words: 16.52 %
Results when removing unknown-word marks (stars)
Edit distance: 479
Word error rate (WER): 26.91 %
Number of position-independent correct words: 1427
Position-independent word error rate (PER): 19.83 %
Catalan to Portuguese
- bidix: 7576 pairs (excluding proper names)
- coverage: 87.6% (calculated using a Wikipedia corpus with 2.6 million words)
Workplan
| Week | Dates | Goals | Bidix (excluding proper names) |
WER | Coverage |
|---|---|---|---|---|---|
| Post-application period | 10 March - 26 May |
|
~9,000 (cat-ita) ~7,500 cat-por |
~30% (cat > ita) ~30% (cat > por) ~30% (por > cat) |
~89% (cat > ita) ~88% (cat > por) ~88% (por > cat) |
| 1 | 27 May - 2 June |
|
~11,000 (cat-ita) | ~90% (ita > cat) | |
| 2 | 3 June- 9 June |
|
~13,000 (cat-ita) | ~90.5% (ita > cat) | |
| 3 | 10 June - 16 June |
|
~14,000 (cat-ita) | <20% (ita > cat) | ~91% (ita > cat) |
| 4 | 17 June - 23 June |
|
~15,000 (cat-ita) | ~90.0% (cat > ita) ~91.5% (ita > cat) | |
| 5 | 24 June - 30 June |
First evaluation (28 June) |
~16,000 (cat-ita) | ~90.5% (cat > ita) ~91.8% (ita > cat) | |
| 6 | 1 July - 7 July |
|
~17,000 (cat-ita) | ~91.0% (cat > ita) ~92.0% (ita > cat) | |
| 7 | 8 June - 14 July |
|
~18,000 (cat-ita) | <15% (cat > ita) <15% (ita > cat) |
~91.5% (cat > ita) ~92.0% (ita > cat) |
| 8 | 15 July - 21 July |
|
~9,500 (cat-por) | ~88% (por > cat) | |
| 9 | 22 July - 28 July |
Second evaluation (26 July) |
~11,500 (cat-por) | ~89% (por > cat) | |
| 10 | 29 July - 4 August |
|
~13,000 (cat-por) | <20% (por > cat) | ~90% (por > cat) |
| 11 | 5 August - 11 August |
|
~14,500 (cat-por) | ~90.5% (por > cat) ~90.0% (por > cat) | |
| 12 | 12 August - 18 August |
|
~16,000 (cat-por) | ~90.8% (por > cat) ~90.5% (por > cat) | |
| 13 | 18 August - 25 August |
Final evaluation (26 August) |
~17,000 (cat-por) | <15% (cat > por) <15% (por > cat) |
~91.0% (por > cat) ~91.0% (por > cat) |
List your skills and give evidence of your qualifications
I once got a computer engineering (Universitat Politècnica de Catalunya, 1988), but I’ve forgotten almost everything on programming. I also got a BA on linguistics (Universitat de Barcelona, 2008). I’ve been working on Apertium for several years. In 2011 I created the Esperanto-French pair and prepared new releases of the Esperanto-Catalan and Esperanto-Spanish pairs. I’ve also been working on new releases of the French-Catalan pair (2017, 2018 and currently a third one is been packaged). I’ve mentored and was strongly involved in the GSoC projects on Italian-Sardinian (2016), Catalan-Sardinian (2017) and French-Occitan (2018). In all these cases we released new one-way language pairs just after the end of the GSoC.
Catalan is my mother tongue, and I’ve been studying it at the university. I'm a fluent speaker of Spanish and French. I read fluently in Italian and Portuguese, among other Romance languages, but my knowledge of them is mainly passive and linguistic. That’s why I’ll work mainly translating from Italian and Portuguese into Catalan, but my knowledge of Italian is good enough to create a first version of a translator from Catalan to Italian.
List any non-Summer-of-Code plans you have for the Summer
I can guarantee at least 30 hours per week of work during the whole Summer. As I love this kind of work, I'm sure I'll be engaged quite more.
I have to submit the last two short works for two subjects of the master I am studying on June 5 and 11. It is possible that I will travel on vacation with my family one or two weeks in July. In the unlikely event that one week I do not reach 30 hours of work, I would recover these hours (I have had this workload in other years and there have been no problems).
