User:Marcriera/proposal
Contents
Contact Information
Name: Marc Riera Irigoyen
Location: Barcelona, Spain
E-mail: marc.riera.irigoyen@gmail.com
IRC: mriera_trad
SourceForge: marcriera
Timezone: UTC+02:00
Why is it you are interested in machine translation?
Before being interested in machine translation, I was very interested in translation and I decided to study a degree in Translation and Interpreting at university. After learning about computer-assisted translation, I became more and more interested in machine translation not as a replacement of human translation, but rather as a way to improve human translation effectiveness and productivity.
Why is it that you are interested in Apertium?
The Apertium project is very interesting thanks to its open-sourced nature. It is an opportunity to build not only a robust and fairly good rule-based machine translator, but also to help with the development of translation pairs that would be extremely difficult to be implemented in statistical machine translators, such as minority language pairs. As a native speaker of one of these languages (Catalan), the fact that Apertium can offer better results than other types of translators is very attractive.
Which of the published tasks are you interested in? What do you plan to do?
Currently, there is an English-Catalan language pair in trunk. However, this pair uses its own monolingual dictionaries, which makes future development more difficult. The aim of this project is to migrate the changes from this old pair to the new one under development in order to get rid of the en-ca pair, and then proceed to expand the dictionaries and transfer rules of the new eng-cat pair. For this purpose, featured Wikipedia articles and public domain books will be used. However, as the existing transfer rules lack any kind of organization, code refactoring will be absolutely necessary before proceeding to expand it.
Apertium now provides an English-Catalan language pair that has been developed enough to allow for assimilation (to a certain extent), but it is still very far from allowing dissemination. Furthermore, translation from Catalan to English still tends to fail and prevents proper assimilation for potential users. Therefore, even if the main purpose of this proposal is to improve the language pair in the EN>CA direction, there will be some rules added to make it work better in the opposite direction too.
Tagger training should not be necessary (at least in English), but depending on the results when translating from Catalan to English, it may be required. For this reason, there is time assigned to this task in the work plan. The same applies to constraint grammar; most of the new rules will be transfer or lexical selection rules, but some CG rules might be needed.
Coverage will be calculated based on Wikipedia. Due to the enormous size of the English version, a word frequency list will be generated from a big part of Wikipedia (at least 100 million words), and coverage will be tested on it. This will allow more efficient token testing.
Title
Adopting English-Catalan language pair to bring it close to state-of-the-art quality
Reasons why Google and Apertium should sponsor it
While there is already an English-Catalan language pair, the quality of the translations can be improved a lot. While other machine translators already offer English-Catalan translation with good results, Apertium has the advantage of being a free and open-source project. This idea matches that of other internet projects such as Wikipedia, which could make use of the Apertium project by default to improve the quality of the translations and reduce post-edition. This is specially the case of specialized texts, which are superior in Apertium than in statistical machine translators in pairs such as en-es ([1]).
How and who it will benefit in society
Catalan is a language with only 10 million speakers, but a very lively one. An improved rule-based machine translator for English and Catalan will allow Catalan speakers to benefit from better English-to-Catalan translations than the current results with statistical translators. The open-source nature of Apertium will hopefully encourage online content creators to start using translations in Catalan or to improve the current ones. English speakers will also benefit from the expanded bidix, and future Apertium developers will be able to further develop the language pair more easily thanks to the unification.
List your skills and give evidence of your qualifications
My mother languages are Catalan and Spanish, and I can also speak English and Japanese. I am currently studying a degree in Translation and Interpreting. I have collaborated with some open-source software projects as an English to Catalan translator, and during the previous term of the current academic year I translated a book for a book publisher. In addition, since 2015 I am an active programmer and translator in the OpenBVE project, an open source railway simulator. I am an experienced Debian and Fedora user, and I know C# and XML.
List any non-Summer-of-Code plans you have for the Summer
The first week of June I have final exams, so I will only be able to work around 20 hours that week. After that date, I will be able to spend at least 30 hours a week on Apertium. I will do some extra hours during the first month to compensate for the hours "lost" during the exam period.
My plan
Major goals
While the most important goal is to merge both (old and new) language pairs, most of the work during the summer will be related to dictionaries and transfer rules:
- Decent WER (~32%)
- Good coverage (~90%)
- Testvoc clean
- New stems in bidix (~2000 stems a week)
- Old rule refactoring
- Additional transfer rules, lexical selection rules and, if necessary, CG.
Workplan
| Week | Dates | Goals | Bidix | WER / PER | Coverage |
|---|---|---|---|---|---|
| Post-application period | 4 April - 29 May |
|
~35,000 |
41.15%/29.34% (en-ca) 47.63%/38.5% (eng-cat) |
~85.9% |
| 1 | 30 May - 4 June |
|
~37,000 | ~45% | ~86.3% |
| 2 | 5 June - 11 June |
|
~39,000 | ~43.5% | ~86.7% |
| 3 | 12 June - 18 June |
|
~41,000 | ~42% | ~87.1% |
| 4 | 19 June - 25 June |
|
~43,000 | ~40.5% | ~87.5% |
| 5 | 26 June - 2 July |
Deliverable #1 |
~45,000 | ~39% | ~87.8% |
| 6 | 3 July - 9 July |
|
~47,000 | ~38% | ~88.1% |
| 7 | 10 July - 16 July |
|
~49,000 | ~37% | ~88.5% |
| 8 | 17 July - 23 July |
|
~51,000 | ~36% | ~88.8% |
| 9 | 24 July - 30 July |
Deliverable #2 |
~53,000 | ~35% | ~89.1% |
| 10 | 31 July - 6 August |
|
~55,000 | ~34% | ~89.4% |
| 11 | 7 August - 13 August |
|
~56,500 | ~33.5% | ~89.6% |
| 12 | 14 August - 20 August |
|
~58,000 | ~33% | ~89.8% |
| 13 | 21 August - 27 August |
Final evaluation |
~59,000 | ~32.5% | ~89.9% |
Coding challenge
During the application period, I decided to test my skills by improving the performance of the en-ca language pair. For this purpose, four 500-word fragments of Wikipedia featured articles were translated using Apertium and post-edited. Two of the texts were later analyzed and new stems and transfer rules were added to improve their translation. (The changes can be seen here [2].) Finally, the four articles were retranslated and both the old and new translations were evaluated using apertium-eval-translator. The test results of the four texts together (2000 words) were the following:
Before the changes
Statistics about input files ------------------------------------------------------- Number of words in reference: 2188 Number of words in test: 2056 Number of unknown words (marked with a star) in test: 159 Percentage of unknown words: 7.73 % Results when removing unknown-word marks (stars) ------------------------------------------------------- Edit distance: 946 Word error rate (WER): 43.24 % Number of position-independent correct words: 1569 Position-independent word error rate (PER): 28.29 % Results when unknown-word marks (stars) are not removed ------------------------------------------------------- Edit distance: 1006 Word Error Rate (WER): 45.98 % Number of position-independent correct words: 1502 Position-independent word error rate (PER): 31.35 % Statistics about the translation of unknown words ------------------------------------------------------- Number of unknown words which were free rides: 60 Percentage of unknown words that were free rides: 37.74 %
After the changes
Statistics about input files ------------------------------------------------------- Number of words in reference: 2188 Number of words in test: 2067 Number of unknown words (marked with a star) in test: 86 Percentage of unknown words: 4.16 % Results when removing unknown-word marks (stars) ------------------------------------------------------- Edit distance: 838 Word error rate (WER): 38.30 % Number of position-independent correct words: 1660 Position-independent word error rate (PER): 24.13 % Results when unknown-word marks (stars) are not removed ------------------------------------------------------- Edit distance: 873 Word Error Rate (WER): 39.90 % Number of position-independent correct words: 1622 Position-independent word error rate (PER): 25.87 % Statistics about the translation of unknown words ------------------------------------------------------- Number of unknown words which were free rides: 35 Percentage of unknown words that were free rides: 40.70 %
