User:Marcriera/proposal - Revision history

Marcriera: /* Workplan */

2017-05-29T13:05:47Z

Workplan

Marcriera at 14:29, 3 April 2017

2017-04-03T14:29:55Z

Show changes

Marcriera: /* Coding challenge */

2017-04-03T12:30:24Z

Coding challenge

Marcriera: /* Which of the published tasks are you interested in? What do you plan to do? */

2017-04-03T08:15:07Z

Which of the published tasks are you interested in? What do you plan to do?

Marcriera: /* Workplan */

2017-04-03T08:01:15Z

Workplan

Marcriera at 16:51, 1 April 2017

2017-04-01T16:51:19Z

Marcriera at 16:41, 1 April 2017

2017-04-01T16:41:25Z

Marcriera at 16:02, 1 April 2017

2017-04-01T16:02:24Z

Marcriera: /* Workplan */

2017-04-01T11:30:05Z

Workplan

Marcriera: /* Workplan */

2017-04-01T11:23:03Z

Workplan

@@ Line 104: / Line 104: @@
 * Add new stems to dictionaries
 * Final testing to get rid of en-ca definitely
+* Write documentation about the current state of transfer rules
 | style="text-align:center" | ~39,000
 | style="text-align:center" | ~43.5%

@@ Line 209: / Line 209: @@
 == Coding challenge ==
+During the application period, I decided to test my skills by improving the performance of the en-ca language pair. For this purpose, four 500-word fragments of Wikipedia featured articles were translated using Apertium and post-edited. Two of the texts were later analyzed and new stems and transfer rules were added to improve their translation. (The changes can be seen here [https://github.com/xavivars/apertium-en-ca/pull/1].) Finally, the four articles were retranslated and both the old and new translations were evaluated using apertium-eval-translator. The test results of the four texts together (2000 words) were the following:
-Currently working on it...
+Before the changes
+<pre>
+Statistics about input files
+-------------------------------------------------------
+Number of words in reference: 2188
+Number of words in test: 2056
+Number of unknown words (marked with a star) in test: 159
+Percentage of unknown words: 7.73 %
+Results when removing unknown-word marks (stars)
+-------------------------------------------------------
+Edit distance: 946
+Word error rate (WER): 43.24 %
+Number of position-independent correct words: 1569
+Position-independent word error rate (PER): 28.29 %
+Results when unknown-word marks (stars) are not removed
+-------------------------------------------------------
+Edit distance: 1006
+Word Error Rate (WER): 45.98 %
+Number of position-independent correct words: 1502
+Position-independent word error rate (PER): 31.35 %
+Statistics about the translation of unknown words
+-------------------------------------------------------
+Number of unknown words which were free rides: 60
+Percentage of unknown words that were free rides: 37.74 %
+</pre>
+After the changes
+<pre>
+Statistics about input files
+-------------------------------------------------------
+Number of words in reference: 2188
+Number of words in test: 2067
+Number of unknown words (marked with a star) in test: 86
+Percentage of unknown words: 4.16 %
+Results when removing unknown-word marks (stars)
+-------------------------------------------------------
+Edit distance: 838
+Word error rate (WER): 38.30 %
+Number of position-independent correct words: 1660
+Position-independent word error rate (PER): 24.13 %
+Results when unknown-word marks (stars) are not removed
+-------------------------------------------------------
+Edit distance: 873
+Word Error Rate (WER): 39.90 %
+Number of position-independent correct words: 1622
+Position-independent word error rate (PER): 25.87 %
+Statistics about the translation of unknown words
+-------------------------------------------------------
+Number of unknown words which were free rides: 35
+Percentage of unknown words that were free rides: 40.70 %
+</pre>

@@ Line 29: / Line 29: @@
 Tagger training should not be necessary (at least in English), but depending on the results when translating from Catalan to English, it may be required. For this reason, there is time assigned to this task in the work plan. The same applies to constraint grammar; most of the new rules will be transfer or lexical selection rules, but some CG rules might be needed.
+Coverage will be calculated based on Wikipedia. Due to the enormous size of the English version, a word frequency list will be generated from a big part of Wikipedia (at least 3 GB of plain text), and coverage will be tested on it. This will allow more efficient token testing.
 === Title ===

@@ Line 115: / Line 115: @@
 |
 * Add new stems to dictionaries
+* Improve semi-automatisation when adding new stems (specially proper nouns)
 | style="text-align:center" | ~47,000
 | style="text-align:center" | ~36%

@@ Line 32: / Line 32: @@
 === Title ===
-'''Bringing English-Catalan language pair to state-of-the-art quality'''
+'''Adopting English-Catalan language pair to bring it close to state-of-the-art quality'''
 === Reasons why Google and Apertium should sponsor it ===
@@ Line 203: / Line 203: @@
 | style="text-align:center" | ~91.2%
 |}
-Final timeline to be determined
 == Coding challenge ==

@@ Line 56: / Line 56: @@
 While the most important goal is to merge both (old and new) language pairs, most of the work during the summer will be related to dictionaries and transfer rules:
-* Good WER (~20%)
+* Good WER (~19%)
-* Excellent coverage (~92%)
+* Excellent coverage (~91%)
 * Testvoc clean
 * New stems in bidix (~3000 stems a week)
@@ Line 81: / Line 81: @@
 .15%/29.34% (en-ca)
 .63%/38.5% (eng-cat)
+| style="text-align:center" | ~85.9%
-|
 |-
 ! 1
@@ Line 91: / Line 91: @@
 | style="text-align:center" | ~38,000
 | style="text-align:center" | ~44%
+| style="text-align:center" | ~86.6%
-|
 |-
 ! 2
@@ Line 100: / Line 100: @@
 | style="text-align:center" | ~41,000
 | style="text-align:center" | ~41%
+| style="text-align:center" | ~87.1%
-|
 |-
 ! 3
@@ Line 109: / Line 109: @@
 | style="text-align:center" | ~44,000
 | style="text-align:center" | ~38%
+| style="text-align:center" | ~87.7%
-|
 |-
 ! 4
@@ Line 116: / Line 116: @@
 * Add new stems to dictionaries
 | style="text-align:center" | ~47,000
-| style="text-align:center" | ~35%
+| style="text-align:center" | ~36%
+| style="text-align:center" | ~88.2%
-|
 |-
 ! 5
@@ Line 127: / Line 127: @@
 | style="text-align:center" | ~50,000
 | style="text-align:center" | ~32%
+| style="text-align:center" | ~88.7%
-|
 |-
 ! 6
@@ Line 135: / Line 135: @@
 | style="text-align:center" | ~53,000
 | style="text-align:center" | ~30%
+| style="text-align:center" | ~89.1%
-|
 |-
 ! 7
@@ Line 143: / Line 143: @@
 | style="text-align:center" | ~56,000
 | style="text-align:center" | ~28%
+| style="text-align:center" | ~89.5%
-|
 |-
 ! 8
@@ Line 152: / Line 152: @@
 | style="text-align:center" | ~59,000
 | style="text-align:center" | ~26%
+| style="text-align:center" | ~89.9%
-|
 |-
 ! 9
@@ Line 163: / Line 163: @@
 | style="text-align:center" | ~62,000
 | style="text-align:center" | ~24%
+| style="text-align:center" | ~90.3%
-|
 |-
 ! 10
@@ Line 172: / Line 172: @@
 | style="text-align:center" | ~65,000
 | style="text-align:center" | ~22%
+| style="text-align:center" | ~90.7%
-|
 |-
 ! 11
@@ Line 181: / Line 181: @@
 | style="text-align:center" | ~67,000
 | style="text-align:center" | ~21%
+| style="text-align:center" | ~90.9%
 |-
 ! 12
@@ Line 190: / Line 190: @@
 | style="text-align:center" | ~69,000
 | style="text-align:center" | ~20%
+| style="text-align:center" | ~91.1%
-|
 |-
 ! 13
@@ Line 201: / Line 201: @@
 | style="text-align:center" | ~70,000
 | style="text-align:center" | ~19%
-| style="text-align:center" | ~92%
+| style="text-align:center" | ~91.2%
 |}