Difference between revisions of "Maltese and Arabic/Work plan"

From Apertium
Jump to navigation Jump to search
(Google WER was a real post-edit too)
(hide it a bit)
Line 10: Line 10:
 
! Week !! Dates !! B-trimmed Cov. Goal !! B-trimmed Cov. Reached (news, wp) !! Testvoc !! Evaluation !! WER Reached !! (Google WER) !! Notes
 
! Week !! Dates !! B-trimmed Cov. Goal !! B-trimmed Cov. Reached (news, wp) !! Testvoc !! Evaluation !! WER Reached !! (Google WER) !! Notes
 
|-
 
|-
| 0 || <s>23/04&mdash;20/05</s> || 45% || 68.6%, 68.1% || {{tag|sent}} {{tag|cm}} {{tag|ij}} || 500 words || 3.17 % || 47.44 % || '''Preliminary evaluation'''. Translate the story total coverage and without diagnostics. Get a baseline WER. Aim to get the two mt verb scripts merged.
+
| 0 || <s>23/04&mdash;20/05</s> || 45% || 68.6%, 68.1% || {{tag|sent}} {{tag|cm}} {{tag|ij}} || 500 words || 3.17 % || (47.44 %) || '''Preliminary evaluation'''. Translate the story total coverage and without diagnostics. Get a baseline WER. Aim to get the two mt verb scripts merged.
 
|-
 
|-
 
| 1 || <s>21/05&mdash;27/05</s> || 50% || 74.2%, 73.8% || {{tag|num}} {{tag|pr}} || - || - || - || Basic numerals, and prepositions should be testvoc clean. Maltese verb scripts merged.
 
| 1 || <s>21/05&mdash;27/05</s> || 50% || 74.2%, 73.8% || {{tag|num}} {{tag|pr}} || - || - || - || Basic numerals, and prepositions should be testvoc clean. Maltese verb scripts merged.
Line 16: Line 16:
 
| 2 || <s>28/05&mdash;03/06</s> || 53% || 75.3%, 74.5% || {{tag|cnjcoo}} {{tag|cnjadv}} {{tag|cnjsub}} || - || - || - || Create ar verb script based on mt, add ar open-class words according to frequency
 
| 2 || <s>28/05&mdash;03/06</s> || 53% || 75.3%, 74.5% || {{tag|cnjcoo}} {{tag|cnjadv}} {{tag|cnjsub}} || - || - || - || Create ar verb script based on mt, add ar open-class words according to frequency
 
|-
 
|-
| 3 || <s>04/06&mdash;10/06</s> || 59% || 75.3%, 74.4% || {{tag|adv}} || 200 words || 8.70 % || 20.94 % || Continue adding open class words to ar (adding to bidix/mt as you go)
+
| 3 || <s>04/06&mdash;10/06</s> || 59% || 75.3%, 74.4% || {{tag|adv}} || 200 words || 8.70 % || (20.94 %) || Continue adding open class words to ar (adding to bidix/mt as you go)
 
|-
 
|-
 
| 4 || <s>11/06&mdash;17/06</s> || 63% || 76.0%, 75.1% || {{tag|prn}} {{tag|det}} || - || - || - || Monodix completion; closed classes (apart from verb-attached pronouns) should be testvoc clean.
 
| 4 || <s>11/06&mdash;17/06</s> || 63% || 76.0%, 75.1% || {{tag|prn}} {{tag|det}} || - || - || - || Monodix completion; closed classes (apart from verb-attached pronouns) should be testvoc clean.
Line 22: Line 22:
 
| 5 || <s>18/06&mdash;24/06</s> || 68% || 79.0%, 78.3% || {{tag|adj}} || - || - || - || Bidix expansion; bidix adjectives should be testvoc clean.
 
| 5 || <s>18/06&mdash;24/06</s> || 68% || 79.0%, 78.3% || {{tag|adj}} || - || - || - || Bidix expansion; bidix adjectives should be testvoc clean.
 
|-
 
|-
| 6 || <s>25/06&mdash;01/07</s> || 70% || 79.1%, 78.4% || {{tag|n}} {{tag|np}} || 500 words || 23.11% || 33.53 % || '''Midterm evaluation'''. Bidix expansion, multiwords.
+
| 6 || <s>25/06&mdash;01/07</s> || 70% || 79.1%, 78.4% || {{tag|n}} {{tag|np}} || 500 words || 23.11% || (33.53 %) || '''Midterm evaluation'''. Bidix expansion, multiwords.
 
|-
 
|-
 
| 7 || <s>02/07&mdash;08/07</s> || 73% || 79.4%, 78.6% || - || - || - || - || Transfer
 
| 7 || <s>02/07&mdash;08/07</s> || 73% || 79.4%, 78.6% || - || - || - || - || Transfer
Line 28: Line 28:
 
| 8 || <s>09/07&mdash;15/07</s> || 75% || 79.5%, 78.8% || - || - || - || - || Transfer, corpus testvoc
 
| 8 || <s>09/07&mdash;15/07</s> || 75% || 79.5%, 78.8% || - || - || - || - || Transfer, corpus testvoc
 
|-
 
|-
| 9 || <s>16/07&mdash;22/07</s> || 77% || 80.5%, 80.0% || - || 200 words || 17.28% || 35.89 % || Disambiguation, retrain tagger
+
| 9 || <s>16/07&mdash;22/07</s> || 77% || 80.5%, 80.0% || - || 200 words || 17.28% || (35.89 %) || Disambiguation, retrain tagger
 
|-
 
|-
 
| 10 || <s>23/07&mdash;29/07</s> || 80% || 80.9%, 80.2% || {{tag|v}} || - || - || - || Testvoc
 
| 10 || <s>23/07&mdash;29/07</s> || 80% || 80.9%, 80.2% || {{tag|v}} || - || - || - || Testvoc

Revision as of 09:55, 6 August 2012

This is a workplan for development efforts for the Maltese and Arabic translator in Google Summer of Code 2012.

  • Bidix-trimmed coverage means the coverage of the morphological analyser after being trimmed according to the bilingual dictionary of the pair, that is, only containing stems which are also in the bilingual dictionary (but omissions leading to generation errors do not change the trimmed coverage)
  • Testvoc for a category means that the category is testvoc clean, in both translation directions.
  • Evaluation is taking words and performing an evaluation for post-edition word error rate (WER). The output for those words should be clean, ie. no #/@, and no (or as few as possible) *, for all WER tests.


Week Dates B-trimmed Cov. Goal B-trimmed Cov. Reached (news, wp) Testvoc Evaluation WER Reached (Google WER) Notes
0 23/04—20/05 45% 68.6%, 68.1% <sent> <cm> <ij> 500 words 3.17 % (47.44 %) Preliminary evaluation. Translate the story total coverage and without diagnostics. Get a baseline WER. Aim to get the two mt verb scripts merged.
1 21/05—27/05 50% 74.2%, 73.8% <num> <pr> - - - Basic numerals, and prepositions should be testvoc clean. Maltese verb scripts merged.
2 28/05—03/06 53% 75.3%, 74.5% <cnjcoo> <cnjadv> <cnjsub> - - - Create ar verb script based on mt, add ar open-class words according to frequency
3 04/06—10/06 59% 75.3%, 74.4% <adv> 200 words 8.70 % (20.94 %) Continue adding open class words to ar (adding to bidix/mt as you go)
4 11/06—17/06 63% 76.0%, 75.1% <prn> <det> - - - Monodix completion; closed classes (apart from verb-attached pronouns) should be testvoc clean.
5 18/06—24/06 68% 79.0%, 78.3% <adj> - - - Bidix expansion; bidix adjectives should be testvoc clean.
6 25/06—01/07 70% 79.1%, 78.4% <n> <np> 500 words 23.11% (33.53 %) Midterm evaluation. Bidix expansion, multiwords.
7 02/07—08/07 73% 79.4%, 78.6% - - - - Transfer
8 09/07—15/07 75% 79.5%, 78.8% - - - - Transfer, corpus testvoc
9 16/07—22/07 77% 80.5%, 80.0% - 200 words 17.28% (35.89 %) Disambiguation, retrain tagger
10 23/07—29/07 80% 80.9%, 80.2% <v> - - - Testvoc
11 30/07—05/08 80% - - - - - Testvoc
12 06/08—12/08 80% - all categories clean 500 words ? - Final evaluation. Tidying up, releasing