Difference between revisions of "Maltese and Arabic/Work plan"

From Apertium
Jump to navigation Jump to search
 
(33 intermediate revisions by 2 users not shown)
Line 1: Line 1:
This is a workplan for development efforts for the [[Maltese and Arabic]] translator in [[Google Summer of Code]] 2012.
This is a workplan for development efforts for the [[Maltese and Arabic]] translator in [[Google Summer of Code]] 2012.


* Trimmed coverage means the coverage of both of the morphological analysers after being trimmed according to the bilingual dictionary of the pair, that is, only containing stems which are also in the bilingual dictionary.
* Bidix-trimmed coverage means the coverage of the morphological analyser after being trimmed according to the bilingual dictionary of the pair, that is, only containing stems which are also in the bilingual dictionary (but omissions leading to generation errors do not change the trimmed coverage)
* Testvoc for a category means that the category is [[testvoc]] clean, in both translation directions.
* Testvoc for a category means that the category is [[testvoc]] clean, in both translation directions.
* Evaluation is taking <math>n</math> words and performing an [[evaluation]] for post-edition word error rate (WER). The output for those <math>n</math> words should be clean.
* Evaluation is taking <math>n</math> words and performing an [[evaluation]] for post-edition word error rate (WER). The output for those <math>n</math> words should be clean, ie. no #/@, and no (or as few as possible) *, for all WER tests.
** Note on the WER results: both tests use references that are post-edited from the respective MT outputs, but the texts were used in apertium development, so unrestricted texts would probably look quite different. Texts are in SVN in apertium-mt-ar/dev/story/



{|class=wikitable
{|class=wikitable
|-
|-
! Week !! Dates !! Trimmed coverage !! Testvoc !! Evaluation !! Notes
! Week !! Dates !! B-trimmed Cov. Goal !! B-trimmed Cov. Reached (news, wp) !! Testvoc !! Evaluation !! WER Reached !! (Google WER) !! Notes
|-
|-
| 0 || 23/04&mdash;21/05 || 45% || {{tag|postadv}} {{tag|ij}} || 500 words || '''Preliminary evaluation'''. Translate the story total coverage and without diagnostics. Get a baseline WER. Work on disambiguation, the morphological ambiguities in the story should be resolved. Aim to get the two mt verb scripts merged.
| 0 || <s>23/04&mdash;20/05</s> || 45% || 68.6%, 68.1% || {{tag|sent}} {{tag|cm}} {{tag|ij}} || 500 words || 3.17 % || (47.44 %) || '''Preliminary evaluation'''. Translate the story total coverage and without diagnostics. Get a baseline WER. Aim to get the two mt verb scripts merged.
|-
|-
| 1 || 21/05&mdash;27/05 || 50% || {{tag|num}} {{tag|post}} || - || Basic numerals, and postpositions should be clean.
| 1 || <s>21/05&mdash;27/05</s> || 50% || 74.2%, 73.8% || {{tag|num}} {{tag|pr}} || - || - || - || Basic numerals, and prepositions should be testvoc clean. Maltese verb scripts merged.
|-
|-
| 2 || 28/05&mdash;03/06 || 53% || {{tag|cnjcoo}} {{tag|cnjadv}} {{tag|cnjsub}} || - ||
| 2 || <s>28/05&mdash;03/06</s> || 53% || 75.3%, 74.5% || {{tag|cnjcoo}} {{tag|cnjadv}} {{tag|cnjsub}} || - || - || - || Create ar verb script based on mt, add ar open-class words according to frequency
|-
|-
| 3 || 04/06&mdash;10/06 || 59% || {{tag|adv}} || 200 words || Work on disambiguation.
| 3 || <s>04/06&mdash;10/06</s> || 59% || 75.3%, 74.4% || {{tag|adv}} || 200 words || 8.70 % || (20.94 %) || Continue adding open class words to ar (adding to bidix/mt as you go)
|-
|-
| 4 || 11/06&mdash;17/06 || 63% || {{tag|prn}} {{tag|det}} || - ||
| 4 || <s>11/06&mdash;17/06</s> || 63% || 76.0%, 75.1% || {{tag|prn}} {{tag|det}} || - || - || - || Monodix completion; closed classes (apart from verb-attached pronouns) should be testvoc clean.
|-
|-
| 5 || 18/06&mdash;24/06 || 68% || {{tag|adj}} {{tag|adj><advl}} || - || Adjectives with attributive and adverbial function should be clean. Numerals + affixes should have their phonology solved.
| 5 || <s>18/06&mdash;24/06</s> || 68% || 79.0%, 78.3% || {{tag|adj}} || - || - || - || Bidix expansion; bidix adjectives should be testvoc clean.
|-
|-
| 6 || 25/06&mdash;01/07 || 70% || {{tag|n}} {{tag|num><subst}} {{tag|np}} {{tag|adj><subst}} || 500 words || '''Midterm evaluation'''. Work on disambiguation.
| 6 || <s>25/06&mdash;01/07</s> || 70% || 79.1%, 78.4% || {{tag|n}} {{tag|np}} || 500 words || 23.11 % || (33.53 %) || '''Midterm evaluation'''. Bidix expansion, multiwords.
|-
|-
| 7 || 02/07&mdash;08/07 || 73% || - || - ||
| 7 || <s>02/07&mdash;08/07</s> || 73% || 79.4%, 78.6% || - || - || - || - || Transfer
|-
|-
| 8 || 09/07&mdash;15/07 || 75% || - || - ||
| 8 || <s>09/07&mdash;15/07</s> || 75% || 79.5%, 78.8% || - || - || - || - || Transfer, corpus testvoc
|-
|-
| 9 || 16/07&mdash;22/07 || 77% || - || 200 words || Work on disambiguation.
| 9 || <s>16/07&mdash;22/07</s> || 77% || 80.5%, 80.0% || - || 200 words || 17.28 % || (35.89 %) || Disambiguation, retrain tagger
|-
|-
| 10 || 23/07&mdash;29/07 || 80% || {{tag|v}} || - ||
| 10 || <s>23/07&mdash;29/07</s> || 80% || 80.9%, 80.2% || {{tag|v}} || - || - || - || Testvoc
|-
|-
| 11 || 30/07&mdash;05/08 || 80% || - || - || Work on disambiguation.
| 11 || <s>30/07&mdash;05/08</s> || 80% || 81.2%, 80.5% || - || - || - || - || Testvoc
|-
|-
| 12 || 06/08&mdash;12/08 || 80% || ''all categories clean'' || 500 words || '''Final evaluation'''. Tidying up, releasing
| 12 || <s>06/08&mdash;12/08</s> || 80% || 82.0%, 80.9% || ''all categories clean'' || 500 words || 21.34 % || (39.00 %) || '''Final evaluation'''. Tidying up, releasing
|-
|-
|}
|}


[[Category:Maltese and Arabic]]
[[Category:Maltese and Arabic]]
[[Category:Google Summer of Code]]

Latest revision as of 15:28, 8 March 2013

This is a workplan for development efforts for the Maltese and Arabic translator in Google Summer of Code 2012.

  • Bidix-trimmed coverage means the coverage of the morphological analyser after being trimmed according to the bilingual dictionary of the pair, that is, only containing stems which are also in the bilingual dictionary (but omissions leading to generation errors do not change the trimmed coverage)
  • Testvoc for a category means that the category is testvoc clean, in both translation directions.
  • Evaluation is taking words and performing an evaluation for post-edition word error rate (WER). The output for those words should be clean, ie. no #/@, and no (or as few as possible) *, for all WER tests.
    • Note on the WER results: both tests use references that are post-edited from the respective MT outputs, but the texts were used in apertium development, so unrestricted texts would probably look quite different. Texts are in SVN in apertium-mt-ar/dev/story/


Week Dates B-trimmed Cov. Goal B-trimmed Cov. Reached (news, wp) Testvoc Evaluation WER Reached (Google WER) Notes
0 23/04—20/05 45% 68.6%, 68.1% <sent> <cm> <ij> 500 words 3.17 % (47.44 %) Preliminary evaluation. Translate the story total coverage and without diagnostics. Get a baseline WER. Aim to get the two mt verb scripts merged.
1 21/05—27/05 50% 74.2%, 73.8% <num> <pr> - - - Basic numerals, and prepositions should be testvoc clean. Maltese verb scripts merged.
2 28/05—03/06 53% 75.3%, 74.5% <cnjcoo> <cnjadv> <cnjsub> - - - Create ar verb script based on mt, add ar open-class words according to frequency
3 04/06—10/06 59% 75.3%, 74.4% <adv> 200 words 8.70 % (20.94 %) Continue adding open class words to ar (adding to bidix/mt as you go)
4 11/06—17/06 63% 76.0%, 75.1% <prn> <det> - - - Monodix completion; closed classes (apart from verb-attached pronouns) should be testvoc clean.
5 18/06—24/06 68% 79.0%, 78.3% <adj> - - - Bidix expansion; bidix adjectives should be testvoc clean.
6 25/06—01/07 70% 79.1%, 78.4% <n> <np> 500 words 23.11 % (33.53 %) Midterm evaluation. Bidix expansion, multiwords.
7 02/07—08/07 73% 79.4%, 78.6% - - - - Transfer
8 09/07—15/07 75% 79.5%, 78.8% - - - - Transfer, corpus testvoc
9 16/07—22/07 77% 80.5%, 80.0% - 200 words 17.28 % (35.89 %) Disambiguation, retrain tagger
10 23/07—29/07 80% 80.9%, 80.2% <v> - - - Testvoc
11 30/07—05/08 80% 81.2%, 80.5% - - - - Testvoc
12 06/08—12/08 80% 82.0%, 80.9% all categories clean 500 words 21.34 % (39.00 %) Final evaluation. Tidying up, releasing
Retrieved from "https://wiki.apertium.org/w/index.php?title=Maltese_and_Arabic/Work_plan&oldid=39478"