English and Catalan/GSOC 2017 - Revision history

Marcriera at 22:24, 31 August 2017

2017-08-31T22:24:38Z

Marcriera at 14:59, 28 August 2017

2017-08-28T14:59:35Z

Marcriera: /* Transfer rules */

2017-08-28T14:55:34Z

Transfer rules

Marcriera: /* Final words */

2017-08-28T14:41:53Z

Final words

Marcriera at 09:31, 28 August 2017

2017-08-28T09:31:29Z

Marcriera at 08:11, 28 August 2017

2017-08-28T08:11:27Z

Marcriera at 22:45, 27 August 2017

2017-08-27T22:45:58Z

Marcriera: /* Dictionaries */

2017-08-27T17:41:19Z

Dictionaries

Marcriera: /* Automatisation */

2017-08-26T21:54:58Z

Automatisation

Marcriera: /* Supporting work */

2017-08-26T17:10:32Z

Supporting work

@@ Line 42: / Line 42: @@
 Given that these scripts may be very useful to other developers, they are now available in [https://github.com/MarcRiera/apertium-scripts this repository].
+==Future work==
+===Work needed for release===
+* Make the pair testvoc clean.
+* Fix the gender and number of Catalan proper nouns.
+===Other plans===
+* Rewrite the transfer rules related to verb conjugation.
+* Improve the translation of some words using lexical seletion rules.
+* Make better use of separable verbs and spelling variants.
 ==Final words==

← Older revision		Revision as of 14:59, 28 August 2017
Line 1:		Line 1:
	This page serves as a summary of all the work done in the English-Catalan pair during Google Summer of Code 2017. For a more detailed workplan, please check [[English_and_Catalan/Workplan\|this page]].		This page serves as a summary of all the work done in the English-Catalan pair during Google Summer of Code 2017. For a more detailed workplan of the project, please check [[English_and_Catalan/Workplan\|this page]].

	==Main work==		==Main work==

@@ Line 15: / Line 15: @@
 ===Transfer rules===
-Transfer rules were (and still are) perhaps the most fragile component of this pair. The number of rules is quite high given the differences between the two languages, several rules take place through different transfer stages, and the code is quite convoluted. Work related to transfer rules has focused on code updates to ensure everything is applied correctly and some new rule additions, mostly related to verbs.
+Transfer rules were (and still are) perhaps the most fragile component of this pair. The number of rules is quite high given the differences between the two languages, several rules take place through different transfer stages, and the code is quite convoluted. Work related to transfer rules has focused on code updates to ensure everything is applied correctly and some new rule additions, mostly related to verbs. The final WER/PER (51.08%/33.68%) is higher than expected, but it will surely improve once the pair is testvoc clean and proper nouns are corrected.
 ===Separable verbs===

@@ Line 43: / Line 43: @@
 Given that these scripts may be very useful to other developers, they are now available in [https://github.com/MarcRiera/apertium-scripts this repository].
-===Final words===
+==Final words==
 Working with Apertium as part of Google Summer of Code has been an amazing experience. As a translator and not a professional programmer, there was a lot to learn at the beginning, but once I had become familiar with the file formats everything felt easier. Using the English-Catalan pair every day has made me see its pros and cons, as well as its importance. Even though it may never become a perfect translator, I am committed to keep working on it and improve it even more.

@@ Line 9: / Line 9: @@
 ===Dictionaries===
-On the one hand, the bilingual dictionary has grown from 35,000 entries to more than 66,000. This is significantly higher than the initial goal of 59,000 entries thanks to supporting work done on automatisation, which has been very helpful. In addition to new entries (added from frequency lists and crossdics), entries broken due to tag changes have been fixed, and the dictionary has been reorganised and cleaned. The monolingual dictionaries have been reorganised too, so developers of other pairs using the English and Catalan dictionaries will notice improvements.
+On the one hand, the bilingual dictionary has grown from 35,000 entries to more than 66,000. This is significantly higher than the initial goal of 59,000 entries thanks to supporting work done on automation, which has been very helpful. In addition to new entries (added from frequency lists and crossdics), entries broken due to tag changes have been fixed, and the dictionary has been reorganised and cleaned. The monolingual dictionaries have been reorganised too, so developers of other pairs using the English and Catalan dictionaries will notice improvements.
 On the other hand, unexpected testvoc errors (over 7,000 for EN>CA and around 10,000 for CA>EN) prevent the pair from being released in trunk immediately after GSoC. However, the pair is mature enough to completely replace the old apertium-en-ca pair once it becomes "testvoc clean". There is also pending work regarding proper noun tags in Catalan (they lack gender and number), but this is mostly complete and there are only ~900 lemmas left to change before the fixes can be merged.
@@ Line 37: / Line 37: @@
 * [[Perceptron tagger]]
-===Automatisation===
+===Automation===
 In order to make pair development easier and faster, several bash scripts have been created and used. The most remarkable one is ''transfer_documentation.sh'', which allows the user to create wikitables with a lot of information about transfer rules from data embedded into the rule files (T1X, T2X and T3X). There are other scripts that can help with dictionary entries too, specially with proper nouns, as they usually do not need obscure paradigms.

← Older revision		Revision as of 22:45, 27 August 2017
Line 1:		Line 1:
	This page serves as a summary of all the work done in the English-Catalan pair during Google Summer of Code 2017.		This page serves as a summary of all the work done in the English-Catalan pair during Google Summer of Code 2017. For a more detailed workplan, please check [[English_and_Catalan/Workplan\|this page]].

	==Main work==		==Main work==

@@ Line 26: / Line 26: @@
 ==Supporting work==
+===Documentation===
+The lack of documentation regarding the language pair, the monolingual dictionaries or even the tagger has made me put an effort on documenting as many things as possible to help current and future Apertium developers and to define the current status of the different modules better. The most important work has been to build a catalogue of all the transfer rules and macros in the pair, but there is also new documentation about the dictionaries and guidelines to make team work easier:
+* [[English]]
+* [[Catalan]]
+* [[English and Catalan]]
+* [[English and Catalan/Transfer Rules]]
+* [[Perceptron tagger]]
+===Automatisation===