User:Deltamachine/proposal2018 - Revision history

Deltamachine: /* Which of the published tasks are you interested in? What do you plan to do? */

2018-03-26T09:19:18Z

Which of the published tasks are you interested in? What do you plan to do?

Deltamachine at 13:30, 24 March 2018

2018-03-24T13:30:32Z

Deltamachine: /* Contact information */

2018-03-24T13:23:00Z

Contact information

Deltamachine: /* Choosing a language pair(s) to experiment with and collecting/processing data. */

2018-03-23T17:43:50Z

Choosing a language pair(s) to experiment with and collecting/processing data.

Deltamachine at 10:19, 23 March 2018

2018-03-23T10:19:33Z

Deltamachine: /* Coding challenge */

2018-03-23T09:22:03Z

Coding challenge

Deltamachine: /* Improving of existing methods */

2018-03-23T09:20:27Z

Improving of existing methods

Deltamachine: /* Reasons why Google and Apertium should sponsor it */

2018-03-23T09:19:34Z

Reasons why Google and Apertium should sponsor it

Deltamachine: /* Work stages */

2018-03-23T09:16:52Z

Work stages

Deltamachine: /* Why is it that you are interested in Apertium? */

2018-03-23T09:06:58Z

Why is it that you are interested in Apertium?

@@ Line 57: / Line 57: @@
 ==== Choosing a language pair(s) to experiment with and collecting/processing data. ====
-<u>Abour collecting and processing data</u>
+<u>About collecting and processing data</u>
 There might be two approaches.
-* Using Mediawiki JSON postediting data from https://dumps.wikimedia.org/other/contenttranslation/. <p>'''+''' the target (postedited) side is very close to the given machine translation because it is basically based on it.</p> <p>'''-''' Mediawiki articles often contain very long and very specific sentences and these factors can affect the quality of extracted triplets, because the current method of extracting them is built on edit distance algorithm. </p> <p>'''-''' also there will surely be words and sentences in other languages (like translations, links and so on, typical Wikipedia content) and it will make our data noisy.</p>
+* Using Mediawiki JSON postediting data from https://dumps.wikimedia.org/other/contenttranslation/. <p>'''+''' the target (postedited) side is very close to the given machine translation because it is basically based on it.</p> <p>'''-''' Mediawiki articles often contain very long and very specific sentences and these factors can affect the quality of extracted triplets, because the current method of extracting them is built on edit distance algorithm. </p> <p>'''-''' there will surely be words and sentences in other languages (like translations, links and so on, typical Wikipedia content) and it will make our data noisy.</p><p>'''-''' sometimes posteditors change the contents of a paragraph to make the article better: they split original sentences, add new information, etc. But these cases could probably be filtered.</p>
 * Using parallel corpora and Apertium translation of the source side. <p>'''+''' in parallel corpora, especially in those, which are not Europarl, sentences are usually not very long and complicated and contain pretty common words and phrases (Tatoeba might be a good example). This is not a rule, but, in my opinion, parallel corpora are still less specific than Mediawiki articles.</p> <p>'''+''' parallel corpora are more likely to contain less noise.</p> <p>'''-''' the target side might be very different from Apertium-translated one, especially if we talk about long and complicated sentences.</p>

@@ Line 38: / Line 38: @@
 Also I have participated in Google Code-In 2017 as Apertium mentor and it was great too. So I am very interested in further contributing to Apertium.
-This organisation works on things which are very interesting for me as a computational linguist: (rule-based) machine translation, minority languages, NLP and so on. I love that Apertium works with minority languages because it is very important for saving such languages.
+This organisation works on things which are very interesting for me as a computational linguist: (rule-based) machine translation, minority languages, NLP and so on. I love that Apertium works with minority languages because it is very important for support those languages.
 Also Apertium community is very friendly and open to new members, people here are always ready to help you. It encourages me to work with these people.
@@ Line 52: / Line 52: @@
 * MT(S): machine translation of S
 * PE(MT(S)): post-editing of the machine translation of S
-* O(s, mt, pe): set of extracted postediting operations where s in S, mt in MT(S), pe in PE(MT(S))
+* O(s, mt, pe): set of extracted postediting operations where s in substr(S), mt in substr(MT(S)), pe in substr(PE(MT(S)))
 === Work stages ===
@@ Line 61: / Line 61: @@
 There might be two approaches.
-* Using Mediawiki JSON postediting data from https://dumps.wikimedia.org/other/contenttranslation/. <p>'''+''' the target (postedited) side is very close to the given machine translation because it is basically based on it.</p> <p>'''-''' Mediawiki articles often contain very long and very specific sentences and these factors can affect the quality of extracted triplets, because the current method of extracting them is built on edit distance algorithm. </p> <p>'''-''' also there will surely be words and sentences in other languages (like translations, links and so on, typical Wikipedia content) and it will noise our data.</p>
+* Using Mediawiki JSON postediting data from https://dumps.wikimedia.org/other/contenttranslation/. <p>'''+''' the target (postedited) side is very close to the given machine translation because it is basically based on it.</p> <p>'''-''' Mediawiki articles often contain very long and very specific sentences and these factors can affect the quality of extracted triplets, because the current method of extracting them is built on edit distance algorithm. </p> <p>'''-''' also there will surely be words and sentences in other languages (like translations, links and so on, typical Wikipedia content) and it will make our data noisy.</p>
 * Using parallel corpora and Apertium translation of the source side. <p>'''+''' in parallel corpora, especially in those, which are not Europarl, sentences are usually not very long and complicated and contain pretty common words and phrases (Tatoeba might be a good example). This is not a rule, but, in my opinion, parallel corpora are still less specific than Mediawiki articles.</p> <p>'''+''' parallel corpora are more likely to contain less noise.</p> <p>'''-''' the target side might be very different from Apertium-translated one, especially if we talk about long and complicated sentences.</p>
@@ Line 67: / Line 67: @@
 The question of choosing an approach is pretty discussable. I think we might experiment with both approaches or even mix different types of data and see if there any difference.
-<u>About language pair</u>
+<u>About language pairs</u>
 I would like to work with languages I know or at least can more or less understand. Since I'm a native Russian speaker, it seems to be a good idea to work with bel-rus and ukr-rus. Though these pairs are already released, there are still a lot of work to be done. Also those are closely related languages so there will be less problems with alignment.
@@ Line 81: / Line 81: @@
 * Russian -> Ukranian Mediawiki corpus of Apertium translated and postedited data = 60 sentences.
 * Tatoeba parallel corpus = about 6500 sentences
-* OpenSubtitles2016 parallel corpus = about 400000 sentences (might contain loose translations)
+* OpenSubtitles2016 parallel corpus = about 400000 sentences (might contain free translations)
-* OpenSubtitles2018 parallel corpus = about 600000 sentences (might contain loose translations)
+* OpenSubtitles2018 parallel corpus = about 600000 sentences (might contain free translations)
 * A few specific parallel corpora like KDE and GNOME
@@ Line 135: / Line 135: @@
 ==== Searching for extracted postediting operations that improve the quality of translation ====
-The next step is to process a big train set using different parameters, extract potential postediting operations, apply them to the test set and find those that improve the quality of original Apertium translation on a regular basis.
+The next step is to process a big training set using different parameters, extract potential postediting operations, apply them to the test set and find those that improve the quality of original Apertium translation on a regular basis.
 A language model score might be a criteria of quality improvement.  A safety of a postediting operation might be determined by statistical methods.
@@ Line 238: / Line 238: @@
 <ul>
-<li>''parse_ct_json.py:'' A script that parses Mediawiki JSON file and splits the whole corpus on train and test sets of a given size.</li>
+<li>''parse_ct_json.py:'' A script that parses Mediawiki JSON file and splits the whole corpus on training and test sets of a given size.</li>
 <li>''estimate_changes.py:'' A script that takes a file generated by ''apply_postedits.py'' and scores sentences which were processed with postediting rules on a language model.</li>
 </ul>

@@ Line 1: / Line 1: @@
 == Contact information ==
 <p>'''Name:''' Anna Kondrateva</p>
-<p>'''Location:''' Moscow, Russia</p>
+<p>'''Location:''' Moscow/Yekaterinburg, Russia</p>
 <p>'''E-mail:''' an-an-kondratjeva@yandex.ru</p>
 <p>'''Phone number:''' +79250374221</p>
 <p>'''IRC:''' deltamachine</p>
-<p>'''SourceForge:''' deltamachine</p>
+<p>'''Github:''' deltamachine</p>
-<p>'''Timezone:''' UTC+3</p>
+<p>'''Timezone:''' UTC+3 (Moscow) / UTC+5 (Yekaterinburg)</p>
 == Skills and experience ==

@@ Line 57: / Line 57: @@
 ==== Choosing a language pair(s) to experiment with and collecting/processing data. ====
-<u>About language pair</u>
-I would like to work with languages I know or at least can more or less understand. Since I'm a native Russian speaker, it seems to be a good idea to work with bel-rus and ukr-rus. Though these pairs are already released, there are still a lot of work to be done. Also those are closely related languages so there will be less problems with alignment.
-But the methods I'm going to develop are not going to be tied to a language pair.
 <u>Abour collecting and processing data</u>
@@ Line 73: / Line 66: @@
 The question of choosing an approach is pretty discussable. I think we might experiment with both approaches or even mix different types of data and see if there any difference.
+<u>About language pair</u>
+I would like to work with languages I know or at least can more or less understand. Since I'm a native Russian speaker, it seems to be a good idea to work with bel-rus and ukr-rus. Though these pairs are already released, there are still a lot of work to be done. Also those are closely related languages so there will be less problems with alignment.
+The problem with working with bel-rus and ukr-rus is the comparably small amount of postedited data and more or less suitable parallel corpora. I'm still looking for data, but current situation looks like this:
+''' bel-rus '''
+* Russian -> Belarusian Mediawiki corpus of Apertium translated and postedited data = 1895 sentences.
+* Tatoeba parallel corpus = about 1800 sentences
+* A few specific parallel corpora like KDE and GNOME
+''' ukr-rus'''
+* Russian -> Ukranian Mediawiki corpus of Apertium translated and postedited data = 60 sentences.
+* Tatoeba parallel corpus = about 6500 sentences
+* OpenSubtitles2016 parallel corpus = about 400000 sentences (might contain loose translations)
+* OpenSubtitles2018 parallel corpus = about 600000 sentences (might contain loose translations)
+* A few specific parallel corpora like KDE and GNOME
+But the methods I'm going to develop are not going to be tied to a language pair. We might choose another language pair or start our experiments with a small amount of data. The question is discussable.
 ==== Improving of existing methods ====

← Older revision		Revision as of 10:19, 23 March 2018
Line 232:		Line 232:

	Also I have refactored and documented the old code from https://github.com/mlforcada/naive-automatic-postediting/blob/master/learn_postedits.py. A new version is stored in a given folder in ''cleaned_learn_postedits.py''		Also I have refactored and documented the old code from https://github.com/mlforcada/naive-automatic-postediting/blob/master/learn_postedits.py. A new version is stored in a given folder in ''cleaned_learn_postedits.py''

			[[Category:GSoC 2018 student proposals\|Deltamachine]]

@@ Line 228: / Line 228: @@
 <li>''estimate_changes.py:'' A script that takes a file generated by ''apply_postedits.py'' and scores sentences which were processed with postediting rules on a language model.</li>
 </ul>
+In addition, I have added cache function to learn_postedits.py (https://github.com/deltamachine/naive-automatic-postediting/blob/master/lib/explain2_cache.py):
 Also I have refactored and documented the old code from https://github.com/mlforcada/naive-automatic-postediting/blob/master/learn_postedits.py. A new version is stored in a given folder in ''cleaned_learn_postedits.py''

@@ Line 80: / Line 80: @@
 . The main problem is a pretty low speed of learn_postedits.py and apply_postedits.py. Repeating Apertium calls and big size of maximum source/target segment length make the process pretty slow. It makes extracting triplets from a big corpus very hard and using a small corpus is just useless.
-I have implemented cache function for learn_postedits.py (https://github.com/deltamachine/naive-automatic-postediting/blob/master/lib/explain2_cache.py): now instead of calling Apertium every time the program needs to translate/analyze any subsegment, it firstly checks, is this subsegment already stored in a database. If yes, it takes it from here, if no, it calls Apertium and then adds new information to the database.
+I have added cache function to learn_postedits.py (https://github.com/deltamachine/naive-automatic-postediting/blob/master/lib/explain2_cache.py): now instead of calling Apertium every time the program needs to translate/analyze any subsegment, it firstly checks, is this subsegment already stored in a database. If yes, it takes it from here, if no, it calls Apertium and then adds new information to the database.
 Results of running two versions of learn_postedits.py on 100 sentences corpus with different parameters:
@@ Line 118: / Line 118: @@
 . There might be problems with current alignment method because currently postediting operations look pretty strange even if they were extracted with high fuzzy match threshold. Alignment implementation should be checked carefully.
 . Some old code refactoring needs to be done.
 ==== Searching for extracted postediting operations that improve the quality of translation ====

@@ Line 150: / Line 150: @@
 == Reasons why Google and Apertium should sponsor it ==
-This toolbox might become a great way of improving language pairs by filling gaps in dictionaries and reducing the amount of human work at the same time. Even the released Apertium pairs are not perfect and sometimes do mistakes which can be easily fixed.
+This toolbox might become a great way of improving language pairs by filling gaps in dictionaries and reducing the amount of human work at the same time. Even the released Apertium pairs are not perfect and sometimes do mistakes that can be easily fixed.
 For example, Apertium translates Belarusian sentence ''"Нехта тут размаўляе па-руску?"'' ("Does somebody here speak Russian?") in Russian as ''"Кто-то здесь размаўляе по-русски?"'' when a correct translation would be ''"Кто-то здесь говорит по-русски?"''. The problem with this example is obvious: Apertium doesn't know the word "размаўлять". But it can be easily fixed with the methods described above (sections 5.2.3, 5.2.4).

@@ Line 60: / Line 60: @@
 <u>About language pair</u>
-I would like to work with languages I know or at least can more or less understand. Since I“m a native Russian speaker, it seems to be a good idea to work with bel-rus and ukr-rus. Though these pairs are already released, there are still a lot of work to be done. But the methods I“m going to develop won“t be tied to a language pair.
+I would like to work with languages I know or at least can more or less understand. Since I'm a native Russian speaker, it seems to be a good idea to work with bel-rus and ukr-rus. Though these pairs are already released, there are still a lot of work to be done. Also those are closely related languages so there will be less problems with alignment.
+But the methods I'm going to develop are not going to be tied to a language pair.
 <u>Abour collecting and processing data</u>
@@ Line 112: / Line 114: @@
 * Caching function for apply_postedits.py
-* Search for other ways we can improve the speed.
+* Searching for other ways of improving the speed.
 . There might be problems with current alignment method because currently postediting operations look pretty strange even if they were extracted with high fuzzy match threshold. Alignment implementation should be checked carefully.
@@ Line 118: / Line 120: @@
 . Some old code refactoring needs to be done.
-==== Search of extracted postediting operations that improve the quality of translation ====
+==== Searching for extracted postediting operations that improve the quality of translation ====
 The next step is to process a big train set using different parameters, extract potential postediting operations, apply them to the test set and find those that improve the quality of original Apertium translation on a regular basis.
-A language model score might be a criteria of a quality improvement.  A safety of a postediting operation might be determined by statistical methods.
+A language model score might be a criteria of quality improvement.  A safety of a postediting operation might be determined by statistical methods.
 ==== Classifying of successful postediting operations ====
@@ Line 139: / Line 141: @@
 For example, to identify potential bidix entries, we might choose a set of triplets from O such that s = mt for all (s, pe) from O.
-But it won't be enough for identifying potential lexical selection rules: we should carefully look at the context to find those.
+But it obviously won't be enough for identifying potential lexical selection rules: we should carefully look at the context to find those.
 ==== Creating tools for inserting useful information into a language pair ====
-The last step is to create tools for inserting useful information into a language pair. It might be some scripts which automatically create monodix/bidix entries or write rules based on a given data and its type and insert it in dictionaries.
+The last step is to create tools for inserting useful information into a language pair. It might be some scripts which automatically create monodix/bidix entries or write rules based on a given data and its type and insert it in dictionaries. Also it might be a new module in a pipeline. The final decision will depend on the results of the previous stage.
 == Reasons why Google and Apertium should sponsor it ==

@@ Line 35: / Line 35: @@
 == Why is it that you are interested in Apertium? ==
-I have participated in GSoC 2017 with Apertium and it was a great experience. I have successfully finished my project, learned a lot of new things and had a lot of fun. My programming and NLP skills became much better and I want to develop them more.
+I have participated in Google Summer of Code 2017 with Apertium and it was a great experience. I have successfully finished my project, learned a lot of new things and had a lot of fun. My programming and NLP skills became much better and I want to develop them more.
-Also I have participated in GCI 2017 as a mentor for Apertium and it was great too. So I am very interested in further contributing to Apertium.
+Also I have participated in Google Code-In 2017 as Apertium mentor and it was great too. So I am very interested in further contributing to Apertium.
-This organisation works on things which are very interesting for me as a computational linguist: (rule-based) machine translation, minority languages, NLP and so on. I love that Apertium works with minority languages because it is very important and is not a mainstream at the same time.
+This organisation works on things which are very interesting for me as a computational linguist: (rule-based) machine translation, minority languages, NLP and so on. I love that Apertium works with minority languages because it is very important for saving such languages.
 Also Apertium community is very friendly and open to new members, people here are always ready to help you. It encourages me to work with these people.