User:Deltamachine/proposal - Revision history

Deltamachine: Blanked the page

2018-03-23T09:56:07Z

Blanked the page

Show changes

Deltamachine: /* Which of the published tasks are you interested in? What do you plan to do? */

2017-04-03T13:46:36Z

Which of the published tasks are you interested in? What do you plan to do?

Deltamachine at 13:02, 3 April 2017

2017-04-03T13:02:10Z

Deltamachine at 12:35, 3 April 2017

2017-04-03T12:35:29Z

Deltamachine at 10:10, 3 April 2017

2017-04-03T10:10:31Z

Deltamachine: /* Reasons why Google and Apertium should sponsor it */

2017-04-02T07:30:28Z

Reasons why Google and Apertium should sponsor it

Deltamachine at 23:38, 1 April 2017

2017-04-01T23:38:39Z

Deltamachine at 22:55, 1 April 2017

2017-04-01T22:55:21Z

Deltamachine: /* Community bonding period */

2017-04-01T19:33:29Z

Community bonding period

Deltamachine: /* Reasons why Google and Apertium should sponsor it */

2017-04-01T11:43:52Z

Reasons why Google and Apertium should sponsor it

@@ Line 61: / Line 61: @@
 |}
-For each language will be created its own model, moreover, models will be trained just on sequences of morphological tags, not, for example, on tokens + tags. It means that models should not be overweight, so it will not slow down the entire workflow.
+For each language will be created its own model, moreover, models will be trained just on sequences of morphological tags, not, for example, on tokens + tags. It means that models should not be overweight, so they will not slow down the entire workflow.
 The classifier analyzes the given sequence of morphological tags, gives a sequence of labels as an output and the labeller applies these labels to the original string. The result could look like this:

@@ Line 24: / Line 24: @@
 == Why is it you are interested in machine translation? ==
-I am deeply interested in machine translation, because it combines my two most favourite fields of studies - linguistics and programming. As a computational linguist, I would like to know how machine translation systems are builded, how they work with language material and how we can improve results of their work. So, on the one hand, I can learn a lot of new things about structures of different languages while working with machine translation system like Apertium. On the other hand, I can significantly improve my coding skills, learn more about natural language processing and create something great and useful.
+I am deeply interested in machine translation, because it combines my two most favourite fields of studies - linguistics and programming. As a computational linguist, I would like to know how machine translation systems are built, how they work with language material and how we can improve results of their work. So, on the one hand, I can learn a lot of new things about structures of different languages while working with machine translation system like Apertium. On the other hand, I can significantly improve my coding skills, learn more about natural language processing and create something great and useful.
 == Why is it that you are interested in Apertium? ==
 There are three main reasons why I want to work with Apertium:
 <p>1. Apertium works with a lot of minority languages, which is great, because it is quite unusual for machine translation system. There are a lot of systems which can translate from English to German well enough, but there are very few which can translate, for example, from Kazakh to Tatar. Apertium is one of the said systems, and I believe they do a very important job.</p>
-<p>2. Apertium does rule-based machine translation which is unusual too. But as a linguist I am very curious about learning more about this approach, because rule-based translation requires close working with language structure and a big amount of language data.</p>
+<p>2. Apertium does rule-based machine translation which is unusual too. But as a linguist I am very curious about learning more about this approach, because rule-based translation requires close work with language structure and a big amount of language data.</p>
 <p>3. Apertium community is very friendly, helpful, responsive and open to new members, which is very attractive.</p>
@@ Line 61: / Line 61: @@
 |}
-For each language will be created its own model, moreover, models will be trained just on sequences of morphological tags, not, for example, on tokens + tags. It means that models should not be overweight and will not slow down the entire workflow.
+For each language will be created its own model, moreover, models will be trained just on sequences of morphological tags, not, for example, on tokens + tags. It means that models should not be overweight, so it will not slow down the entire workflow.
 The classifier analyzes the given sequence of morphological tags, gives a sequence of labels as an output and the labeller applies these labels to the original string. The result could look like this:

@@ Line 1: / Line 1: @@
 == Contact information ==
-<p>'''Name:''' Anna Kondratjeva</p>
+<p>'''Name:''' Anna Kondrateva</p>
 <p>'''Location:''' Moscow, Russia</p>
 <p>'''E-mail:''' an-an-kondratjeva@yandex.ru</p>
@@ Line 28: / Line 28: @@
 == Why is it that you are interested in Apertium? ==
 There are three main reasons why I want to work with Apertium:
-<p>1. Apertium works with a lot of minority languages, which is great, because it is quite unusual for machine translation system. There are a lot of systems, which can translate from English to German well enough, but there are very few, which can translate, for example, from Kazakh to Tatar. Apertium is one of the said systems, and I believe they do a very important job.</p>
+<p>1. Apertium works with a lot of minority languages, which is great, because it is quite unusual for machine translation system. There are a lot of systems which can translate from English to German well enough, but there are very few which can translate, for example, from Kazakh to Tatar. Apertium is one of the said systems, and I believe they do a very important job.</p>
-<p>2. Apertium does rule-based machine translation, which is unusual too. But as a linguist I am very curious about learning more about this approach, because rule-based translation requires close working with language structure and a big amount of language data.</p>
+<p>2. Apertium does rule-based machine translation which is unusual too. But as a linguist I am very curious about learning more about this approach, because rule-based translation requires close working with language structure and a big amount of language data.</p>
 <p>3. Apertium community is very friendly, helpful, responsive and open to new members, which is very attractive.</p>
@@ Line 42: / Line 42: @@
 '''A brief concept:'''
-The shallow syntactic function labeller takes a string in Apertium stream format, parses it into a sequence of morphological tags and gives it to a classifier. The classifier is a sequence-to-sequence model trained on prepared datasets, which were made from parsed UD-treebanks.
+The shallow syntactic function labeller takes a string in Apertium stream format, parses it into a sequence of morphological tags and gives it to a classifier. The classifier is a sequence-to-sequence model trained on prepared datasets, which were made from parsed syntax-labelled corpora (for instance, UD-treebanks).
 The dataset for an encoder contains sequences of morphological tags, the dataset for a decoder contains sequences of labels, in both cases one sequence is a one sentence. UD-tags in datasets are replaced with suitable tags from Apertium tagset. Here is an example of this transformation:
@@ Line 77: / Line 77: @@
 '''Integration into Apertium pipeline:'''
-As it said on the page of GSoC ideas, the main task is only to create a tool, which could be adapted and used for some language pairs after Google Summer of Code.
+As it has been said on the page of Google Summer of Code ideas, the main task is only to create a tool, which could be adapted and used for some language pairs after Google Summer of Code.
-However, it seems that we are able to test this approach during the summer work. We may adopt the labeller for North Sámi → Norwegian Bokmål language pair, which already works with syntactic labelling and then measure, how well it works. In the North Sámi → Norwegian Bokmål pipeline morphological disambiguation and syntax labelling were run as one CG module. Now they are split into two different parts (mor.rlx.bin and syn.rlx.bin), so we can try to replace the syntax labelling part with our machine-learned module and then test it.
+However, it seems that we are able to test this approach during the summer work. We may adapt the labeller for North Sámi - Norwegian Bokmål language pair, which already works with syntactic labelling, and then measure, how well it works. In the North Sámi - Norwegian Bokmål pipeline morphological disambiguation and syntax labelling were run as one CG module. Now they are split into two different parts (mor.rlx.bin and syn.rlx.bin), so we can try to replace the syntax labelling part with our machine-learned module and then test it.
 == Reasons why Google and Apertium should sponsor it ==

@@ Line 75: / Line 75: @@
 The task can be done with Tensorflow, but we may need a library, which is not so complex and has a simple runtime. The idea also can be realised with Keras (it has seq2seq add-on) and Theano as a backend, these libraries are not as massive as Tensorflow, which is usually being used for creating sequence-to-sequence models, so the labeller should work comparably fast. Moreover, Keras/Theano model can be run in regular hardware.
+'''Integration into Apertium pipeline:'''
+As it said on the page of GSoC ideas, the main task is only to create a tool, which could be adapted and used for some language pairs after Google Summer of Code.
+However, it seems that we are able to test this approach during the summer work. We may adopt the labeller for North Sámi → Norwegian Bokmål language pair, which already works with syntactic labelling and then measure, how well it works. In the North Sámi → Norwegian Bokmål pipeline morphological disambiguation and syntax labelling were run as one CG module. Now they are split into two different parts (mor.rlx.bin and syn.rlx.bin), so we can try to replace the syntax labelling part with our machine-learned module and then test it.
 == Reasons why Google and Apertium should sponsor it ==
@@ Line 122: / Line 127: @@
 <li>Setting up Linux and getting used to it</li>
 <li>Learning more about machine learning, looking for more researches about sequence-to-sequence models</li>
-<li>Learning more about UD treebanks</li>
+<li>Learning more about UD/VISL treebanks and tagsets and North Sámi syntax-labelled corpus</li>
 </ul>
 === Community bonding period ===
 <ul>
-<li>Choosing language pairs, with which shallow function labeller will work. Currently I am thinking about Basque, English, Russian/Finnish, maybe Spanish, but it needs to be discussed. Also we can try to realise the idea for North Sámi → Norwegian Bokmål language pair, which already uses syntactic labelling, but in this case UD syntactic tags in datasets for these languages should be replaced with VISL tags.</li>
+<li>Choosing language pairs, with which shallow function labeller will work. Currently I am thinking about Basque, English, Russian/Finnish, maybe Spanish, but it needs to be discussed.
+Also I will create a module for North Sámi → Norwegian Bokmål language pair, which already uses syntactic labelling, in order to evaluate the quality of the prototype.</li>
 <li>Choosing the most suitable Python ML library</li>
-<li>Thinking about how to integrate the classifier into Apertium
+<li>Thinking about how to integrate the classifier into North Sámi → Norwegian Bokmål pipeline
-<li>Learning more about possible problems</li>
+<li>Learning more about possible problems, especially about discrepancies between all needed tagsets</li>
 </ul>
@@ Line 138: / Line 144: @@
 <p></p>
 <li>'''Week 1:''' writing a script for parsing UD-treebanks</li>
-<li>'''Week 2:''' comparing UD and Apertium tagsets, writing a script for replacing UD tags with suitable Apertium tags</li>
+<li>'''Week 2:''' writing a script for parsing North Sámi syntax-labelled corpus</li>
+<li>'''Week 3:''' comparing UD and Apertium tagsets, writing a script for replacing UD tags with suitable Apertium tags, writing scripts for handling other possible discrepancies between all needed tagsets</li>
-<li>'''Week 3:''' creating datasets (in a few possible variants)</li>
-<li>'''Week 4:''' writing a script for parsing a string in Apertium stream format into a sequence of morphological tags</li>
+<li>'''Week 4:''' creating datasets (in a few possible variants), writing a script for parsing a string in Apertium stream format into a sequence of morphological tags</li>
 <li>'''Deliverable #1, June 26 - 30'''</li>
 <p></p>
@@ Line 153: / Line 159: @@
 <p></p>
-==== Part 3, weeks 9-12: integrating the labeller into Apertium ====
+==== Part 3, weeks 9-12: testing the labeller on North Sámi → Norwegian Bokmål language pair ====
 <p></p>
-<li>'''Week 9:''' collecting all parts of the labeller together</li>
+<li>'''Week 9:''' collecting all parts of the labeller together, adding machine-learned module instead of the syntax labelling part of CG module</li>
-<li>'''Week 10:''' integrating the labeller into Apertium</li>
+<li>'''Week 10:''' adding machine-learned module instead of the syntax labelling part of CG module</li>
-<li>'''Week 11:''' integrating the labeller into Apertium: testing, fixing bugs</li>
+<li>'''Week 11:''' testing, fixing bugs</li>
 <li>'''Week 12:''' cleaning up the code, writing documentation</li>
 <li>'''Project completed:''' the prototype shallow syntactic function labeller, which is able to label sentences well enough and works with several languages.</li>

@@ Line 99: / Line 99: @@
 |}
-The information about syntactic functions of words could help to make the translation better: for example, we could translate '''<abs><nsubj>''' with English nominative and '''<abs><obj>''' with English instrumentative.
+If we would know the information about syntactic function of word in absolutive case, we could change the word order in translated English sentence and get the better translation.
 </li>

@@ Line 70: / Line 70: @@
 So, in the end of the work there will be:
 <ul>
-<li>The labeller itself, which parse the string, restores a model for a needed language from a file, gives a sequence of tags to the model, gets a sequence of labels as an output and applies these labels to the original string</li>
+<li>The labeller itself, which parses the string, restores a model for a needed language from a file, gives a sequence of tags to the model, gets a sequence of labels as an output and applies these labels to the original string</li>
 <li>Files with trained models, which are saved in a suitable format (it could be, for example, JSON)</li>
 </ul>
@@ Line 81: / Line 81: @@
 A few examples of such problems:
 <ul>
-<li>When you are working with an ergative language, it may be useful to know, if an absolutive is subject or object.</li>
+<li>When you are working with an ergative language, it may be useful to know, if an absolutive is subject or object. Here is an example from Basque - English:
+{| class = "wikitable" style = "background-color: white;"
+|-
+|'''Basque'''
+|'''Current Apertium translation'''
+|'''English'''
+|-
+|Otsoa'''<abs><nsubj>''' etorri da
+|The wolf he has come
+|The wolf has come
+|-
+|Ehiztariak'''<erg><nsubj>''' otsoa'''<abs><obj>''' harrapatu du
+|The hunter the wolf he has caught
+|The hunter has caught the wolf // The wolf was caught by the hunter
+|-
+|}
+The information about syntactic functions of words could help to make the translation better: for example, we could translate '''<abs><nsubj>''' with English nominative and '''<abs><obj>''' with English instrumentative.
+</li>
 <li>There may be cases like classical Russian example "Мать любит дочь", which equally could mean "Mother loves daughter" or "Daughter loves mother". Machine translation systems always prefer the first variant, but due to the comparably free word order in Russian the meaning actually depends on syntactic functions of words.</li>
 <li>Also there are a lot of cases when case translation is ambiguous and it could be really helpful for disambiguation to know the syntactic function of the word. Russian dative can be translated with English dative or with English nominative, but the choice depends on the syntactic function of the word. For example, "дай '''мне<dat><iobj>''' ручку" should be translated as "give '''me''' the pen", but "что '''мне<dat><obl>''' делать" should be translated as "what should '''I''' do". </li>