Shallow syntactic function labeller/Workplan - Revision history

Deltamachine: /* Workplan */

2017-08-22T12:56:38Z

Workplan

Deltamachine at 12:55, 22 August 2017

2017-08-22T12:55:16Z

Deltamachine: /* Progress */

2017-08-15T07:33:20Z

Progress

Deltamachine: /* Progress */

2017-08-13T17:20:22Z

Progress

Deltamachine: /* Workplan */

2017-08-13T17:13:07Z

Workplan

Deltamachine: Created page with "A workplan and all progress notes about Shallow syntactic function labeller GSoC 2017 project. == Workplan == {|class=wikitable |- ! Week !! Dates !! To do ..."

2017-08-13T16:53:49Z

Created page with "A workplan and all progress notes about Shallow syntactic function labeller GSoC 2017 project. == Workplan == {|class=wikitable |- ! Week !! Dates !! To do ..."

New page

A workplan and all progress notes about [[Shallow syntactic function labeller]] GSoC 2017 project.

== Workplan ==

{|class=wikitable
|-
! Week !! Dates !! To do
|-
| 1 || 30th May — 5th June ||
* <s>Handling discrepancies between Apertium sme-nob and Sami corpus tagsets</s>
* <s>Writing a script for parsing Sami corpus</s>
|-
| 2
| 6th June — 12th June
| rowspan="2" align=left " | <s>Writing scripts for replacing UD tags with suitable Apertium tags and parsing UD-tree into a dataset for Kazakh, Breton and English UD dependency treebanks</s>

|-
| 3
| 13th June — 19th June

|-
| 4 || 20th June — 26th June || <s>Writing scripts for converting UD-treebanks (dev and test) of needed languages in Apertium stream format (converted treebanks will be useful for evaluating the quality of the labeller</s>)

|-
! '''First evaluation''' !! colspan="2" align=left |
Ready-to-use datasets
|-
| 5
| 27th June — 3rd July
| rowspan="2" align=left " | <s>Building and training the classifier</s>
|-
| 6
| 4th Jule — 10th July
|-
| 7 || 11th July — 17th July ||
<s>* Further training
* Working on improvements of the model</s>
|-
| 8 || 18th July — 24th July ||
<s>Working on improvements of the model</s>
|-
!'''Second evaluation''' || colspan="2" align=left |
Well-trained models
|-
| 9 || 25th July — 31th July ||
* Collecting all parts of the labeller together
* Adding machine-learned module instead of the syntax labelling part of sme-nob CG module to test it
|-
| 10 || 1st August — 7th August ||
* Adding machine-learned module instead of the syntax labelling part of sme-nob CG module to test it
|-
| 11 || 8th August — 14th August ||
* Testing
* Fixing bugs
|-
| 12 || 15th August — 21th August ||
* Cleaning up the code
* Writing documentation
|-
!'''Final evaluation''' || colspan="2" align=left |
The prototype shallow syntactic function labeller.
|-
|}

== Progress ==
'''Week 1:''' Datasets for North Sami were created.
* Some tags in the original corpus were replaced with Apertium North Sami tags (like here: https://victorio.uit.no/langtech/trunk/langs/sme/tools/mt/apertium/tagsets/modify-tags.regex).
* Some tags were removed from the original corpus as irrelevant: ABBR, ACR, Allegro, G3, G7, <ext>, Foc_, Qst.
* In cases when there were two lines with analysis for one word, only one analysis has been left.
* Information about derivation was removed too.
* Special "fake" syntactical functions were added for CLB and PUNCT: @CLB and @PUNCT.
* Two types of datasets were created: the first type contains tags for punctuation and clause boundaries and the second does not.

'''Weeks 2-3:''' Datasets for Kazakh, Breton and English were created.

NB: the datasets for North Sami and English seem to be pretty big, when Kazakh is comparably small and Breton is even smaller. But it gives us opportunity to check how many data will be enough for training the labeller and is it possible to achieve pretty good results having very small amount of data (like in case of Breton)
* All dependency treebanks were "flattened": words with the @conj and the @parataxis relation took the label of their head (https://github.com/deltamachine/wannabe_hackerman/blob/master/flatten_conllu.py).
* For all languages two types of datasets were created: the first type contains tags for punctuation and the second does not.
* ''Kazakh''
** some mistakes in conllu file were corrected
** double lines were removed
* ''English''
** double lines were removed
** all UD POS and features tags were replaced with Apertium tags
* ''Breton''
** some mistakes in conllu file were corrected
** double lines were removed
** all UD features tags were replaced with Apertium tags

'''Week 4:''' Scripts for converting Kazakh, Breton and English UD-treebanks in Apertium stream format were written

'''Weeks 5-6:'''
Two types of networks were built: a simple RNN network and an encoder-decoder network with attentive mechanism.
It seems that simple RNN shows better results in all our cases. Encoder-decoder network shows acceptable results only
on very big datasets, like English dataset, but in case of small corpus, like Breton, it is useless, and simple RNN can work with small datasets.

'''Weeks 7-8:'''
What new was created:
* New datasets (tokens + tags instead of just tags)
* Word2vec and fastText embeddings

Fasttext embeddings helped to improve accuracy of Kazakh model on 10% and accuracy of English model on 3%.
Current results: 76% for Breton, 78% for Kazakh, 82% for North Sami and 80% for English.
However, models are still need to be improved somehow.

@@ Line 57: / Line 57: @@
 * <s> Cleaning up the code </s>
 * <s> Writing documentation </s>
+|-
 | 13      || 22nd August — 29th August  ||
 * Testing

@@ Line 51: / Line 51: @@
 |-
 | 11      || 8th August — 14th August  ||
-* Testing
+* <s> Testing </s>
-* Fixing bugs
+* <s> Fixing bugs </s>
 |-
 | 12      || 15th August — 21th August  ||
-* Cleaning up the code
+* <s> Cleaning up the code </s>
-* Writing documentation
+* <s> Writing documentation </s>
+| 13      || 22nd August — 29th August  ||
+* Testing
+* Improving the perfomance of models
 |-
 !'''Final evaluation''' ||  colspan="2" align=left |
@@ Line 107: / Line 110: @@
 The labeller was built and successfully added instead of the original syntax module in sme-nob pipeline. Also the script for changing the modes was written.
 Everything seems to okay, though the translation may be not as good as with the original module. Perhaps this is because of some additional functionality of the original sme-nob.syn.rlx.bin.
+'''Weeks 11-12:'''
+The labeller was built and added instead of the original syntax module in kmr-eng pipeline. The testpack for two language pairs was built. All code was cleaned up, some docstrings were written. Also there is almost no trash in the github repository anymore.
 The main things to do:
+* Continue improving the perfomance of models.
-* <s>Add an ability to handle more than one sentence.</s>
 * Do more tests.
-* Write docstrings and refactore the main code.
+* Refactore the main code.

@@ Line 109: / Line 109: @@
 The main things to do:
-* Add an ability to handle more than one sentence.
+* <s>Add an ability to handle more than one sentence.</s>
 * Do more tests.
 * Write docstrings and refactore the main code.

@@ Line 103: / Line 103: @@
 Current results: 76% for Breton, 78% for Kazakh, 82% for North Sami and 80% for English.
 However, models are still need to be improved somehow.
+'''Weeks 9-10:'''
+The labeller was built and successfully added instead of the original syntax module in sme-nob pipeline. Also the script for changing the modes was written.
+Everything seems to okay, though the translation may be not as good as with the original module. Perhaps this is because of some additional functionality of the original sme-nob.syn.rlx.bin.
+The main things to do:
+* Add an ability to handle more than one sentence.
+* Do more tests.
+* Write docstrings and refactore the main code.

@@ Line 34: / Line 34: @@
 |-
 | 7     || 11th July — 17th July  ||
-<s>* Further training
+* <s> Further training
 * Working on improvements of the model</s>
 |-
@@ Line 44: / Line 44: @@
 |-
 | 9     || 25th July — 31th July ||
-* Collecting all parts of the labeller together
+* <s>Collecting all parts of the labeller together
-* Adding machine-learned module instead of the syntax labelling part of sme-nob CG module to test it
+* Adding machine-learned module instead of the syntax labelling part of sme-nob CG module to test it </s>
 |-
 | 10      || 1st August — 7th August   ||
-* Adding machine-learned module instead of the syntax labelling part of sme-nob CG module to test it
+* <s> Adding machine-learned module instead of the syntax labelling part of sme-nob CG module to test it </s>
 |-
 | 11      || 8th August — 14th August  ||