Difference between revisions of "Shallow syntactic function labeller"

From Apertium
Jump to navigation Jump to search
Line 88: Line 88:
 
== To do ==
 
== To do ==
 
* <s>Add an ability to handle more than one sentence.</s>
 
* <s>Add an ability to handle more than one sentence.</s>
  +
* <s>Write docstrings.</s>
 
* <s>Take the trash out of the github repository before the final evaluation.</s>
 
* Do more tests. MORE.
 
* Do more tests. MORE.
* Write docstrings and refactore the main code.
+
* Refactore the main code.
* Take the trash out of the github repository before the final evaluation.
 
 
* Continue improving the perfomance of the models.
 
* Continue improving the perfomance of the models.

Revision as of 12:44, 22 August 2017

This is Google Summer of Code 2017 project

A repository for the whole project: https://github.com/deltamachine/shallow_syntactic_function_labeller

A workplan and progress notes can be found here: Shallow syntactic function labeller/Workplan

Description

The shallow syntactic function labeller takes a string in Apertium stream format, parses it into a sequence of morphological tags and gives it to a classifier. The classifier is a simple RNN model trained on prepared datasets which were made from parsed syntax-labelled corpora (mostly UD-treebanks). The classifier analyzes the given sequence of morphological tags, gives a sequence of labels as an output and the labeller applies these labels to the original string.

Labeller in the pipeline

The labeller runs between morphological analyzer or disambiguator and pretransfer.

For example, in sme-nob it runs between sme-nob-disam and sme-nob-pretransfer, like an original syntax module.

... | cg-proc 'sme-nob.mor.rlx.bin' | python 'sme-nob-labeller.py' | apertium-pretransfer | lt-proc -b 'sme-nob.autobil.bin' | ...

Language pairs support

Currently the labeller works with following language pairs:

  • sme-nob: the labeller may fully replace the original syntax module (it doesn't have all the functionality of the original CG, but works pretty good anyway)
  • kmr-eng: may be tested in the pipeline, but the pair has only a few rules that look at syntax labels

Also there is all the needed data for Breton, Kazakh and English (https://github.com/deltamachine/shallow_syntactic_function_labeller/tree/master/models), but at this moment br-fr, kk-tat and en-ca just don't have syntax rules, so we can not test the labeller.

Labelling performance

The results of validating the labeller on the test set (accuracy = mean accuracy score on the test set).

Language Accuracy
North Sami 81,6%
Kurmanji 84%
Breton 79,7%
Kazakh 82,6%
English 79,8%

Installation

Prerequisites

1. Python libraries:

2. Precompiled language pairs which support the labeller (sme-nob, kmr-eng)

How to install a testpack

NB: currently the testpack contains syntax modules only for sme-nob and kmr-eng.

git clone https://github.com/deltamachine/sfl_testpack.git
cd sfl_testpack

Script setup.py adds all the needed files in language pair directory and changes all files with modes.

Arguments:

  • work_mode: -lb for installing the labeller and changing modes, -cg for backwarding changes and using the original syntax module (sme-nob.syn.rlx.bin or kmr-eng.prob) in the pipeline.
  • lang: -sme for installing/uninstalling the labeller only for sme-nob, -kmr - only for kmr-eng, -all - for both.

For example, this script will install the labeller and add it to the pipeline for both pairs:

python setup.py -lb -all

And this script will backward modes changes for sme-nob:

python setup.py -cg -sme

To do

  • Add an ability to handle more than one sentence.
  • Write docstrings.
  • Take the trash out of the github repository before the final evaluation.
  • Do more tests. MORE.
  • Refactore the main code.
  • Continue improving the perfomance of the models.