Difference between revisions of "Comparison of part-of-speech tagging systems"
Jump to navigation
Jump to search
Line 18: | Line 18: | ||
| Russian || 451 || 10,171 || ?|| 75.63 || 79.52 || [70.49, 72.94] || [74.68, 78.65] || n/a || n/a |
| Russian || 451 || 10,171 || ?|| 75.63 || 79.52 || [70.49, 72.94] || [74.68, 78.65] || n/a || n/a |
||
|- |
|- |
||
− | | Kazakh || 403 || 4,348 || ? || 80. |
+ | | Kazakh || 403 || 4,348 || ? || 80.79 || 86.19 || [84.36, 87.79] || [85.56, 88.72] || n/a || n/a |
|- |
|- |
||
| Portuguese || 119 || 3,823 || ? || 72.54 || 87.34 || [77.10, 87.72] || [84.05, 91.96] || || |
| Portuguese || 119 || 3,823 || ? || 72.54 || 87.34 || [77.10, 87.72] || [84.05, 91.96] || || |
||
Line 32: | Line 32: | ||
* <code>1st</code>: Selects the first analysis from the morphological analyser |
* <code>1st</code>: Selects the first analysis from the morphological analyser |
||
* <code>CG</code>: Uses the CG (from the monolingual language package in [[languages]]) to preprocess the input. |
* <code>CG</code>: Uses the CG (from the monolingual language package in [[languages]]) to preprocess the input. |
||
− | * <code>Unigram</code>: Lexicalised unigram tagger |
+ | * <code>Unigram</code>: Lexicalised [[unigram tagger]] |
* <code>apertium-tagger</code>: Uses the bigram HMM tagger included with Apertium. |
* <code>apertium-tagger</code>: Uses the bigram HMM tagger included with Apertium. |
||
Revision as of 15:19, 15 January 2016
Apertium would like to have really good part-of-speech tagging, but in many cases falls below the state-of-the-art (around 97% tagging accuracy). This page intends to collect a comparison of tagging systems in Apertium and give some ideas of what could be done to improve them.
In the following table, the intervals represent the [low, high] values from 10-fold cross validation.
Language | Corpus | System | |||||||
---|---|---|---|---|---|---|---|---|---|
Sent | Tok | Amb | 1st | CG+1st | Unigram | CG+Unigram | apertium-tagger | CG+apertium-tagger | |
Catalan | 1,413 | 24,144 | ? | 81.85 | 83.96 | [75.65, 78.46] | [87.76, 90.48] | [94.16, 96.28] | [93.92, 96.16] |
Spanish | 1,271 | 21,247 | ? | 86.18 | 86.71 | [78.20, 80.06] | [87.72, 90.27] | [90.15, 94.86] | [91.84, 93.70] |
Serbo-Croatian | 1,190 | 20,128 | ? | 75.22 | 79.67 | [75.36, 78.79] | [75.36, 77.28] | ||
Russian | 451 | 10,171 | ? | 75.63 | 79.52 | [70.49, 72.94] | [74.68, 78.65] | n/a | n/a |
Kazakh | 403 | 4,348 | ? | 80.79 | 86.19 | [84.36, 87.79] | [85.56, 88.72] | n/a | n/a |
Portuguese | 119 | 3,823 | ? | 72.54 | 87.34 | [77.10, 87.72] | [84.05, 91.96] | ||
Swedish | 11 | 239 | ? | 72.90 | 73.86 | [56.00, 82.97] |
Sent = sentences, Tok = tokens, Amb = average ambiguity from the morphological analyser
Systems
1st
: Selects the first analysis from the morphological analyserCG
: Uses the CG (from the monolingual language package in languages) to preprocess the input.Unigram
: Lexicalised unigram taggerapertium-tagger
: Uses the bigram HMM tagger included with Apertium.
Corpora
The tagged corpora used in the experiments are found in the monolingual packages in languages, under the texts/
subdirectory.
Todo
- Implement this tagger: https://spacy.io/blog/part-of-speech-POS-tagger-in-python