Apertium has moved from SourceForge to GitHub.
If you have any questions, please come and talk to us on #apertium on irc.freenode.net or contact the GitHub migration team.

Comparison of part-of-speech tagging systems

From Apertium
(Difference between revisions)
Jump to: navigation, search
Line 4: Line 4:
   
 
In the following table, the intervals represent the [low, high] values from 10-fold cross validation.
 
In the following table, the intervals represent the [low, high] values from 10-fold cross validation.
  +
  +
{|class=wikitable
  +
!rowspan=3|System !!colspan=7|Language
  +
|-
  +
! Catalan !! Spanish !! Serbo-Croatian !! Russian !! Kazakh !! Portuguese !! Swedish
  +
|-
  +
! <small>24,144</small> !! <small>21,247</small> !! <small>20,128</small> !! <small>10,171</small> !! <small>4,348</small> !! <small>3,823 </small> !! <small>239</small>
  +
|-
  +
| '''1st''' || 81.85 || 86.18 || 75.22 ||75.63 || 80.79|| 72.54|| 72.90
  +
|-
  +
|}
   
 
{|class=wikitable
 
{|class=wikitable

Revision as of 14:50, 23 April 2016

Contents

Apertium would like to have really good part-of-speech tagging, but in many cases falls below the state-of-the-art (around 97% tagging accuracy). This page intends to collect a comparison of tagging systems in Apertium and give some ideas of what could be done to improve them.

In the following table, the intervals represent the [low, high] values from 10-fold cross validation.

System Language
Catalan Spanish Serbo-Croatian Russian Kazakh Portuguese Swedish
24,144 21,247 20,128 10,171 4,348 3,823 239
1st 81.85 86.18 75.22 75.63 80.79 72.54 72.90
Language Corpus System
Sent Tok Amb 1st CG+1st Unigram CG+Unigram apertium-tagger CG+apertium-tagger
Catalan 1,413 24,144  ? 81.85 83.96 [75.65, 78.46] [87.76, 90.48] [94.16, 96.28] [93.92, 96.16]
Spanish 1,271 21,247  ? 86.18 86.71 [78.20, 80.06] [87.72, 90.27] [90.15, 94.86] [91.84, 93.70]
Serbo-Croatian 1,190 20,128  ? 75.22 79.67 [75.36, 78.79] [75.36, 77.28]
Russian 451 10,171  ? 75.63 79.52 [70.49, 72.94] [74.68, 78.65] n/a n/a
Kazakh 403 4,348  ? 80.79 86.19 [84.36, 87.79] [85.56, 88.72] n/a n/a
Portuguese 119 3,823  ? 72.54 87.34 [77.10, 87.72] [84.05, 91.96]
Swedish 11 239  ? 72.90 73.86 [56.00, 82.97]

Sent = sentences, Tok = tokens, Amb = average ambiguity from the morphological analyser

Systems

  • 1st: Selects the first analysis from the morphological analyser
  • CG: Uses the CG (from the monolingual language package in languages) to preprocess the input.
  • Unigram: Lexicalised unigram tagger
  • apertium-tagger: Uses the bigram HMM tagger included with Apertium.

Corpora

The tagged corpora used in the experiments are found in the monolingual packages in languages, under the texts/ subdirectory.

Todo

Personal tools