Translation quality statistics

From Apertium
Revision as of 12:17, 19 April 2010 by Francis Tyers (talk | contribs)
Jump to navigation Jump to search

This page aims to give an overview of the quality of various translators available in the Apertium platform. Word Error Rate (WER) and Position-independent Word Error Rate (PWER) are measures of post-edition effort. The number gives the expected number of words needed to be corrected in 100 words of running text. So, a WER of 4.7% indicates that in a given 100 words of text, 4.7 of them will need to be corrected by the post-editor.

Precise numbers may vary due to differences in how sentences are selected to be evaluated. In some pairs, unknown words may be taken into account, in others not.

Translator Date Version Direction WER PWER BLEU Reference
apertium-nn-nb 12th October 2009 0.6.1 nn → nb - - - Unhammer and Trosterud, 2009 (two reference translations)
nb → nn 32.5%, 17.7% - 0.74
apertium-br-fr March 2010 0.2.0 br → fr 38 % 22 % - Tyers, 2010
fr → br - - -
apertium-sv-da 12th October 2009 0.5.0 sv → da 30.3 % 27.7 % - http://wiki.apertium.org/w/index.php?title=Swedish_and_Danish/Evaluation&oldid=14881
da → sv - - -
apertium-eu-es 2nd September 2009 eu → es 72.4 % 39.8 % - Ginestí-Rosell et al., 2009
es → eu - - -
apertium-cy-en 2nd January 2009 cy → en 55.7 % 30.5 % - Tyers and Donnelly, 2009
en → cy - - -
apertium-eo-en 08 May 2009 0.9.0 en → eo 21.0 % 19.0 % - http://wiki.apertium.org/w/index.php?title=English_and_Esperanto/Evaluation&oldid=12418
eo → en - - -
apertium-es-pt 15th May 2006 es → pt 4.7 % - - Armentano et al., 2006
pt → es 11.3 % - -
apertium-oc-ca 10th May 2006 oc → ca 9.6 % - - Armentano and Forcada, 2006
ca → oc - - -
apertium-pt-ca 28th July 2008 pt → ca 16.6% - - Armentano and Forcada, 2008
ca → pt 14.1% - -
apertium-en-es May 2009 en → es - - 0.1851 Sánchez-Martínez, 2009
es → en - - 0.1881


References