Difference between revisions of "Meta-evaluation"

From Apertium
Jump to navigation Jump to search
Line 1: Line 1:
 
Apertium language modules and translation pairs are subject to the following types of evaluation:
 
Apertium language modules and translation pairs are subject to the following types of evaluation:
   
  +
* Morphology coverage / regression testing
 
  +
* Size of system
  +
** Number of stems in lexc, monodix, bidix
  +
** Number of disambiguation rules
  +
** Number of lexical selection rules
  +
** Number of transfer rules
 
* Naïve coverage
 
* Naïve coverage
 
** Monolingual naïve coverage
 
** Monolingual naïve coverage

Revision as of 22:03, 30 May 2019

Apertium language modules and translation pairs are subject to the following types of evaluation:

  • Morphology coverage / regression testing
  • Size of system
    • Number of stems in lexc, monodix, bidix
    • Number of disambiguation rules
    • Number of lexical selection rules
    • Number of transfer rules
  • Naïve coverage
    • Monolingual naïve coverage
    • Trimmed naïve coverage (i.e., using a trimmed dictionary)
  • Accuracy of analyses
    • Precision/Recall/F-score
  • Accuracy of translation
    • WER/PER/BLEU
  • Clenliness of translation output
    • Testvoc