Meta-evaluation

Apertium language modules and translation pairs are subject to the following types of evaluation:

• Morphology coverage / regression testing
• Size of system
  • Number of stems in lexc, monodix, bidix
  • Number of disambiguation rules
  • Number of lexical selection rules
  • Number of transfer rules
• Naïve coverage
  • Monolingual naïve coverage
  • Trimmed naïve coverage (i.e., using a trimmed dictionary)
• Accuracy of analyses
  • Precision/Recall/F-score
• Accuracy of translation
  • WER/PER/BLEU
• Clenliness of translation output
  • Testvoc