Difference between revisions of "Ideas for Google Summer of Code/Apertium assimilation evaluation toolkit"
Jump to navigation
Jump to search
(Created page with 'Many Apertium language pairs are designed for assimilation (gisting) purposes. The evaluation described would measure how helpful they are in the task. Starting from files cont…') |
(No difference)
|
Revision as of 19:16, 7 March 2012
Many Apertium language pairs are designed for assimilation (gisting) purposes. The evaluation described would measure how helpful they are in the task.
Starting from files containing sentences in the source language and reference translations, generate tests for human evaluation consisting of:
- (optionally) the source sentence,
- (optionally) the machine-translated version of the source sentences and
- a reference translation of the sentence in which one or more content words have been deleted.
The idea is to measure how the ability of human subjects to fill in the holes improves when the source or a machine translation of it are presented. The task involves also generating a program that computes the success as a function of the information presented to the user, and utilities to make the whole process automatic given an Apertium language pair.
