Difference between revisions of "Talk:Parallel corpus pruning"
Jump to navigation
Jump to search
(Removing all content from page) |
(stuff) |
||
| Line 1: | Line 1: | ||
| + | * One of the ways would be to discard phrases which can't be produced by the MT system (presumably in terms of lemma matches). |
||
| + | |||
| + | Surely the easiest way to determine this is to invoke the MT system? |
||
| + | |||
| + | Alternatively, apertium-transfer-tools has a mechanism for pruning alignments based on lemma matches (along with a means of specifying stop words) |
||
| + | |||
| + | * Look at the ratio of unaligned words. The higher the ratio, the more likely the translation is "freer" |
||
| + | |||
| + | It might be better to first consider P(e|f) of the POS alignments, before pruning based on ratio, to not discard lexicalised phrases. -- [[User:Jimregan|Jimregan]] 15:45, 18 March 2009 (UTC) |
||
Revision as of 15:45, 18 March 2009
- One of the ways would be to discard phrases which can't be produced by the MT system (presumably in terms of lemma matches).
Surely the easiest way to determine this is to invoke the MT system?
Alternatively, apertium-transfer-tools has a mechanism for pruning alignments based on lemma matches (along with a means of specifying stop words)
- Look at the ratio of unaligned words. The higher the ratio, the more likely the translation is "freer"
It might be better to first consider P(e|f) of the POS alignments, before pruning based on ratio, to not discard lexicalised phrases. -- Jimregan 15:45, 18 March 2009 (UTC)
