Difference between revisions of "Ideas for Google Summer of Code/Weighted transfer rules"
Jump to navigation
Jump to search
| Line 11: | Line 11: | ||
! ID !! Rule !! Input !! Output || Frequency |
! ID !! Rule !! Input !! Output || Frequency |
||
|- |
|- |
||
| − | | 1 || <math>x</math> de <math>y</math> → <math>x</math> <math>y</math> || memoría de traducción || translation memory || |
+ | | 1 || <math>x</math> de <math>y</math> → <math>x</math> <math>y</math> || memoría de traducción || translation memory || 90 |
|- |
|- |
||
| − | | 2 || <math>x</math> de <math>y</math> → <math>y</math>'s <math>x</math> || memoría de traducción || translation's memory || |
+ | | 2 || <math>x</math> de <math>y</math> → <math>y</math>'s <math>x</math> || memoría de traducción || translation's memory || 0 |
|- |
|- |
||
| − | | 3 || <math>x</math> de <math>y</math> → <math>x</math> of <math>y</math> || memoría de traducción || memory of translation || |
+ | | 3 || <math>x</math> de <math>y</math> → <math>x</math> of <math>y</math> || memoría de traducción || memory of translation || 0 |
|- |
|- |
||
|} |
|} |
||
| Line 21: | Line 21: | ||
So here we would have something like: |
So here we would have something like: |
||
| − | * Rule 1 (x=memoría, y=traducción, weight=) |
+ | * Rule 1 (x=memoría, y=traducción, weight=1.0) |
| − | * Rule 2 (x=memoría, y=traducción, weight=) |
+ | * Rule 2 (x=memoría, y=traducción, weight=0.0) |
| − | * Rule 3 (x=memoría, y=traducción, weight=) |
+ | * Rule 3 (x=memoría, y=traducción, weight=0.0) |
==Example== |
==Example== |
||
Revision as of 14:57, 9 February 2015
At the moment Apertium transfer rules are combinations of fixed-length patterns and actions. Conflicts are "solved" by selecting the first rule that matches. The idea of this task is to allow rules to conflict and to select the most adequate rules for an input using weights.
- Unlexicalised rule weights
- Depend only on the category of the input words.
- Lexicalised rules weights
- Depend on one or more of the lemmas of the input words. e.g.
| ID | Rule | Input | Output | Frequency |
|---|---|---|---|---|
| 1 | de → | memoría de traducción | translation memory | 90 |
| 2 | de → 's | memoría de traducción | translation's memory | 0 |
| 3 | de → of | memoría de traducción | memory of translation | 0 |
So here we would have something like:
- Rule 1 (x=memoría, y=traducción, weight=1.0)
- Rule 2 (x=memoría, y=traducción, weight=0.0)
- Rule 3 (x=memoría, y=traducción, weight=0.0)
Example
Transfer rules:
| ID | Rule | Input | Output |
|---|---|---|---|
| 1 | de → | memoria de traducción | translation memory |
| 2 | de → 's | la hermana de mi novia | my girlfriend's sister |
| 3 | de → of | el estado de la cuestión | the state of the art |
Training
- Take a big corpus
- For each sentence:
- Apply transfer rules
- For each possible combination of transfer rules
- Translate the sentence and score on language model
- Each sentence gets a count 1. This count is shared between the transfer rules.
- Example
| La canciller se reúne hoy con el presidente de EE UU para limar asperezas y preparar la cumbre del miércoles con Putin. | |||
| 1 1 | The chancellor gathers today with [the U.S. president] for mend fences and prepare [the Wednesday summit] with Putin. | -74.55 | 0.1093 |
| 2 1 | The chancellor gathers today with [the U.S.'s president] for mend fences and prepare [the Wednesday summit] with Putin. | -69.51 | 0.1019 |
| 3 1 | The chancellor gathers today with [the president of the U.S.] for mend fences and prepare [the Wednesday summit] with Putin. | -74.47 | 0.1091 |
| 1 2 | The chancellor gathers today with [the U.S. president] for mend fences and prepare [the Wednesday's summit] with Putin. | -75.02 | 0.1100 |
| 2 2 | The chancellor gathers today with [the U.S.'s president] for mend fences and prepare [the Wednesday's summit] with Putin. | -69.98 | 0.1026 |
| 3 2 | The chancellor gathers today with [the president of the U.S.] for mend fences and prepare [the Wednesday's summit] with Putin. | -74.94 | 0.1098 |
| 1 3 | The chancellor gathers today with [the U.S. president] for mend fences and prepare [the summit of the Wednesday] with Putin. | -82.88 | 0.1215 |
| 2 3 | The chancellor gathers today with [the U.S.'s president] for mend fences and prepare [the summit of the Wednesday] with Putin. | -77.84 | 0.1141 |
| 3 3 | The chancellor gathers today with [the president of the U.S.] for mend fences and prepare [the summit of the Wednesday] with Putin. | -82.80 | 0.1214 |
You can then feed the fractional counts to some supervised machine learning program to get appropriate weights.
Tasks
- Implement in C++ and integrate into Apertium.
