User:Francis Tyers/Sandbox

Lexical selection

Information

• Surface form -- tud etc.
• Lemma -- den etc.
• Category -- n.f etc.
• Syntax -- @SUBJ etc.

Ideas

For some things linguistic knowledge is better, or easier. It is also better for hacking. For other things, statistics are better. Wider coverage for cheaper. The lexical selection module(s) should allow both the use of rules and of statistics. Rules for things we "know", statistics for those we don't.

Inferring rules from collocations

Rules as described below are already used in apertium-cy-en, apertium-br-fr and apertium-sme-smj. This stage would be the first pass of lexical selection.

• The bilingual dictionary has several translations for each ambiguous word.
• Rules are created to select between them based on context.
• For each word in the bilingual dictionary, collocations (n-grams) are extracted from a source language corpus.
  • reisa þetta hús og fullgjöra
  • reisa þetta hús og fullgjöra
  • niður þetta hús Guðs í
  • gjört fyrir hús Guðs himnanna
  • inn í hús Semaja Delajasonar
• For each ambiguous word, these collocations are run with each of the entries in the bilingual dictionary through the translator.
• Translations are scored on a target language corpus.
• Where the difference in score between one translation and another reaches a threshold, a rule is created in the form of:
  • MAP (sense1) ("hús") IF (1 ("Guðs"));
• Syntax could also be included.
  • MAP (sense1) ("hús") IF (1 @SUBJ);
• It would be interesting to see if rules can be learnt which use different discriminators (e.g. surface form, syntax) etc.

Advantages

• Fairly straightforward -- the rules can be created automatically in constraint grammar.
• Human readable / editable.
• Doesn't require parallel corpus -- although might work better with one.
• Unsupervised.

Disadvantages

• Many rules will be slow.
• Might not work very well.

Relevant prior work

• Jin Yang (1999) "Towards the Automatic Acquisition of Lexical Selection Rules"
• Eckhard Bick (2005) "Dan2eng: Wide-Coverage Danish-English Machine Translation"

Examples

Pediñ can translate as 'prier' or 'inviter'. If it is used transitively it means "inviter", intransitively it means "prier"

• o huñvreal muioc'h eget o pediñ .
  • Leur *huñvreal plus que en train de prier .
• Koulskoude e tiviz Francis pediñ e zaou vreur d'ober ...
  • Pourtant il décide Francis prier ses deux frères à faire ...
• O fal a zo pediñ arzourien a bep seurt evel kizellerien
  • Leur objectif il est inviter des artistes de toute sorte comme les sculpteurs
• ... bleunioù ha peadra da yac'haat o zreid hag o pediñ evito ...
  • ... de fleurs et des moyens à guérir leurs pieds et en train de prier pour eux ...
• ha tu a oa bet d'al labourerien pediñ o familhoù hag o mignoned
  • ... et il y avait moyen été aux travailleurs prier leurs familles et leurs amis ...
• Raktresoù all a zo ivez : pediñ skrivagnerien a-benn eskemm ganto
  • ... de Projets autres il est aussi : inviter des écrivains pour échanger avec eux ...
• Sharon Stone eo bet an hini gwellañ evit pediñ an embregerien da zisammañ
  • *Sharon *Stone il a été les ceux le plus mieux pour prier les entrepreneurs à décharger ...