Talk:Constraint-based lexical selection module - Revision history

Ilnar.salimzyan: s/ure/sure/

2014-08-19T14:10:37Z

s/ure/sure/

Francis Tyers: /* Real training program */

2014-08-19T12:52:11Z

Real training program

Francis Tyers: /* Work plan (12th March 2012) */

2012-12-03T11:04:21Z

Work plan (12th March 2012)

Francis Tyers: /* Lists */

2012-09-13T19:57:03Z

Lists

Francis Tyers at 19:52, 13 September 2012

2012-09-13T19:52:27Z

Francis Tyers: /* Work plan (12th March 2012) */

2012-03-28T13:14:10Z

Work plan (12th March 2012)

Francis Tyers: /* Work plan (12th March 2012) */

2012-03-28T12:54:33Z

Work plan (12th March 2012)

Francis Tyers: /* Work plan (12th March 2012) */

2012-03-28T10:17:27Z

Work plan (12th March 2012)

Francis Tyers: /* Work plan (12th March 2012) */

2012-03-26T11:08:41Z

Work plan (12th March 2012)

Francis Tyers: /* Work plan (12th March 2012) */

2012-03-16T11:20:33Z

Work plan (12th March 2012)

@@ Line 5: / Line 5: @@
 Here is a suggestion of how lists should work.
-The list would be intepreted as a sequence of "match" tags, eg. when we compile, we just expand "list-items" to "match"s. This means that if we want it to be a list in the normal sense, we need to put it in an "or". So, it could also be used as a kind of templatey thing. But I'm not ure if this is overloading.
+The list would be intepreted as a sequence of "match" tags, eg. when we compile, we just expand "list-items" to "match"s. This means that if we want it to be a list in the normal sense, we need to put it in an "or". So, it could also be used as a kind of templatey thing. But I'm not sure if this is overloading.
 <pre>

@@ Line 369: / Line 369: @@
 ** Read biltrans output, make list of ambig words
 ** Translate + normalise + count.
+==Idea for scanning==
+<pre>
+<rule>
+  <match lemma="foo"/>
+  <any>
+    <match tags="adv"/>
+  </any>
+  <match lemma="bar"><select lemma="баз"/></match>
+</rule>
+</pre>
+Would make "foo adv* bar:баз"
+<pre>
+<rule>
+  <match lemma="foo"/>
+  <any>
+    <or><match tags="adv"/><match tags="adj"/></or>
+  </any>
+  <match lemma="bar"><select lemma="баз"/></match>
+</rule>
+</pre>
+Would make "foo (adv|adj)* bar:баз"

@@ Line 362: / Line 362: @@
 </pre>
+==Real training program==
-==Work plan (12th March 2012)==
+* Depend on lttoolbox + IRSTLM
-# Learn rules from a corpus where the best translation is selected by the TLM
+* Input: biltrans output + biltrans-to-multitrans + translation
-# Try changing the parameters of the LM
+* Process:
-## Remove stopwords
+** Read biltrans output, make list of ambig words
-## Rank on lemmata
+** Translate + normalise + count.
-# Try neighbouring word permutations, and ranking on LM
-## Remove stopwords
-## Rank on lemmata
-# Run tests on eu-es
-{|class=wikitable
-!            !! LER !! WER !! BLEU
-|-
-| TLM (s5)   || 248/603	41.1%    ||     ||
-|-
-| TLM (l5)   || 211/603	34.9%    ||     ||
-|-
-| TLM (n5)   || 271/603 44.9%    ||     ||
-|-
-| TLM (l5+flip2)   || 220/603 36.4%    ||     ||
-|-
-| TLM (ptes-s5)   || -/603 48.4%    ||     ||
-|-
-| TLM (fres-s5)   || 280/603 46.4%    ||     ||
-|-
-|}
-Programs to make:
-# Program that takes biltrans as output, generates all possible combos, translates, scores on language model takes the biltrans output with the highest LM score, and compares against a reference biltrans output.

@@ Line 12: / Line 12: @@
   <lists>
-    <list n="CUMPLIR-N">
+    <list n="cumplir-n">
       <list-item lemma="urte"/>
       <list-item lemma="agindu"/>
@@ Line 34: / Line 34: @@
   <rule>
     <or>
-      <match list="CUMPLIR-N"/>
+      <match list="cumplir-n"/>
     </or>
     <match lemma="ez"/>
@@ Line 44: / Line 44: @@
   <rule>
     <or>
-      <list v="CUMPLIR-N"/>
+      <match list="cumplir-n"/>
     </or>
     <match lemma="ez"/>

@@ Line 1: / Line 1: @@
 {{TOCD}}
+==Lists==
+Here is a suggestion of how lists should work.
+The list would be intepreted as a sequence of "match" tags, eg. when we compile, we just expand "list-items" to "match"s. This means that if we want it to be a list in the normal sense, we need to put it in an "or". So, it could also be used as a kind of templatey thing. But I'm not ure if this is overloading.
+<pre>
+<rules>
+  <lists>
+    <list n="CUMPLIR-N">
+      <list-item lemma="urte"/>
+      <list-item lemma="agindu"/>
+      <list-item lemma="amets"/>
+      <list-item lemma="arau"/>
+      <list-item lemma="asmo"/>
+      <list-item lemma="baldintza"/>
+      <list-item lemma="esan" tags="n"/>
+      <list-item lemma="eskubide"/>
+      <list-item lemma="helburu"/>
+      <list-item lemma="hitz"/>
+      <list-item lemma="irizpide"/>
+      <list-item lemma="lan"/>
+      <list-item lemma="lege"/>
+      <list-item lemma="obligazio"/>
+      <list-item lemma="proposamen"/>
+      <list-item lemma="erregelamendu"/>
+    </list>
+  </lists>
+  <rule>
+    <or>
+      <match list="CUMPLIR-N"/>
+    </or>
+    <match lemma="ez"/>
+    <match lemma="izan"/>
+    <match lemma="bete"><select lemma="cumplir"/></match>
+  <rule><!--SELECT ("cumplir" n) IF (-3 ("urte") OR CUMPLIR-N LINK 0 abs)(-2 ("ez"))(-1 ("izan")); -->
+  <rule>
+    <or>
+      <list v="CUMPLIR-N"/>
+    </or>
+    <match lemma="ez"/>
+    <match lemma="izan"/>
+    <match lemma="la"/>
+    <match lemma="bete"><select lemma="cumplir"/></match>
+  <rule><!--SELECT ("cumplir" n) IF (-3 ("urte") OR CUMPLIR-N LINK 0 abs)(-2 ("ez"))(-1 ("izan")); -->
+</rules>
+</pre>
 ==Rule formats==

@@ Line 329: / Line 329: @@
 |-
 | TLM (ptes-s5)   || 248/603 48.4%    ||     ||
+|-
+| TLM (fres-s5)   || 248/603 46.4%    ||     ||
 |-
 |}