Constraint-based lexical selection module - Revision history

Firespeaker: /* Sequences */

2023-04-11T15:26:24Z

Sequences

Firespeaker: /* Matching an unknown word */

2023-04-08T04:12:29Z

Matching an unknown word

Hectoralos: /* Macros */

2022-07-07T06:33:09Z

Macros

Hectoralos: /* Operator REPEAT */

2022-07-07T05:57:22Z

Operator REPEAT

Hectoralos: Macros

2022-07-07T03:51:56Z

Macros

Hectoralos: /* Operator REPEAT */

2022-07-06T20:08:21Z

Operator REPEAT

Hectoralos: /* Sequences */

2022-07-06T19:43:40Z

Sequences

Hectoralos: Operator OR, sequences

2022-07-06T19:19:26Z

Operator OR, sequences

Hectoralos: /* Matching an unknown word */

2022-07-06T19:07:04Z

Matching an unknown word

Hectoralos: /* Rule format */

2022-07-06T19:03:52Z

Rule format

@@ Line 211: / Line 211: @@
     <match lemma="passat" tags="adj.*"><select lemma="dernier"/></match>
   </rule>
+</pre>
+Note that if you add a <code>&lt;def-seqs&gt;</code> section and you had only a <code>&lt;rules&gt;</code> section already, then you'll need to put both inside of an <code>&lt;lrx&gt;</code> section:
+<pre>
+  <lrx>
+    <def-seqs>
+    ...
+    </def-seqs>
+    <rules>
+    ...
+    </rules>
+  </lrx>
 </pre>

@@ Line 406: / Line 406: @@
   </rule>
 </pre>
+=== Some more new stuff ===
+* [https://github.com/apertium/apertium-lex-tools/commit/8f5493f28d944c9c1591e3449f6df8c4718ab2c6 contains]
+* [https://github.com/apertium/apertium-lex-tools/commit/371b3740e74c27a2e6ff4278ce9286ee9e0b2319 suffix]
+* [https://github.com/apertium/apertium-lex-tools/pull/92 <code>glob="star"</code>]
 ==Writing and generating rules==

@@ Line 318: / Line 318: @@
 ===Macros===
+A macro is a set of rules for a common purpose that can be used for several words. For instance, quite often a verb has a different translation if it is pronominal or not, or if it is transitive or not.
-(will continue)
+Let's take as a example the Occitan verb "recordar" that is usually translated into French as "rappeler" ("remind"), but as a pronominal verb it would be "(se) souvenir" ("remember"). The problem is that to recognise a pronominal context one needs quite a few rules to prove that there is a personal unstressed pronoun before (or after) the verb and that it has the same person and number as the verb. So a macro could be created like:
+<pre>
+    <!-- p1 = verb oci, p2 = verb fra no pron, p3 = verb fra pron -->
+    <def-macro n="verb_nopron_pron" npar="3">
+      <rule weight="0.8">
+        <match plemma="1" tags="vblex.*"><select plemma="2"/></match>
+      </rule>
+      <rule weight="1.0">
+        <match tags="prn.pro.p1.*.sg"/>
+        <match plemma="1" tags="vblex.*.p1.sg"><select plemma="3"/></match>
+      </rule>
+      <rule weight="1.0">
+        <match tags="prn.pro.p1.*.sg"/>
+        <match tags="prn.pro.*"/>
+        <match plemma="1" tags="vblex.*.p1.sg"><select plemma="3"/></match>
+      </rule>
+      <rule weight="1.0">
+        <match plemma="1" tags="vblex.*.p1.sg"><select plemma="3"/></match>
+        <match tags="prn.enc.p1.*.sg"/>
+      </rule>
+      <rule weight="1.0">
+        <match tags="prn.pro.p2.*.sg"/>
+        <match plemma="1" tags="vblex.*.p2.sg"><select plemma="3"/></match>
+      </rule>
+      <rule weight="1.0">
+        <match tags="prn.pro.p2.*.sg"/>
+        <match tags="prn.pro.*"/>
+        <match plemma="1" tags="vblex.*.p2.sg"><select plemma="3"/></match>
+      </rule>
+      <rule weight="1.0">
+        <match plemma="1" tags="vblex.*.p2.sg"><select plemma="3"/></match>
+        <match tags="prn.enc.p2.*.sg"/>
+      </rule>
+    </def-macro>
+</pre>
+The call to the rule should be:
+<pre>
+  <macro n="verb_nopron_pron"><with-param v="recordar"/><with-param v="rappeler"/><with-param v="souvenir"/></macro>
+</pre>
+For other verbs a call to the same macro is sufficient. The code is much more readable and maintainable than without macros.
 ===Special cases===

@@ Line 230: / Line 230: @@
 </pre>
-But this will not word if we have an adjective that follows "còrn" (usually adjectives follow the nouns in Occitan). We could add a rule like:
+But this will not match if we have an adjective that follows "còrn" (usually adjectives follow the nouns in Occitan). We could add a rule like:
 <pre>
@@ Line 243: / Line 243: @@
 </pre>
-But using the operator REPEAT we can have a more compact way just expanding rule 2:
+Using the operator REPEAT we can have a more compact way just expanding rule 2:
 <pre>
@@ Line 258: / Line 258: @@
 </pre>
-Note that now we are even accepting two adjectives after "còrn" instead of only one.
+Note that now we are even accepting two adjectives after "còrn" instead of only one (without adding a fourth rule for dealing with two adjectives).
 And, if we think that horn can be not only big, but also "very big", we can improve the rule this way:
@@ Line 277: / Line 277: @@
   </rule>
 </pre>
+Next, a second REPEAT block could be added between the preposition "de" and the sequence to deal with the possible existence of determiners, adjectives, etc.
 ====REPEAT hack====
@@ Line 312: / Line 314: @@
 </pre>
-Rule 2 selects "cousin" if, at most, after four words there is a family word. Rule 3 does the same, but looking at up to 4 words in front. Note the OR operator within the REPEAT: <i><nowiki><match tags="*"/></nowiki></i> matches any known word (i.e. that gets a morphological analysis), while <i><nowiki><match tags=""/></nowiki></i> matches unknown words (i.e. without any morphological tag).
+Rule 2 selects "cousin" if, at most, after four words there is a family word. Rule 3 does the same, but looking at up to 4 words in front. Note the OR operator within the REPEAT: <i><nowiki><match tags="*"/></nowiki></i> matches any known word (i.e. that gets a morphological analysis), while <i><nowiki><match tags=""/></nowiki></i> matches unknown words (i.e. without any morphological tag). Without the OR operation, the rules would try to match precisely a sequence of one unknown word followed by one known one.
 ===Macros===

@@ Line 280: / Line 280: @@
 ====REPEAT hack====
-Sometimes, a lexical selection has unclear rules. For instance the Occitan noun "cosina" may be a (female) cousin or a kitchen. We can decide that the latter is the most usual translation, so it will be the default. On the other hand, we will select "cousin" if there is another parent term nearby, such as "father", "mother" or "brother". For this we can do something like:
+Sometimes, a lexical selection has unclear rules. For instance the Occitan noun "cosina" may be "(female) cousin" or "kitchen". We can decide that the latter is the most usual translation, so it will be the default. On the other hand, we will select "cousin" if there is another parent term nearby, such as "father", "mother" or "brother". For this we can do something like:
 <pre>
@@ Line 312: / Line 312: @@
 </pre>
-Rule 2 selects "cousin" if, at most, after four words there is a family word. Rule 3 does the same, but looking at up to 4 words in front. Note the OR operator within the REPEAT: <i>match tags="*"</i> matches any known word (i.e. that gets a morphological analysis), while <i>match tags=""</i> matches unknown words (i.e. without any morphological tag).
+Rule 2 selects "cousin" if, at most, after four words there is a family word. Rule 3 does the same, but looking at up to 4 words in front. Note the OR operator within the REPEAT: <i><nowiki><match tags="*"/></nowiki></i> matches any known word (i.e. that gets a morphological analysis), while <i><nowiki><match tags=""/></nowiki></i> matches unknown words (i.e. without any morphological tag).
+===Macros===
+(will continue)
 ===Special cases===

@@ Line 277: / Line 277: @@
   </rule>
 </pre>
+====REPEAT hack====
+Sometimes, a lexical selection has unclear rules. For instance the Occitan noun "cosina" may be a (female) cousin or a kitchen. We can decide that the latter is the most usual translation, so it will be the default. On the other hand, we will select "cousin" if there is another parent term nearby, such as "father", "mother" or "brother". For this we can do something like:
+<pre>
+  <rule weight="0.8">
+    <match lemma="cosina" tags="n.*"><select lemma="kitchen"/></match>
+  </rule>
+  <rule weight="1.0" >
+    <match lemma="cosina" tags="n.*"><select lemma="cousin"/></match>
+    <repeat from="0" upto="4">
+      <or>
+        <match tags=""/>
+        <match tags="*"/>
+      </or>
+    </repeat>
+    <or>
+      <seq n="familia"/>
+    </or>
+  </rule>
+  <rule weight="1.0" >
+    <or>
+      <seq n="familia"/>
+    </or>
+    <repeat from="0" upto="4">
+      <or>
+        <match tags=""/>
+        <match tags="*"/>
+      </or>
+    </repeat>
+    <match lemma="cosina" tags="n.*"><select lemma="cousin"/></match>
+  </rule>
+</pre>
+Rule 2 selects "cousin" if, at most, after four words there is a family word. Rule 3 does the same, but looking at up to 4 words in front. Note the OR operator within the REPEAT: <i>match tags="*"</i> matches any known word (i.e. that gets a morphological analysis), while <i>match tags=""</i> matches unknown words (i.e. without any morphological tag).
 ===Special cases===

@@ Line 210: / Line 210: @@
     </or>
     <match lemma="passat" tags="adj.*"><select lemma="dernier"/></match>
+  </rule>
+</pre>
+===Operator REPEAT===
+Imagine the translation of the Occitan word "còrn" that may be "corner" or "horn" (of an animal). We could have as a first version:
+<pre>
+  <rule weight="0.8">
+    <match lemma="còrn" tags="n.*"><select lemma="corner"/></match>
+  </rule>
+  <rule weight="1.0" >
+    <match lemma="còrn" tags="n.*"><select lemma="horn"/></match>
+    <match lemma="de" tags="pr"/>
+    <or>
+      <seq n="animals"/>
+    </or>
+  </rule>
+</pre>
+But this will not word if we have an adjective that follows "còrn" (usually adjectives follow the nouns in Occitan). We could add a rule like:
+<pre>
+  <rule weight="1.0" >
+    <match lemma="còrn" tags="n.*"><select lemma="horn"/></match>
+    <match tags="adj.*"/>
+    <match lemma="de" tags="pr"/>
+    <or>
+      <seq n="animals"/>
+    </or>
+  </rule>
+</pre>
+But using the operator REPEAT we can have a more compact way just expanding rule 2:
+<pre>
+  <rule weight="1.0" >
+    <match lemma="còrn" tags="n.*"><select lemma="horn"/></match>
+    <repeat from="0" upto="2">
+      <match tags="adj.*"/>
+    </repeat>
+    <match lemma="de" tags="pr"/>
+    <or>
+      <seq n="animals"/>
+    </or>
+  </rule>
+</pre>
+Note that now we are even accepting two adjectives after "còrn" instead of only one.
+And, if we think that horn can be not only big, but also "very big", we can improve the rule this way:
+<pre>
+  <rule weight="1.0" >
+    <match lemma="còrn" tags="n.*"><select lemma="horn"/></match>
+    <repeat from="0" upto="3">
+      <or>
+        <match tags="adv"/>
+        <match tags="adj.*"/>
+      </or>
+    </repeat>
+    <match lemma="de" tags="pr"/>
+    <or>
+      <seq n="animals"/>
+    </or>
   </rule>
 </pre>

@@ Line 125: / Line 125: @@
 If we have "un animal ferotge", the translation "farouche" will get 0.8 points, and "féroce" will get 1.0. The latter will be chosen.
+===Operator OR===
+The boolean operator OR can be used, as shown in the previous example:
+<pre>
+  <rule weight="1.0">
+    <or>
+      <match lemma="animal" tags="n.*"/>
+      <match lemma="animau" tags="n.*"/>
+    </or>
+    <match lemma="ferotge" tags="adj.*"><select lemma="féroce"/></match>
+  </rule>
+</pre>
+===Sequences===
+Often, the same words are used in OR's. For readability and maintainability, they can be defined in a special sequence bloc, for instance:
+<pre>
+  <def-seqs>
+    <def-seq n="jorns"><or>
+      <match lemma="diluns" tags="n.*"/>
+      <match lemma="dimars" tags="n.*"/>
+      <match lemma="dimècres" tags="n.*"/>
+      <match lemma="dimèrcs" tags="n.*"/>
+      <match lemma="dijòus" tags="n.*"/>
+      <match lemma="dijaus" tags="n.*"/>
+      <match lemma="divendres" tags="n.*"/>
+      <match lemma="divés" tags="n.*"/>
+      <match lemma="dissabte" tags="n.*"/>
+      <match lemma="dimenge" tags="n.*"/>
+    </or></def-seq>
+    <def-seq n="meses"><or>
+      <match lemma="genèr" tags="n.*"/>
+      <match lemma="genièr" tags="n.*"/>
+      <match lemma="janvièr" tags="n.*"/>
+      <match lemma="gèr" tags="n.*"/>
+      <match lemma="febrièr" tags="n.*"/>
+      <match lemma="heurèr" tags="n.*"/>
+      <match lemma="hrevèr" tags="n.*"/>
+      <match lemma="herevèr" tags="n.*"/>
+      <match lemma="herbèr" tags="n.*"/>
+      <match lemma="hiurèr" tags="n.*"/>
+      <match lemma="març" tags="n.*"/>
+      <match lemma="abrial" tags="n.*"/>
+      <match lemma="abril" tags="n.*"/>
+      <match lemma="abriu" tags="n.*"/>
+      <match lemma="abrieu" tags="n.*"/>
+      <match lemma="mai" tags="n.*"/>
+      <match lemma="junh" tags="n.*"/>
+      <match lemma="julh" tags="n.*"/>
+      <match lemma="juin" tags="n.*"/>
+      <match lemma="gulh" tags="n.*"/>
+      <match lemma="julhet" tags="n.*"/>
+      <match lemma="gulhet" tags="n.*"/>
+      <match lemma="junhsèga" tags="n.*"/>
+      <match lemma="agost" tags="n.*"/>
+      <match lemma="aost" tags="n.*"/>
+      <match lemma="setembre" tags="n.*"/>
+      <match lemma="seteme" tags="n.*"/>
+      <match lemma="octobre" tags="n.*"/>
+      <match lemma="octòbre" tags="n.*"/>
+      <match lemma="novembre" tags="n.*"/>
+      <match lemma="noveme" tags="n.*"/>
+      <match lemma="decembre" tags="n.*"/>
+      <match lemma="deceme" tags="n.*"/>
+    </or></def-seq>
+  </def-seqs>
+</pre>
+They have to be referenced in the rules as follows:
+<pre>
+  <rule weight="1.0">
+    <or>
+      <seq n="jorns"/>
+      <seq n="meses"/>
+      <match lemma="prima" tags="n.*"/>
+      <match lemma="estiu" tags="n.*"/>
+      <match lemma="auton" tags="n.*"/>
+      <match lemma="ivèrn" tags="n.*"/>
+    </or>
+    <match lemma="passat" tags="adj.*"><select lemma="dernier"/></match>
+  </rule>
+</pre>
 ===Special cases===

@@ Line 155: / Line 155: @@
     <match lemma="mossèn" tags="n.m.sg"><select lemma="monseigneur"/></match>
     <match tags=""/>
+  </rule>
+</pre>
+The last rule can be improved specifying that the unknown word should be capitalized:
+<pre>
+  <rule weight="1.0">
+    <match lemma="mossèn" tags="n.m.sg"><select lemma="monseigneur"/></match>
+    <match tags="" case="Aa"/>
   </rule>
 </pre>