A long introduction to transfer rules - Revision history

Unhammer: /* Overview of a transfer file */

2022-03-17T12:25:21Z

Overview of a transfer file

Unhammer: /* Overview of a transfer file */

2022-03-17T12:24:56Z

Overview of a transfer file

Unhammer: /* Overview of a transfer file */

2022-03-17T12:24:15Z

Overview of a transfer file

Unhammer: /* Overview of a transfer file */

2022-03-17T12:19:31Z

Overview of a transfer file

Unhammer: /* Overview of a transfer file */

2022-03-17T12:17:27Z

Overview of a transfer file

Unhammer: /* Some preliminaries */

2022-03-17T10:39:32Z

Some preliminaries

Unhammer: /* How lexical transfer is processed */

2022-03-17T10:38:30Z

How lexical transfer is processed

Unhammer at 10:36, 17 March 2022

2022-03-17T10:36:24Z

Pmodi: /* Step 1 */

2020-05-16T11:56:09Z

Step 1

Ron l: /* Step 4 */

2016-12-16T19:17:13Z

Step 4

@@ Line 119: / Line 119: @@
 </pre>
-I'll try and give a tag-by-tag account. (Confusingly, tags can be {{xtag|XML}} tags like the above or they can be linguistic tags in the input like {{tag|adv}}, both types surrounded by &lt; and &gt;; I try to be explicitly call them "XML tags" or "input tags" here.)
+I'll try and give a tag-by-tag account. (Confusingly, tags can be {{xtag|XML}} tags like the above or they can be linguistic tags in the input like {{tag|adv}}, both types surrounded by &lt; and &gt;; I try to explicitly call them "XML tags" or "input tags" here.)
 The {{xtag|transfer}} and {{xtag|/transfer}} XML tags don't do anything. They just encapsulate the rest of the sections.

@@ Line 119: / Line 119: @@
 </pre>
-I'll try and give a tag-by-tag account. (Confusingly, tags can be XML tags like the above or they can be linguistic tags in the input like {{tag|adv}}, both types surrounded by &lt; and &gt;; I try to be explicitly call them "XML tags" or "input tags" here.)
+I'll try and give a tag-by-tag account. (Confusingly, tags can be {{xtag|XML}} tags like the above or they can be linguistic tags in the input like {{tag|adv}}, both types surrounded by &lt; and &gt;; I try to be explicitly call them "XML tags" or "input tags" here.)
 The {{xtag|transfer}} and {{xtag|/transfer}} XML tags don't do anything. They just encapsulate the rest of the sections.

@@ Line 119: / Line 119: @@
 </pre>
+I'll try and give a tag-by-tag account. (Confusingly, tags can be XML tags like the above or they can be linguistic tags in the input like {{tag|adv}}, both types surrounded by &lt; and &gt;; I try to be explicitly call them "XML tags" or "input tags" here.)
-I'll try and give a tag-by-tag account.
-The {{tag|transfer}} and {{tag|/transfer}} XML tags don't do anything. They just encapsulate the rest of the sections.
+The {{xtag|transfer}} and {{xtag|/transfer}} XML tags don't do anything. They just encapsulate the rest of the sections.
-The {{tag|section-def-cats}} contains definitions of pattern categories that can be '''matched''' by rules. Each {{tag|def-cat}} has a name in the attribute {{attr|n}} and a list of {{tag|cat-item}}'s with the input patterns it should match. A pattern can be defined by {{attr|tags}} or {{attr|lemma}} or both. The {{attr|tags}} attribute often contains a {{attr|.*}} meaning "one or more arbitrary input tags".
+The {{xtag|section-def-cats}} contains definitions of pattern categories that can be '''matched''' by rules. Each {{xtag|def-cat}} has a name in the attribute {{attr|n}} and a list of {{xtag|cat-item}}'s with the input patterns it should match. A pattern can be defined by {{attr|tags}} or {{attr|lemma}} or both. The {{attr|tags}} attribute often contains a {{attr|.*}} meaning "one or more arbitrary input tags".
-The {{tag|section-def-attrs}} contains definitions of input tag sets that can be '''used''' (read to, changed or output) by rules (using the {{tag|clip}} feature). Each {{tag|def-attr}} has a name in the attribute {{attr|n}} and a list of {{tag|attr-item}}'s each with a list of consecutive {{attr|tags}}.
+The {{xtag|section-def-attrs}} contains definitions of input tag sets that can be '''used''' (read to, changed or output) by rules (using the {{xtag|clip}} feature). Each {{xtag|def-attr}} has a name in the attribute {{attr|n}} and a list of {{xtag|attr-item}}'s each with a list of consecutive {{attr|tags}}.
-The {{tag|section-def-vars}} contains variable names that can be used in rules. Each {{tag|def-var}} has a name in the attribute {{attr|n}}.
+The {{xtag|section-def-vars}} contains variable names that can be used in rules. Each {{xtag|def-var}} has a name in the attribute {{attr|n}}.
-The follows the rules in {{tag|section-rules}}, each rule is inside {{tag|rule}}..{{tag|/rule}} and has first a {{tag|pattern}} with a list of one or more {{tag|pattern-item}}'s each referring to the {{attr|n}}ame of a {{tag|cat-item}}. This is used for ''matching'' on the input. Then follows the {{tag|action}} which specifies what to do with the matched words.
+The follows the rules in {{xtag|section-rules}}, each rule is inside {{xtag|rule}}..{{xtag|/rule}} and has first a {{xtag|pattern}} with a list of one or more {{xtag|pattern-item}}'s each referring to the {{attr|n}}ame of a {{xtag|cat-item}}. This is used for ''matching'' on the input. Then follows the {{xtag|action}} which specifies what to do with the matched words.
-In this example, we use {{tag|let}} to change the target language side ({{attr|side}}="tl") part of the first matched pattern ({{tag|clip}} with {{attr|pos}}=1), changing the input tags that match the {{tag|attr-item}} with {{attr|n}}ame "some_feature_of_a_word" to contain the input tag {{tag|''myotherfeature''}} (using {{tag|lit-tag}} with that value, given by the attribute {{attr|v}}). So the attribute "some_feature_of_a_word" can match either the input tag {{tag|''myfeature''}} or {{tag|''myotherfeature''}}, but in the output it will only be {{tag|''myotherfeature''}}. (If the input tag wasn't there, nothing is changed). Finally, after changing that input tag, the rule uses {{tag|out}} to print the matched and changed pattern. The {{tag|lu}} ensures the pattern is surrounded by {{attr|^}} and {{attr|$}}. The <code><nowiki>&lt;clip pos="1" side="tl" part="whole"&gt;</nowiki></code> refers to the whole lemma + tag sequence of the target language side of the first matched pattern.
+In this example, we use {{xtag|let}} to change the target language side ({{attr|side}}="tl") part of the first matched pattern ({{xtag|clip}} with {{attr|pos}}=1), changing the input tags that match the {{xtag|attr-item}} with {{attr|n}}ame "some_feature_of_a_word" to contain the input tag {{tag|myotherfeature}} (using {{xtag|lit-tag}} with that value, given by the attribute {{attr|v}}). So the attribute "some_feature_of_a_word" can match either the input tag {{tag|myfeature}} or {{tag|myotherfeature}}, but in the output it will only be {{tag|myotherfeature}}. (If the input tag wasn't there, nothing is changed). Finally, after changing that input tag, the rule uses {{xtag|out}} to print the matched and changed pattern. The {{xtag|lu}} ensures the pattern is surrounded by {{attr|^}} and {{attr|$}}. The <code><nowiki>&lt;clip pos="1" side="tl" part="whole"&gt;</nowiki></code> refers to the whole lemma + tag sequence of the target language side of the first matched pattern.
 ===How rules are applied===
-Rules are tried from top to bottom in the file, so first rules are prioritised. But the longest match (reading from left to right) is used ([[LRLM]]), so longer matches, ie. matching more {{tag|pattern-item}}s, will be prioritised over shorter.
+Rules are tried from top to bottom in the file, so first rules are prioritised. But the longest match (reading from left to right) is used ([[LRLM]]), so longer matches, ie. matching more {{xtag|pattern-item}}s, will be prioritised over shorter.
 ==Practical examples==

@@ Line 131: / Line 131: @@
 The follows the rules in {{tag|section-rules}}, each rule is inside {{tag|rule}}..{{tag|/rule}} and has first a {{tag|pattern}} with a list of one or more {{tag|pattern-item}}'s each referring to the {{attr|n}}ame of a {{tag|cat-item}}. This is used for ''matching'' on the input. Then follows the {{tag|action}} which specifies what to do with the matched words.
-In this example, we use {{tag|let}} to change the target language side ({{attr|side="tl"}}) part of the first matched pattern ({{tag|clip pos="1"}}), changing the input tags that match the {{tag|attr-item}} with {{attr|n}}ame "some_feature_of_a_word" to contain the input tag {{tag|''myotherfeature''}} (using {{tag|lit-tag}} with that value, given by the attribute {{attr|v}}). So the attribute "some_feature_of_a_word" can match either the input tag {{tag|''myfeature''}} or {{tag|''myotherfeature''}}, but in the output it will only be {{tag|''myotherfeature''}}. (If the input tag wasn't there, nothing is changed). Finally, after changing that input tag, the rule uses {{tag|out}} to print the matched and changed pattern. The {{tag|lu}} ensures the pattern is surrounded by {{attr|^}} and {{attr|$}}. The {{tag|clip pos="1" side="tl" part="whole"}} refers to the whole lemma + tag sequence of the target language side of the first matched pattern.
+In this example, we use {{tag|let}} to change the target language side ({{attr|side}}="tl") part of the first matched pattern ({{tag|clip}} with {{attr|pos}}=1), changing the input tags that match the {{tag|attr-item}} with {{attr|n}}ame "some_feature_of_a_word" to contain the input tag {{tag|''myotherfeature''}} (using {{tag|lit-tag}} with that value, given by the attribute {{attr|v}}). So the attribute "some_feature_of_a_word" can match either the input tag {{tag|''myfeature''}} or {{tag|''myotherfeature''}}, but in the output it will only be {{tag|''myotherfeature''}}. (If the input tag wasn't there, nothing is changed). Finally, after changing that input tag, the rule uses {{tag|out}} to print the matched and changed pattern. The {{tag|lu}} ensures the pattern is surrounded by {{attr|^}} and {{attr|$}}. The <code><nowiki>&lt;clip pos="1" side="tl" part="whole"&gt;</nowiki></code> refers to the whole lemma + tag sequence of the target language side of the first matched pattern.
 ===How rules are applied===

@@ Line 84: / Line 84: @@
 ==Overview of a transfer file==
 It's hard to give a step-by-step overview of what a transfer file looks like because there is quite a lot of obligatory parts that need to go into even the most basic file. But, it's important to get a general view before we go into the details. Here is an example in which I'm deliberately not going to use linguistic names for the different parts, to try and avoid assumptions.
 <pre>
@@ Line 102: / Line 102: @@
   <section-def-vars>
     <def-var n="blank"/>
   </section-def-vars>
   <section-rules>
     <rule>
@@ Line 119: / Line 119: @@
 </pre>
-I'll try and give a tag-by-tag account... the {{tag|transfer}} and {{tag|/transfer}} tags don't do anything. They just encapsulate the rest of the sections.
+I'll try and give a tag-by-tag account.
+The {{tag|transfer}} and {{tag|/transfer}} XML tags don't do anything. They just encapsulate the rest of the sections.
+The {{tag|section-def-cats}} contains definitions of pattern categories that can be '''matched''' by rules. Each {{tag|def-cat}} has a name in the attribute {{attr|n}} and a list of {{tag|cat-item}}'s with the input patterns it should match. A pattern can be defined by {{attr|tags}} or {{attr|lemma}} or both. The {{attr|tags}} attribute often contains a {{attr|.*}} meaning "one or more arbitrary input tags".
+The {{tag|section-def-attrs}} contains definitions of input tag sets that can be '''used''' (read to, changed or output) by rules (using the {{tag|clip}} feature). Each {{tag|def-attr}} has a name in the attribute {{attr|n}} and a list of {{tag|attr-item}}'s each with a list of consecutive {{attr|tags}}.
+The {{tag|section-def-vars}} contains variable names that can be used in rules. Each {{tag|def-var}} has a name in the attribute {{attr|n}}.
+The follows the rules in {{tag|section-rules}}, each rule is inside {{tag|rule}}..{{tag|/rule}} and has first a {{tag|pattern}} with a list of one or more {{tag|pattern-item}}'s each referring to the {{attr|n}}ame of a {{tag|cat-item}}. This is used for ''matching'' on the input. Then follows the {{tag|action}} which specifies what to do with the matched words.
+In this example, we use {{tag|let}} to change the target language side ({{attr|side="tl"}}) part of the first matched pattern ({{tag|clip pos="1"}}), changing the input tags that match the {{tag|attr-item}} with {{attr|n}}ame "some_feature_of_a_word" to contain the input tag {{tag|''myotherfeature''}} (using {{tag|lit-tag}} with that value, given by the attribute {{attr|v}}). So the attribute "some_feature_of_a_word" can match either the input tag {{tag|''myfeature''}} or {{tag|''myotherfeature''}}, but in the output it will only be {{tag|''myotherfeature''}}. (If the input tag wasn't there, nothing is changed). Finally, after changing that input tag, the rule uses {{tag|out}} to print the matched and changed pattern. The {{tag|lu}} ensures the pattern is surrounded by {{attr|^}} and {{attr|$}}. The {{tag|clip pos="1" side="tl" part="whole"}} refers to the whole lemma + tag sequence of the target language side of the first matched pattern.
 ===How rules are applied===
+Rules are tried from top to bottom in the file, so first rules are prioritised. But the longest match (reading from left to right) is used ([[LRLM]]), so longer matches, ie. matching more {{tag|pattern-item}}s, will be prioritised over shorter.
 ==Practical examples==

@@ Line 78: / Line 78: @@
 ==Some preliminaries==
+* Pattern: What we match. Patterns are given numbers based on what their order was, so the first pattern is pos="1", second is pos="2", etc.
-* Pattern:
+* Action: What we do with the matched pattern, including outputting it.
-* Action:
 ==Overview of a transfer file==

@@ Line 74: / Line 74: @@
 </pre>
-Note that the target-language lexical form as defined in the bilingual dictionary is produced by replacing two tags on the source side (i.e. <code><s n="tag1"/><s n="tag2"/></code>) with one tag on the target side (i.e. <code><s n="tag1"/></code>). Important: any source-language tags not matched in the bilingual dictionary entry are copied into the output on the target language side. In our example <code><tag3></code> gets copied from the input to the output.
+Note that the target-language lexical form as defined in the bilingual dictionary is produced by replacing two tags on the source side (i.e. <code><nowiki><s n="tag1"/><s n="tag2"/></nowiki></code>) with one tag on the target side (i.e. <code><nowiki><s n="tag1"/></nowiki></code>). Important: any source-language tags not matched in the bilingual dictionary entry are copied into the output on the target language side. In our example <code><tag3></code> gets copied from the input to the output.
 ==Some preliminaries==

@@ Line 10: / Line 10: @@
 * There are no recursive rules. Rules match fixed-length patterns. There is no optionality at the level of words. There is no way of saying one-or-more, it's just one.
+** (but see [[Apertium-recursive]] if you're starting a new language pair)
 * Apertium's rules are very tied to the [[Apertium stream format]]. If you don't understand the stream format, it will be a lot more difficult to understand the rules.
 * Rules contain both ''declarative'' parts and ''procedural'' parts. You can't just expect to say ''what'' you want or ''how'' you want to do it. You need to do both -- but in different places (but it's quite intuitive).

@@ Line 266: / Line 266: @@
 </pre>
-Why do we need <code>.*.*</code> ? -- Because of how the matching system in categories works. In the middle of tag sequences, a <code>*</code> is counted as a single tag. At the end, it is counted as any sequence of tags. So, we have {{tag|vblex}} followed by any tag, followed by any tag, followed by {{tag|lp}} followed by any sequence of tags. People familiar with regular expressions should take a special note that <code>.*</code> does not mean "anything", and "*" does not many "any amount of times" (except at the end of a pattern). <code>*</code> is more like a wildcard.
+Why do we need <code>.*.*</code> ? -- Because of how the matching system in categories works. In the middle of tag sequences, a <code>*</code> is counted as a single tag. At the end, it is counted as any sequence of tags. So, we have {{tag|vblex}} followed by any tag, followed by any tag, followed by {{tag|lp}} followed by any sequence of tags. People familiar with regular expressions should take a special note that <code>.*</code> does not mean "anything", and "*" does not mean "any amount of times" (except at the end of a pattern). <code>*</code> is more like a wildcard.
 . Edit the example rule (notice that the example rule outputs only the first among two tokens, so that our biti-clt token will disappear) and replace the pattern.

@@ Line 458: / Line 458: @@
 The {{tag|lu}} outputs a <code>^</code> and the {{tag|/lu}} outputs a <code>$</code>. The {{tag|lit}} tag outputs a string literal, and the {{tag|lit-tag}} outputs a tag sequence which starts with a <code>&lt;</code>, ends with a <code>&gt;</code> and where each <code>.</code> is replaced with <code>&gt;&lt;</code>. The {{tag|b/}} outputs a single space ' ' character.
-. Defining features for person and number.
+. Define features for person and number.
-The next step is to declare that features from the verb be output. To do this, we need to again define some features and their possible values.
+The next step is to declare the features from the verb be output. To do this, we need to again define some features and their possible values.
 <pre>