Postgenerator - Revision history

Popcorndude: /* Postgeneration Using apertium-separable */ we don't use literal slashes anymore

2024-03-01T19:39:46Z

Postgeneration Using apertium-separable: we don't use literal slashes anymore

Popcorndude at 20:59, 22 December 2022

2022-12-22T20:59:31Z

Popcorndude: lsx-proc -p

2022-12-22T20:27:42Z

lsx-proc -p

Francis Tyers at 15:47, 24 August 2019

2019-08-24T15:47:03Z

Francis Tyers at 15:46, 24 August 2019

2019-08-24T15:46:25Z

Francis Tyers: Created page with "Sometimes you want to be able to merge two tokens in output, for example for contractions, e.g. ''de'' + ''el'' = ''del''. You can do this using the postgenerator. First mak..."

2019-08-24T15:45:10Z

Created page with "Sometimes you want to be able to merge two tokens in output, for example for contractions, e.g. ''de'' + ''el'' = ''del''. You can do this using the postgenerator. First mak..."

New page

Sometimes you want to be able to merge two tokens in output, for example for contractions, e.g. ''de'' + ''el'' = ''del''.

You can do this using the postgenerator.

First make sure you add the postgenerator wakeup symbol to your monolingual dictionary, e.g. <tt>apertium-aaa.aaa.dix</tt>

<tt>apertium-aaa.aaa.dix</tt>:

<pre>
<pardef n="/de__pr">
<e r="LR"><p><l>de</l><r>de<s n="pr"/></r></p></e>
<e r="RL"><p><l><a/>de</l><r>de<s n="pr"/></r></p></e>
</pardef>

...

<e lm="de"><i></i><par n="/de__pr"/></e>

...
</pre>

You should get entries like:

<pre>
de:>:de<pr>
~de:<:de<pr>
</pre>

from <tt>lt-expand apertium-aaa.aaa.dix</tt>.
<tt>apertium-aaa.post-aaa.dix</tt>:

<pre>

<?xml version="1.0" encoding="UTF-8"?>
<dictionary>
<alphabet/>
<sdefs>
<sdef n="test"/>
</sdefs>
<section id="main" type="standard">

<e> <p><l><a/>de<b/>el</l><r>del</r></p></e>
</section>
</dictionary>
</pre>

You can compile it like:

<pre>
$ lt-comp lr apertium-aaa.post-aaa.dix aaa.autopgen.bin
main@standard 7 6
</pre>

And use it like:

<pre>
$ echo "~de el" | lt-proc -p aaa.autopgen.bin
del
</pre>

In your modes file:

<pre>

</pre>

@@ Line 91: / Line 91: @@
   <section id="main" type="standard">
     <e>
-      <i>de<s n="pr"/>/d</i>
+      <i>de<s n="pr"/><f/>d</i>
       <p><l>e</l><r></r></p>
-      <i><d space="no"/>el<s n="det"/><t/>/el</i>
+      <i><d space="no"/>el<s n="det"/><t/><f/>el</i>
     </e>
   </section>
@@ Line 99: / Line 99: @@
 </syntaxhighlight>
+The <code><f/></code> on lines 10 and 12 represents a reading separator (<code>/</code> in stream format).
-Note the literal slashes on lines 10 and 12.
 This will turn <code>^de<pr>/de$ ^el<det><def><m><sg>/el$</code> into <code>^de<pr>/d$^el<det><def><m><sg>/el$</code>.
@@ Line 107: / Line 107: @@
 <syntaxhighlight lang="xml" line>
 <e>
-  <i>de<s n="pr"/>/d</i>
+  <i>de<s n="pr"/><f/>d</i>
   <p><l>e</l><r></r></p>
   <i><d space="no"/>el</i>
@@ Line 113: / Line 113: @@
 </syntaxhighlight>
-Here there is a slash in the first word, because we are listing both the analysis and the surface form, but there is no slash in the second because we are only listing the surface form.
+Here there is a <code><f/></code>, representing a slash, in the first word, because we are listing both the analysis and the surface form, but there is no <code><f/></code>/slash in the second because we are only listing the surface form.
 This is, admittedly, a bit more complicated to write than the <code>lt-proc</code> way of doing postgen, but it does allow the rules to apply to each other's output if <code>lsx-proc</code> is run with <code>-r</code>/<code>--repeat</code>, and it also allows case handling to be moved out of the postgenerator to the [[Capitalization restoration|dedicated module]], both of which may significantly reduce the number of rules. In addition, the morphological dictionaries do not need to create dedicated entries with the alarm character.

@@ Line 83: / Line 83: @@
 <syntaxhighlight lang="xml" line>
 <?xml version="1.0" encoding="UTF-8"?>
-<dictionary>
+<dictionary type="separable">
   <alphabet/>
   <sdefs>
-    <sdef n="test"/>
+    <sdef n="pr"/>
+    <sdef n="det"/>
   </sdefs>
   <section id="main" type="standard">
@@ Line 92: / Line 93: @@
       <i>de<s n="pr"/>/d</i>
       <p><l>e</l><r></r></p>
-      <i><d space="no"/>el<t/>/el</i>
+      <i><d space="no"/>el<s n="det"/><t/>/el</i>
     </e>
   </section>
 </dictionary>
 </syntaxhighlight>
+Note the literal slashes on lines 10 and 12.
+This will turn <code>^de<pr>/de$ ^el<det><def><m><sg>/el$</code> into <code>^de<pr>/d$^el<det><def><m><sg>/el$</code>.
+If this were a rule that applied for any word beginning with an e rather than just the definite article, we could skip the lemma and tags and just write
+<syntaxhighlight lang="xml" line>
+<e>
+  <i>de<s n="pr"/>/d</i>
+  <p><l>e</l><r></r></p>
+  <i><d space="no"/>el</i>
+</e>
+</syntaxhighlight>
+Here there is a slash in the first word, because we are listing both the analysis and the surface form, but there is no slash in the second because we are only listing the surface form.
+This is, admittedly, a bit more complicated to write than the <code>lt-proc</code> way of doing postgen, but it does allow the rules to apply to each other's output if <code>lsx-proc</code> is run with <code>-r</code>/<code>--repeat</code>, and it also allows case handling to be moved out of the postgenerator to the [[Capitalization restoration|dedicated module]], both of which may significantly reduce the number of rules. In addition, the morphological dictionaries do not need to create dedicated entries with the alarm character.
 [[Category:Modules]]

@@ Line 72: / Line 72: @@
 </pre>
+== Postgeneration Using apertium-separable ==
+If you have at least version 0.7.0 of apertium-separable, you can accomplish the same as above using <code>lsx-proc</code>.
+This allows you to write postgeneration rules conditioned on lemmas and tags rather than needing multiple copies of each relevant dictionary entry.
+For the <code>de + el → del</code> rule above, we can write the following:
+<syntaxhighlight lang="xml" line>
+<?xml version="1.0" encoding="UTF-8"?>
+<dictionary>
+  <alphabet/>
+  <sdefs>
+    <sdef n="test"/>
+  </sdefs>
+  <section id="main" type="standard">
+    <e>
+      <i>de<s n="pr"/>/d</i>
+      <p><l>e</l><r></r></p>
+      <i><d space="no"/>el<t/>/el</i>
+    </e>
+  </section>
+</dictionary>
+</syntaxhighlight>
 [[Category:Modules]]

← Older revision		Revision as of 15:47, 24 August 2019
Line 72:		Line 72:

	</pre>		</pre>

			[[Category:Modules]]

@@ Line 62: / Line 62: @@
 <pre>
+...
+      <program name="lt-proc $1">
+        <file name="aaa-bbb.autogen.bin"/>
+      </program>
+      <program name="lt-proc -p">
+        <file name="aaa-bbb.autopgen.bin"/>
+      </program>
+...
 </pre>