Inconditional section - Revision history

Unhammer at 07:20, 8 June 2019

2019-06-08T07:20:27Z

Unhammer at 07:11, 8 June 2019

2019-06-08T07:11:58Z

Unhammer: /* inconditional */

2019-06-08T07:09:49Z

inconditional

Bech: Link to French page

2014-10-07T11:44:43Z

Link to French page

Unhammer: /* postblank / preblank */

2013-08-15T10:39:39Z

postblank / preblank

Unhammer: /* See also */

2013-08-15T10:38:58Z

Unhammer at 10:37, 15 August 2013

2013-08-15T10:37:53Z

Bech: Category:Documentation in English

2012-03-24T10:41:11Z

Category:Documentation in English

Francis Tyers at 11:21, 18 September 2009

2009-09-18T11:21:41Z

Jacob Nordfalk at 19:17, 17 September 2009

2009-09-17T19:17:37Z

@@ Line 2: / Line 2: @@
 The &lt;section&gt; element in a dix file can be of type standard, inconditional, postblank or preblank.[https://github.com/apertium/lttoolbox/blob/master/lttoolbox/dix.dtd#L59]
+The section type is used to change how tokenisation works.
 ==inconditional==
+Normally, analysis must be separated by spaces (or other blanks), but an analysis in an 'inconditional' section can appear right after or before other analyses ('standard' or non-standard) or even right next to unknowns. The 'inconditional' section of a dictionary typically contains punctuation, and such things.
-An 'inconditional' section of a dictionary typically contains punctuation, and such things. The section type is used to change how tokenisation works.
 In detail:
 Analysis in lttoolbox works in a left-to-right longest match fashion. We read characters from input, trying to match them in the transducer, and if we've reached a final transition when at the "end" of the input word, we can output the analysis (or we can try matching something even longer, but if not we use the match we found). But how do we know when we're at the end of an input word? Any "blank" character (not in &lt;alphabet&gt; in .dix files) is allowed to separate words, so e.g. spaces or other strange characters can separate words. But we may also say that certain words-with-analyses can separate words – to mark an analysis as "may act as word-separator", we put it in the 'inconditional' section.
-<!-- This is wrong:
+<!--
+This is wrong:
- "Inconditional means 'if you see this, stop processing immediately and start reading a new word'. Stop when you reach the end of a possible transduction."
+     "Inconditional means 'if you see this, stop processing immediately and start reading a new word'. Stop when you reach the end of a possible transduction."
-We can have '.' in inconditional section and still get one long analysis of "Dr. Octagon" if "Dr. Octagon" is in e.g. the standard section. But if there was no analysis of the string up until the '.', then we may immediately output an unknown.
+- we can have '.' in inconditional section and still get one long analysis of "Dr. Octagon" if "Dr. Octagon" is in e.g. the standard section. But if there was no analysis of the string up until the '.', then we may immediately output an unknown.
  -->
-In summary, a space (or other blank) is not required to end the analysis of the preceding string, and neither is a blank required to start a new analysis afterwards.
 <pre>

@@ Line 1: / Line 1: @@
 [[Section inconditionnelle|En français]]
+The &lt;section&gt; element in a dix file can be of type standard, inconditional, postblank or preblank.[https://github.com/apertium/lttoolbox/blob/master/lttoolbox/dix.dtd#L59]
 ==inconditional==

@@ Line 3: / Line 3: @@
 ==inconditional==
-An inconditional ('unconditional') section of a dictionary typically contains punctuation, and such things.
+An 'inconditional' section of a dictionary typically contains punctuation, and such things. The section type is used to change how tokenisation works.
+In detail:
-The main section of a dictionary works on a longest-match basis.
+Analysis in lttoolbox works in a left-to-right longest match fashion. We read characters from input, trying to match them in the transducer, and if we've reached a final transition when at the "end" of the input word, we can output the analysis (or we can try matching something even longer, but if not we use the match we found). But how do we know when we're at the end of an input word? Any "blank" character (not in &lt;alphabet&gt; in .dix files) is allowed to separate words, so e.g. spaces or other strange characters can separate words. But we may also say that certain words-with-analyses can separate words – to mark an analysis as "may act as word-separator", we put it in the 'inconditional' section.
+<!-- This is wrong:
-Inconditional means 'if you see this, stop processing immediately and start reading a new word'. Stop when you reach the end of a possible transduction.
+ "Inconditional means 'if you see this, stop processing immediately and start reading a new word'. Stop when you reach the end of a possible transduction."
+We can have '.' in inconditional section and still get one long analysis of "Dr. Octagon" if "Dr. Octagon" is in e.g. the standard section. But if there was no analysis of the string up until the '.', then we may immediately output an unknown.
+ -->
-You could say that the "only" difference is that a space is not required to start a new match.
+In summary, a space (or other blank) is not required to end the analysis of the preceding string, and neither is a blank required to start a new analysis afterwards.
 <pre>
@@ Line 16: / Line 20: @@
 </pre>
-It doesn't need the space between 23 and men because numbers are in an 'inconditional' section.
+In the above, we don't need the space between 23 and men because numbers are in an 'inconditional' section.
+We can get some weird effects by putting plain characters in 'inconditional':
 <pre>
 <dictionary>

← Older revision		Revision as of 11:44, 7 October 2014
Line 1:		Line 1:
			[[Section inconditionnelle\|En français]]

	==inconditional==		==inconditional==

@@ Line 40: / Line 40: @@
 == postblank / preblank ==
-Note that ''postblank'' and ''preblank'' sections work exactly like ''inconditional'' with respect to how they tokenise the input. The only difference is that anything in a postblank section will make lt-proc output a space after the token (in preblank, before the token). So if "☃" is in postblank (tagged sent), and "foo" and "bar" are in a regular section (tagged n), then we get:
+The ''postblank'' and ''preblank'' sections work exactly like ''inconditional'' with respect to how they tokenise the input.
+The only difference is that anything in a postblank section will make lt-proc output a space after the token (in preblank, before the token).
+So if "☃" is in postblank (tagged sent), and "foo" and "bar" are in a regular section (tagged n), then we get:
 <pre>
 $ echo 'foo☃bar' | lt-proc analyser.bin

@@ Line 55: / Line 55: @@
 == See also ==
-* [[Morphological dictionary]]
+* [[Morphological dictionary#Sections]]
 [[Category:Terminology]]

← Older revision		Revision as of 10:41, 24 March 2012
Line 38:		Line 38:

	[[Category:Terminology]]		[[Category:Terminology]]
			[[Category:Documentation in English]]

@@ Line 36: / Line 36: @@
 ^aa/aa<aa>$^aa/aa<aa>$^a/a<aa>$
 </pre>
+[[Category:Terminology]]

@@ Line 26: / Line 26: @@
   </section>
 </dictionary>
-jim@jim-laptop:~/Devel/apertium-git/incubator/apertium-en-it$ echo aaa |lt-proc  sample.bin
+$ echo aaa |lt-proc  sample.bin
 ^aa/aa<aa>$^a/a<aa>$
-jim@jim-laptop:~/Devel/apertium-git/incubator/apertium-en-it$ echo aaaa |lt-proc  sample.bin
+$ echo aaaa |lt-proc  sample.bin
 ^aa/aa<aa>$^aa/aa<aa>$
-jim@jim-laptop:~/Devel/apertium-git/incubator/apertium-en-it$ echo aaaaa |lt-proc  sample.bin
+$ echo aaaaa |lt-proc  sample.bin
 ^aa/aa<aa>$^aa/aa<aa>$^a/a<aa>$
 </pre>