https://wiki.apertium.org/w/index.php?action=history&feed=atom&title=User-based_chunking 2026-05-14T02:21:44Z Revision history for this page on the wiki MediaWiki 1.34.1 https://wiki.apertium.org/w/index.php?title=User-based_chunking&diff=72445&oldid=prev 2020-06-21T21:15:56Z

<p></p> <table class="diff diff-contentalign-left" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">← Older revision</td> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">Revision as of 21:15, 21 June 2020</td> </tr><tr> <td colspan="2" class="diff-lineno">Line 13:</td> <td colspan="2" class="diff-lineno">Line 13:</td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>** And then he came back and said "Bu ne lan?"</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>** And then he came back and said "Bu ne lan?"</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>==Limitations==</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>* The words in the chunk should be contiguous. </div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>==Solutions==</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>==Solutions==</div></td> </tr> </table>

Francis Tyers https://wiki.apertium.org/w/index.php?title=User-based_chunking&diff=72444&oldid=prev 2020-06-21T21:13:36Z

<p><span dir="auto"><span class="autocomment">Solutions</span></span></p> <table class="diff diff-contentalign-left" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">← Older revision</td> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">Revision as of 21:13, 21 June 2020</td> </tr><tr> <td colspan="2" class="diff-lineno">Line 26:</td> <td colspan="2" class="diff-lineno">Line 26:</td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>after lt-proc | agger:</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>after lt-proc | agger:</div></td> </tr> <tr> <td class="diff-marker" data-marker="−"></td> <td class="diff-deletedline diff-side-deleted"><div>^+Microsoft/Microsoft&lt;np&gt;&lt;al&gt;$ ^+365/365&lt;num&gt;$ ^+for/for&lt;pr&gt;$ ^+business/business$ ^is/be&lt;vbser&gt;&lt;pres&gt;&lt;p3&gt;&lt;sg&gt;$ ^the/the&lt;det&gt;&lt;def&gt;&lt;sp&gt;$ right choice for your company</div></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>^+Microsoft/Microsoft&lt;np&gt;&lt;al&gt;$ ^+365/365&lt;num&gt;$ ^+for/for&lt;pr&gt;$ ^+business/business<ins class="diffchange diffchange-inline">&lt;n&gt;</ins>$ ^is/be&lt;vbser&gt;&lt;pres&gt;&lt;p3&gt;&lt;sg&gt;$ ^the/the&lt;det&gt;&lt;def&gt;&lt;sp&gt;$ right choice for your company</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>after apertium-separable:</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>after apertium-separable:</div></td> </tr> </table>

Francis Tyers https://wiki.apertium.org/w/index.php?title=User-based_chunking&diff=72443&oldid=prev 2020-06-21T21:10:21Z

<p><span dir="auto"><span class="autocomment">Solutions</span></span></p> <table class="diff diff-contentalign-left" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">← Older revision</td> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">Revision as of 21:10, 21 June 2020</td> </tr><tr> <td colspan="2" class="diff-lineno">Line 25:</td> <td colspan="2" class="diff-lineno">Line 25:</td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>+Microsoft +365 +for +business is right choice for your company</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>+Microsoft +365 +for +business is right choice for your company</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>after lt-proc | agger:</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>^+Microsoft/Microsoft&lt;np&gt;&lt;al&gt;$ ^+365/365&lt;num&gt;$ ^+for/for&lt;pr&gt;$ ^+business/business$ ^is/be&lt;vbser&gt;&lt;pres&gt;&lt;p3&gt;&lt;sg&gt;$ ^the/the&lt;det&gt;&lt;def&gt;&lt;sp&gt;$ right choice for your company</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>^+Microsoft/Microsoft&lt;np&gt;&lt;al&gt;$ ^+365/365&lt;num&gt;$ ^+for/for&lt;pr&gt;$ ^+business/business$ ^is/be&lt;vbser&gt;&lt;pres&gt;&lt;p3&gt;&lt;sg&gt;$ ^the/the&lt;det&gt;&lt;def&gt;&lt;sp&gt;$ right choice for your company</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>after apertium-separable:</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>^+Microsoft 365 for business/Microsoft 365 for business&lt;np&gt;&lt;al&gt;$ ^is/be&lt;vbser&gt;&lt;pres&gt;&lt;p3&gt;&lt;sg&gt;$ the right choice for your company.</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>^+Microsoft 365 for business/Microsoft 365 for business&lt;np&gt;&lt;al&gt;$ ^is/be&lt;vbser&gt;&lt;pres&gt;&lt;p3&gt;&lt;sg&gt;$ the right choice for your company.</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>after bidix lookup:</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>^+Microsoft 365 for business/Microsoft 365 for business&lt;np&gt;&lt;al&gt;/Microsoft 365 for business&lt;np&gt;&lt;al&gt;$ ^is/be&lt;vbser&gt;&lt;pres&gt;&lt;p3&gt;&lt;sg&gt;/ser&lt;vbser&gt;&lt;pres&gt;&lt;p3&gt;&lt;sg&gt;$ ^the/the&lt;det&gt;&lt;def&gt;&lt;sp&gt;/el&lt;det&gt;&lt;def&gt;&lt;GD&gt;&lt;ND&gt;$ right choice for your company.</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>^+Microsoft 365 for business/Microsoft 365 for business&lt;np&gt;&lt;al&gt;/Microsoft 365 for business&lt;np&gt;&lt;al&gt;$ ^is/be&lt;vbser&gt;&lt;pres&gt;&lt;p3&gt;&lt;sg&gt;/ser&lt;vbser&gt;&lt;pres&gt;&lt;p3&gt;&lt;sg&gt;$ ^the/the&lt;det&gt;&lt;def&gt;&lt;sp&gt;/el&lt;det&gt;&lt;def&gt;&lt;GD&gt;&lt;ND&gt;$ right choice for your company.</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>after apertium-transfer:</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>^+Microsoft 365 for business&lt;np&gt;&lt;al&gt;$ ^ser&lt;vbser&gt;&lt;pri&gt;&lt;p3&gt;&lt;sg&gt;$ ^el&lt;det&gt;&lt;def&gt;&lt;f&gt;&lt;sg&gt;$ ... </div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>^+Microsoft 365 for business&lt;np&gt;&lt;al&gt;$ ^ser&lt;vbser&gt;&lt;pri&gt;&lt;p3&gt;&lt;sg&gt;$ ^el&lt;det&gt;&lt;def&gt;&lt;f&gt;&lt;sg&gt;$ ... </div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div> </div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div> </div></td> </tr> </table>

Francis Tyers https://wiki.apertium.org/w/index.php?title=User-based_chunking&diff=72442&oldid=prev 2020-06-21T21:07:57Z

<p></p> <table class="diff diff-contentalign-left" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">← Older revision</td> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">Revision as of 21:07, 21 June 2020</td> </tr><tr> <td colspan="2" class="diff-lineno">Line 25:</td> <td colspan="2" class="diff-lineno">Line 25:</td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>+Microsoft +365 +for +business is right choice for your company</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>+Microsoft +365 +for +business is right choice for your company</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td class="diff-marker" data-marker="−"></td> <td class="diff-deletedline diff-side-deleted"><div>^+Microsoft/Microsoft&lt;np&gt;&lt;al&gt;$ ^+365/365&lt;num&gt;$ ^+for/for&lt;pr&gt;$ ^+business/business$ ^is/be&lt;vbser&gt;&lt;pres&gt;&lt;p3&gt;&lt;sg&gt;$ the right choice for your company</div></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>^+Microsoft/Microsoft&lt;np&gt;&lt;al&gt;$ ^+365/365&lt;num&gt;$ ^+for/for&lt;pr&gt;$ ^+business/business$ ^is/be&lt;vbser&gt;&lt;pres&gt;&lt;p3&gt;&lt;sg&gt;$ <ins class="diffchange diffchange-inline">^</ins>the<ins class="diffchange diffchange-inline">/the&lt;det&gt;&lt;def&gt;&lt;sp&gt;$</ins> right choice for your company</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>^+Microsoft 365 for business/Microsoft 365 for business&lt;np&gt;&lt;al&gt;$ ^is/be&lt;vbser&gt;&lt;pres&gt;&lt;p3&gt;&lt;sg&gt;$ the right choice for your company.</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>^+Microsoft 365 for business/Microsoft 365 for business&lt;np&gt;&lt;al&gt;$ ^is/be&lt;vbser&gt;&lt;pres&gt;&lt;p3&gt;&lt;sg&gt;$ the right choice for your company.</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>^+Microsoft 365 for business/Microsoft 365 for business&lt;np&gt;&lt;al&gt;/Microsoft 365 for business&lt;np&gt;&lt;al&gt;$ ^is/be&lt;vbser&gt;&lt;pres&gt;&lt;p3&gt;&lt;sg&gt;/ser&lt;vbser&gt;&lt;pres&gt;&lt;p3&gt;&lt;sg&gt;$ ^the/the&lt;det&gt;&lt;def&gt;&lt;sp&gt;/el&lt;det&gt;&lt;def&gt;&lt;GD&gt;&lt;ND&gt;$ right choice for your company.</div></td> </tr> <tr> <td class="diff-marker" data-marker="−"></td> <td class="diff-deletedline diff-side-deleted"><div>...</div></td> <td colspan="2" class="diff-empty diff-side-added"></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>^+Microsoft 365 for business&lt;np&gt;&lt;al&gt;$ ^ser&lt;vbser&gt;&lt;pri&gt;&lt;p3&gt;&lt;sg&gt;$ ^el&lt;det&gt;&lt;def&gt;&lt;f&gt;&lt;sg&gt;$ ... </div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div> </div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Microsoft 365 for business és la ...</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>&lt;/pre&gt;</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>&lt;/pre&gt;</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> </table>

Francis Tyers https://wiki.apertium.org/w/index.php?title=User-based_chunking&diff=72441&oldid=prev 2020-06-21T21:03:19Z

<p></p> <table class="diff diff-contentalign-left" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">← Older revision</td> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">Revision as of 21:03, 21 June 2020</td> </tr><tr> <td colspan="2" class="diff-lineno">Line 31:</td> <td colspan="2" class="diff-lineno">Line 31:</td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>...</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>...</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>&lt;/pre&gt;</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>&lt;/pre&gt;</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>; Automatic </div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>For dealing with quoted text we might be better to have an automatic system that does language identification combined with rules. The same technique could be used, words that receive an analysis would get the surface form prefixed with &lt;tt&gt;+&lt;/tt&gt; after running through the module.</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-lineno">Line 36:</td> <td colspan="2" class="diff-lineno">Line 40:</td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>* Discovering these kind of things should be possible using only monolingual corpora, or resources such as Wikipedia.</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>* Discovering these kind of things should be possible using only monolingual corpora, or resources such as Wikipedia.</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>[[Category:Ideas]]</div></td> </tr> </table>

Francis Tyers https://wiki.apertium.org/w/index.php?title=User-based_chunking&diff=72440&oldid=prev 2020-06-21T21:01:36Z

<p>Created page with &quot;Sometimes in translations it would be useful to be able to mark particular segments/chunks not for translation, but to give them an analysis for the purposes of translation....&quot;</p> <p><b>New page</b></p><div>Sometimes in translations it would be useful to be able to mark particular segments/chunks not for translation, but to give them an analysis for the purposes of translation. <br /> <br /> Some use-cases:<br /> <br /> * Product names, long named entities<br /> <br /> ** Microsoft 365 for business is the right choice for your company<br /> ** A very popular film adaptation is &quot;Gone with the Wind&quot;.<br /> <br /> * The use of quoted chunks as modifiers<br /> <br /> * Quotations in a second (or third) language<br /> <br /> ** And then he came back and said &quot;Bu ne lan?&quot;<br /> <br /> ==Solutions==<br /> <br /> ; User-based, online<br /> <br /> 1. A user could mark in an interface that they don&#039;t want a particular span translating, each word would be prefixed then with a symbol. Sequences of these words would be merged by -separable or something like it. And the whole chunk would be either given a default tag, a tag determined by a classifier or a tag determined by rules.<br /> <br /> &lt;pre&gt;<br /> Microsoft 365 for business is right choice for your company<br /> <br /> +Microsoft +365 +for +business is right choice for your company<br /> <br /> ^+Microsoft/Microsoft&lt;np&gt;&lt;al&gt;$ ^+365/365&lt;num&gt;$ ^+for/for&lt;pr&gt;$ ^+business/business$ ^is/be&lt;vbser&gt;&lt;pres&gt;&lt;p3&gt;&lt;sg&gt;$ the right choice for your company<br /> <br /> ^+Microsoft 365 for business/Microsoft 365 for business&lt;np&gt;&lt;al&gt;$ ^is/be&lt;vbser&gt;&lt;pres&gt;&lt;p3&gt;&lt;sg&gt;$ the right choice for your company.<br /> <br /> ...<br /> &lt;/pre&gt;<br /> <br /> <br /> ; Engine-based, offline<br /> <br /> * Discovering these kind of things should be possible using only monolingual corpora, or resources such as Wikipedia.</div>

Francis Tyers