User:Khannatanmai/GSoC2020 Final Report - Revision history

Khannatanmai: /* Transfer (Pull Request 1, Pull Request 2, Commit 1, Commit 2, Commit 3, Pull Request 3) */

2020-08-30T09:53:14Z

Transfer (Pull Request 1, Pull Request 2, Commit 1, Commit 2, Commit 3, Pull Request 3)

Khannatanmai: /* Transfer (Pull Request 1, Pull Request 2, Commit 1, Commit 2, Commit 3) */

2020-08-29T09:06:41Z

Transfer (Pull Request 1, Pull Request 2, Commit 1, Commit 2, Commit 3)

Khannatanmai: /* Transfer (Pull Request 1, Pull Request 2, Commit 1, Commit 2) */

2020-08-27T12:12:16Z

Transfer (Pull Request 1, Pull Request 2, Commit 1, Commit 2)

Khannatanmai: /* Postgeneration (Pull Request) */

2020-08-26T13:21:19Z

Postgeneration (Pull Request)

Khannatanmai: /* Method */

2020-08-23T21:26:25Z

Method

Khannatanmai: /* Method */

2020-08-23T21:26:00Z

Method

Khannatanmai: /* The solution */

2020-08-23T21:25:16Z

The solution

Khannatanmai: /* The solution */

2020-08-23T21:04:34Z

The solution

Khannatanmai: /* Method */

2020-08-23T06:56:48Z

Method

Khannatanmai: /* Method */

2020-08-23T05:59:16Z

Method

@@ Line 94: / Line 94: @@
 Here is a list of changes made to the Apertium codebase for this project during GSoC 2020. The progress was recorded systematically during the summer [[User:Khannatanmai/GSoC2020Progress|here]].
-== Transfer ([https://github.com/apertium/apertium/pull/90 Pull Request 1], [https://github.com/apertium/apertium/pull/94 Pull Request 2], [https://github.com/apertium/apertium/commit/4b4d930011c080ae1aceec044402141a2635c0d3 Commit 1], [https://github.com/apertium/apertium/commit/ad78aa15b8101449e4d38430710e621ff1206378 Commit 2], [https://github.com/apertium/apertium/commit/c410a1bb39d5dd436d63c6f68f57b162056d8710 Commit 3], [https://github.com/apertium/apertium/pull/102 Pull Request 3]) ==
+== Transfer ([https://github.com/apertium/apertium/pull/90 Pull Request 1], [https://github.com/apertium/apertium/pull/94 Pull Request 2], [https://github.com/apertium/apertium/commit/4b4d930011c080ae1aceec044402141a2635c0d3 Commit 1], [https://github.com/apertium/apertium/commit/ad78aa15b8101449e4d38430710e621ff1206378 Commit 2], [https://github.com/apertium/apertium/commit/c410a1bb39d5dd436d63c6f68f57b162056d8710 Commit 3], [https://github.com/apertium/apertium/pull/102 Pull Request 3], [https://github.com/apertium/apertium/commit/e67343b1d8843689198db4fd282a4357109a2664 Commit 4]) ==
 === Chunker/Single-stage transfer ===
 * Wordbound blanks are a part of transfer word as a new side: blank.

@@ Line 94: / Line 94: @@
 Here is a list of changes made to the Apertium codebase for this project during GSoC 2020. The progress was recorded systematically during the summer [[User:Khannatanmai/GSoC2020Progress|here]].
-== Transfer ([https://github.com/apertium/apertium/pull/90 Pull Request 1], [https://github.com/apertium/apertium/pull/94 Pull Request 2], [https://github.com/apertium/apertium/commit/4b4d930011c080ae1aceec044402141a2635c0d3 Commit 1], [https://github.com/apertium/apertium/commit/ad78aa15b8101449e4d38430710e621ff1206378 Commit 2], [https://github.com/apertium/apertium/commit/c410a1bb39d5dd436d63c6f68f57b162056d8710 Commit 3]) ==
+== Transfer ([https://github.com/apertium/apertium/pull/90 Pull Request 1], [https://github.com/apertium/apertium/pull/94 Pull Request 2], [https://github.com/apertium/apertium/commit/4b4d930011c080ae1aceec044402141a2635c0d3 Commit 1], [https://github.com/apertium/apertium/commit/ad78aa15b8101449e4d38430710e621ff1206378 Commit 2], [https://github.com/apertium/apertium/commit/c410a1bb39d5dd436d63c6f68f57b162056d8710 Commit 3], [https://github.com/apertium/apertium/pull/102 Pull Request 3]) ==
 === Chunker/Single-stage transfer ===
 * Wordbound blanks are a part of transfer word as a new side: blank.
@@ Line 106: / Line 106: @@
 * If rule pattern has only one LU, the wordbound blank gets output with all output LUs of the rule
 * When using apertium-transfer -n, the wblanks print as they're supposed to.
-* Fix null flushing
+* Fix null flushing in transfer
+* Store blanks in queue and output wherever user has <pre></b></pre> in the rule output, so that users don't have to define a blank position anymore. (Blank reordering is done by wordbound blanks now)
 === Interchunk ===
 * No change needed as inter chunk doesn't access LUs inside the chunk.
+* Blank handling changed so the user doesn't have to worry about the blank position anymore.
 === Postchunk ===
@@ Line 120: / Line 122: @@
 * Tests added
 * If rule pattern chunk has only one LU, the wordbound blank gets output with all output LUs of the rule
+* Blank handling changed so the user doesn't have to worry about the blank position anymore.
 == Recursive Transfer ([https://github.com/apertium/apertium-recursive/pull/65 Pull Request]) ==

@@ Line 94: / Line 94: @@
 Here is a list of changes made to the Apertium codebase for this project during GSoC 2020. The progress was recorded systematically during the summer [[User:Khannatanmai/GSoC2020Progress|here]].
-== Transfer ([https://github.com/apertium/apertium/pull/90 Pull Request 1], [https://github.com/apertium/apertium/pull/94 Pull Request 2], [https://github.com/apertium/apertium/commit/4b4d930011c080ae1aceec044402141a2635c0d3 Commit 1], [https://github.com/apertium/apertium/commit/ad78aa15b8101449e4d38430710e621ff1206378 Commit 2]) ==
+== Transfer ([https://github.com/apertium/apertium/pull/90 Pull Request 1], [https://github.com/apertium/apertium/pull/94 Pull Request 2], [https://github.com/apertium/apertium/commit/4b4d930011c080ae1aceec044402141a2635c0d3 Commit 1], [https://github.com/apertium/apertium/commit/ad78aa15b8101449e4d38430710e621ff1206378 Commit 2], [https://github.com/apertium/apertium/commit/c410a1bb39d5dd436d63c6f68f57b162056d8710 Commit 3]) ==
 === Chunker/Single-stage transfer ===
 * Wordbound blanks are a part of transfer word as a new side: blank.
@@ Line 106: / Line 106: @@
 * If rule pattern has only one LU, the wordbound blank gets output with all output LUs of the rule
 * When using apertium-transfer -n, the wblanks print as they're supposed to.
+* Fix null flushing
 === Interchunk ===

@@ Line 147: / Line 147: @@
 * Can be accessed by class member: <code>LexicalUnit.wordbound_blank</code>.
-== Postgeneration ([https://github.com/apertium/lttoolbox/pull/102 Pull Request]) ==
+== Postgeneration ([https://github.com/apertium/lttoolbox/pull/102 Pull Request], [https://github.com/apertium/lttoolbox/commit/3706d959b657b53dd094a75ac336e24a8f1739b2 Commit 1], [https://github.com/apertium/lttoolbox/commit/61d7c0d8f8e4ab4a5ad33f44db4dea60dc0a2422 Commit 2]) ==
 * Wordbound blanks merge when words merge.
 * Wordbound blanks apply to all output words when output of postgen rule are more than input words.

@@ Line 87: / Line 87: @@
 Or, just run:
 <pre>
+$ echo "<i>El perro</i> <b>blanco</b>" | apertium -f html -d . spa-eng
+<i>The</i> <b>white</b> <i>dog</i>
 </pre>

@@ Line 20: / Line 20: @@
 <pre>[[wordboundblank]]word[[/]] word2 word3 [[wordboundblank]]word4[[/]]</pre>
+The commands to translate formatted input were updated to use the modified modules and the new deformatters and reformatters.
+<pre>
+$ echo "<i>El perro</i> <b>blanco</b>" | apertium -f html -d . spa-eng
+<i>The</i> <b>white</b> <i>dog</I>
+</pre>
 == Method ==

@@ Line 24: / Line 24: @@
 A new deformatter and reformatter was written, that converts markup into either superblanks or wordbound blanks, and '''each module in the entire pipeline was modified to work with wordbound blanks, such that in any module that moves, deletes, adds, merges, splits, etc. words and phrases will also do the same to corresponding wordbound blanks.'''
-When wordbound blanks are on text that isn't tokenised using LUs, there's a closing wordbound blank <pre>[[/]]</pre> to limit it's span. After the input is tokenised by the analyser, the program <code>apertium-wblank-attach</code> attaches wordbound blanks to all the LUs inside the span of that wordbound blank and removes the closing wordbound blanks from the stream. Right before the generator, the opposite process happens with the <code>apertium-wblank-detach</code> - closing wordbound blanks are put in the stream again in preparation for the reformatter.
+When wordbound blanks are on text that isn't tokenised using LUs, there's a closing wordbound blank to limit it's span. After the input is tokenised by the analyser, the program <code>apertium-wblank-attach</code> attaches wordbound blanks to all the LUs inside the span of that wordbound blank and removes the closing wordbound blanks from the stream. Right before the generator, the opposite process happens with the <code>apertium-wblank-detach</code> - closing wordbound blanks are put in the stream again in preparation for the reformatter.
 Here's a snippet of the process:

@@ Line 60: / Line 60: @@
 Here is the full pipeline:
 <pre>
-$ echo "<i>El perro</i> <b>blanco</b>"      |
+$ echo "<i>El perro</i> <b>blanco</b>"   |
  tf-extract                              |
  lt-proc -z 'spa-eng.automorf.bin'.      |