https://wiki.apertium.org/w/index.php?action=history&feed=atom&title=Charlifter 2026-04-27T15:16:59Z Revision history for this page on the wiki MediaWiki 1.34.1 https://wiki.apertium.org/w/index.php?title=Charlifter&diff=66187&oldid=prev 2018-03-10T02:07:22Z

<p>GitHub migration</p> <table class="diff diff-contentalign-left" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">← Older revision</td> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">Revision as of 02:07, 10 March 2018</td> </tr><tr> <td colspan="2" class="diff-lineno">Line 1:</td> <td colspan="2" class="diff-lineno">Line 1:</td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>{{Github-unmigrated-tool}}</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>==Training==</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>==Training==</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> </table>

Shardulc https://wiki.apertium.org/w/index.php?title=Charlifter&diff=28342&oldid=prev 2011-09-05T12:11:16Z

<p></p> <table class="diff diff-contentalign-left" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">← Older revision</td> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">Revision as of 12:11, 5 September 2011</td> </tr><tr> <td colspan="2" class="diff-lineno">Line 88:</td> <td colspan="2" class="diff-lineno">Line 88:</td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>* [https://apertium.svn.sourceforge.net/svnroot/apertium/branches/gsoc2011/charlifter Half-finished C++ port]</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>* [https://apertium.svn.sourceforge.net/svnroot/apertium/branches/gsoc2011/charlifter Half-finished C++ port]</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>* [http://sourceforge.net/projects/lingala/files/charlifter/ SourceForge: Download charlifter]</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>[[Category:Tools]]</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>[[Category:Tools]]</div></td> </tr> </table>

Francis Tyers https://wiki.apertium.org/w/index.php?title=Charlifter&diff=27840&oldid=prev 2011-08-25T11:24:55Z

<p><span dir="auto"><span class="autocomment">Training</span></span></p> <table class="diff diff-contentalign-left" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">← Older revision</td> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">Revision as of 11:24, 25 August 2011</td> </tr><tr> <td colspan="2" class="diff-lineno">Line 84:</td> <td colspan="2" class="diff-lineno">Line 84:</td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>&lt;/pre&gt;</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>&lt;/pre&gt;</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>==See also==</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>* [https://apertium.svn.sourceforge.net/svnroot/apertium/branches/gsoc2011/charlifter Half-finished C++ port]</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>[[Category:Tools]]</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>[[Category:Tools]]</div></td> </tr> </table>

Francis Tyers https://wiki.apertium.org/w/index.php?title=Charlifter&diff=27647&oldid=prev 2011-08-19T13:26:49Z

<p></p> <table class="diff diff-contentalign-left" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">← Older revision</td> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">Revision as of 13:26, 19 August 2011</td> </tr><tr> <td colspan="2" class="diff-lineno">Line 23:</td> <td colspan="2" class="diff-lineno">Line 23:</td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>"Pretty clean" dictionary processed...</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>"Pretty clean" dictionary processed...</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>Reading the training text...</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>Reading the training text...</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>1000...</div></td> </tr> <tr> <td class="diff-marker"><a class="mw-diff-movedpara-left" title="Paragraph was moved. Click to jump to new location." href="#movedpara_3_2_rhs">&#x26AB;</a></td> <td class="diff-deletedline diff-side-deleted"><div><a name="movedpara_2_0_lhs"></a>Training texts processed (<del class="diffchange diffchange-inline">1143</del> words)...</div></td> <td colspan="2" class="diff-empty diff-side-added"></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>2000...</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>3000...</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker"><a class="mw-diff-movedpara-right" title="Paragraph was moved. Click to jump to old location." href="#movedpara_2_0_lhs">&#x26AB;</a></td> <td class="diff-addedline diff-side-added"><div><a name="movedpara_3_2_rhs"></a>Training texts processed (<ins class="diffchange diffchange-inline">35919</ins> words)...</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>Computing final probabilities...</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>Computing final probabilities...</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>Dumping plain text hashes to disk...</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>Dumping plain text hashes to disk...</div></td> </tr> <tr> <td colspan="2" class="diff-lineno">Line 50:</td> <td colspan="2" class="diff-lineno">Line 53:</td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>&lt;pre&gt;</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>&lt;pre&gt;</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>$ apertium-eval-translator -t testing/cv-test.txt -r testing/cv-reference.txt </div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>$ apertium-eval-translator -t testing/cv-test.txt -r testing/cv-reference.txt </div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>apertium-eval-translator -t testing/cv-test.txt -r testing/cv-reference.txt </div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>Test file: 'testing/cv-test.txt'</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>Test file: 'testing/cv-test.txt'</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>Reference file 'testing/cv-reference.txt'</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>Reference file 'testing/cv-reference.txt'</div></td> </tr> <tr> <td colspan="2" class="diff-lineno">Line 62:</td> <td colspan="2" class="diff-lineno">Line 66:</td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>Results when removing unknown-word marks (stars)</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>Results when removing unknown-word marks (stars)</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>-------------------------------------------------------</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>-------------------------------------------------------</div></td> </tr> <tr> <td class="diff-marker" data-marker="−"></td> <td class="diff-deletedline diff-side-deleted"><div>Edit distance: <del class="diffchange diffchange-inline">451</del></div></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Edit distance: <ins class="diffchange diffchange-inline">108</ins></div></td> </tr> <tr> <td class="diff-marker" data-marker="−"></td> <td class="diff-deletedline diff-side-deleted"><div>Word error rate (WER): <del class="diffchange diffchange-inline">56</del>.<del class="diffchange diffchange-inline">09</del> %</div></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Word error rate (WER): <ins class="diffchange diffchange-inline">13</ins>.<ins class="diffchange diffchange-inline">43</ins> %</div></td> </tr> <tr> <td class="diff-marker" data-marker="−"></td> <td class="diff-deletedline diff-side-deleted"><div>Number of position-independent word errors: <del class="diffchange diffchange-inline">451</del></div></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Number of position-independent word errors: <ins class="diffchange diffchange-inline">108</ins></div></td> </tr> <tr> <td class="diff-marker" data-marker="−"></td> <td class="diff-deletedline diff-side-deleted"><div>Position-independent word error rate (PER): <del class="diffchange diffchange-inline">56</del>.<del class="diffchange diffchange-inline">09</del> %</div></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Position-independent word error rate (PER): <ins class="diffchange diffchange-inline">13</ins>.<ins class="diffchange diffchange-inline">43</ins> %</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>Results when unknown-word marks (stars) are not removed</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>Results when unknown-word marks (stars) are not removed</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>-------------------------------------------------------</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>-------------------------------------------------------</div></td> </tr> <tr> <td class="diff-marker" data-marker="−"></td> <td class="diff-deletedline diff-side-deleted"><div>Edit distance: <del class="diffchange diffchange-inline">451</del></div></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Edit distance: <ins class="diffchange diffchange-inline">108</ins></div></td> </tr> <tr> <td class="diff-marker" data-marker="−"></td> <td class="diff-deletedline diff-side-deleted"><div>Word Error Rate (WER): <del class="diffchange diffchange-inline">56</del>.<del class="diffchange diffchange-inline">09</del> %</div></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Word Error Rate (WER): <ins class="diffchange diffchange-inline">13</ins>.<ins class="diffchange diffchange-inline">43</ins> %</div></td> </tr> <tr> <td class="diff-marker" data-marker="−"></td> <td class="diff-deletedline diff-side-deleted"><div>Number of position-independent word errors: <del class="diffchange diffchange-inline">451</del></div></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Number of position-independent word errors: <ins class="diffchange diffchange-inline">108</ins></div></td> </tr> <tr> <td class="diff-marker" data-marker="−"></td> <td class="diff-deletedline diff-side-deleted"><div>Position-independent word error rate (PER): <del class="diffchange diffchange-inline">56</del>.<del class="diffchange diffchange-inline">09</del> %</div></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Position-independent word error rate (PER): <ins class="diffchange diffchange-inline">13</ins>.<ins class="diffchange diffchange-inline">43</ins> %</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>Statistics about the translation of unknown words</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>Statistics about the translation of unknown words</div></td> </tr> <tr> <td colspan="2" class="diff-lineno">Line 78:</td> <td colspan="2" class="diff-lineno">Line 82:</td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>Number of unknown words which were free rides: 0</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>Number of unknown words which were free rides: 0</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>Percentage of unknown words that were free rides: 0%</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>Percentage of unknown words that were free rides: 0%</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><br /></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>&lt;/pre&gt;</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>&lt;/pre&gt;</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> </table>

Francis Tyers https://wiki.apertium.org/w/index.php?title=Charlifter&diff=27633&oldid=prev 2011-08-19T03:53:55Z

<p></p> <table class="diff diff-contentalign-left" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">← Older revision</td> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">Revision as of 03:53, 19 August 2011</td> </tr><tr> <td colspan="2" class="diff-lineno">Line 8:</td> <td colspan="2" class="diff-lineno">Line 8:</td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>* &lt;code&gt;XX-clean.txt&lt;/code&gt;: A file with a list of known clean words</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>* &lt;code&gt;XX-clean.txt&lt;/code&gt;: A file with a list of known clean words</div></td> </tr> <tr> <td class="diff-marker" data-marker="−"></td> <td class="diff-deletedline diff-side-deleted"><div>* &lt;code&gt;XX-prettyclean.txt&lt;/code&gt;: A file with a list of probably clean words (can be <del class="diffchange diffchange-inline">really small</del>)</div></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>* &lt;code&gt;XX-prettyclean.txt&lt;/code&gt;: A file with a list of probably clean words (can be <ins class="diffchange diffchange-inline">empty</ins>)</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>;Process</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>;Process</div></td> </tr> </table>

Francis Tyers https://wiki.apertium.org/w/index.php?title=Charlifter&diff=27613&oldid=prev 2011-08-18T16:57:44Z

<p></p> <table class="diff diff-contentalign-left" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">← Older revision</td> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">Revision as of 16:57, 18 August 2011</td> </tr><tr> <td colspan="2" class="diff-lineno">Line 1:</td> <td colspan="2" class="diff-lineno">Line 1:</td> </tr> <tr> <td class="diff-marker" data-marker="−"></td> <td class="diff-deletedline diff-side-deleted"><br /></td> <td colspan="2" class="diff-empty diff-side-added"></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>==Training==</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>==Training==</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-lineno">Line 45:</td> <td colspan="2" class="diff-lineno">Line 44:</td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>&lt;/pre&gt;</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>&lt;/pre&gt;</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>;Evaluation</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>You can use [[apertium-eval-translator]] to evaluate the quality of the restoration:</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>&lt;pre&gt;</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>$ apertium-eval-translator -t testing/cv-test.txt -r testing/cv-reference.txt </div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Test file: 'testing/cv-test.txt'</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Reference file 'testing/cv-reference.txt'</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Statistics about input files</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>-------------------------------------------------------</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Number of words in reference: 804</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Number of words in test: 804</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Number of unknown words (marked with a star) in test: </div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Percentage of unknown words: 0.00 %</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Results when removing unknown-word marks (stars)</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>-------------------------------------------------------</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Edit distance: 451</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Word error rate (WER): 56.09 %</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Number of position-independent word errors: 451</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Position-independent word error rate (PER): 56.09 %</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Results when unknown-word marks (stars) are not removed</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>-------------------------------------------------------</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Edit distance: 451</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Word Error Rate (WER): 56.09 %</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Number of position-independent word errors: 451</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Position-independent word error rate (PER): 56.09 %</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Statistics about the translation of unknown words</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>-------------------------------------------------------</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Number of unknown words which were free rides: 0</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Percentage of unknown words that were free rides: 0%</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>&lt;/pre&gt;</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>[[Category:Tools]]</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>[[Category:Tools]]</div></td> </tr> </table>

Francis Tyers https://wiki.apertium.org/w/index.php?title=Charlifter&diff=27612&oldid=prev 2011-08-18T16:56:29Z

<p></p> <table class="diff diff-contentalign-left" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">← Older revision</td> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">Revision as of 16:56, 18 August 2011</td> </tr><tr> <td colspan="2" class="diff-lineno">Line 1:</td> <td colspan="2" class="diff-lineno">Line 1:</td> </tr> <tr> <td class="diff-marker" data-marker="−"></td> <td class="diff-deletedline diff-side-deleted"><br /></td> <td colspan="2" class="diff-empty diff-side-added"></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>==Training==</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>==Training==</div></td> </tr> <tr> <td colspan="2" class="diff-lineno">Line 20:</td> <td colspan="2" class="diff-lineno">Line 19:</td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>&lt;pre&gt;</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>&lt;pre&gt;</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>$ cat cv-training.crp | perl sf.pl -t -l cv</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>$ cat cv-training.crp | perl sf.pl -t -l cv</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Reading the clean dictionary...</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Clean dictionary processed...</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Reading the "pretty clean" dictionary...</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>"Pretty clean" dictionary processed...</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Reading the training text...</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Training texts processed (1143 words)...</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Computing final probabilities...</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Dumping plain text hashes to disk...</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>&lt;/pre&gt;</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Then:</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>&lt;pre&gt;</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>$ perl sf.pl -m -l cv</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Reading in plain text hashes...</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Saving storable hashes to disk...</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><div>&lt;/pre&gt;</div></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><div>&lt;/pre&gt;</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>Now you should be able to use the program.</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>;Usage</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><br /></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>&lt;pre&gt;</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>$ cat testing/cv-source.txt | perl sf.pl -r -d . -l cv</div></td> </tr> <tr> <td colspan="2" class="diff-empty diff-side-deleted"></td> <td class="diff-marker" data-marker="+"></td> <td class="diff-addedline diff-side-added"><div>&lt;/pre&gt;</div></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> <tr> <td class="diff-marker"></td> <td class="diff-context diff-side-deleted"><br /></td> <td class="diff-marker"></td> <td class="diff-context diff-side-added"><br /></td> </tr> </table>

Francis Tyers https://wiki.apertium.org/w/index.php?title=Charlifter&diff=27611&oldid=prev 2011-08-18T16:52:46Z

<p>Created page with &#039; ==Training== You need a corpus: * &lt;code&gt;XX-corpus.txt&lt;/code&gt;: A clean (orthographically correct) corpus in the language You need two wordlists: * &lt;code&gt;XX-clean.txt&lt;/code&gt;:…&#039;</p> <p><b>New page</b></p><div><br /> <br /> ==Training==<br /> <br /> You need a corpus:<br /> <br /> * &lt;code&gt;XX-corpus.txt&lt;/code&gt;: A clean (orthographically correct) corpus in the language<br /> <br /> You need two wordlists:<br /> <br /> * &lt;code&gt;XX-clean.txt&lt;/code&gt;: A file with a list of known clean words<br /> * &lt;code&gt;XX-prettyclean.txt&lt;/code&gt;: A file with a list of probably clean words (can be really small)<br /> <br /> ;Process<br /> <br /> First do &#039;make&#039;, you might want to edit the &lt;code&gt;makefile&lt;/code&gt; to change your paths.<br /> <br /> Then, put the source files (corpus, wordlists) where &lt;code&gt;sf.pl&lt;/code&gt; can find them, and run:<br /> <br /> &lt;pre&gt;<br /> $ cat cv-training.crp | perl sf.pl -t -l cv<br /> <br /> &lt;/pre&gt;<br /> <br /> <br /> <br /> <br /> [[Category:Tools]]</div>

Francis Tyers