Testvoc - Revision history

Firespeaker: /* Testvoc "lite" */

2021-01-18T22:30:00Z

Testvoc "lite"

Firespeaker: /* Generation testvoc with HFST analyser */

2021-01-18T22:29:35Z

Generation testvoc with HFST analyser

Dharjunior at 13:37, 30 November 2018

2018-11-30T13:37:43Z

Dharjunior at 00:16, 30 November 2018

2018-11-30T00:16:33Z

Francis Tyers at 21:59, 7 March 2018

2018-03-07T21:59:13Z

Unhammer: /* Generation testvoc with lttoolbox analyser */

2016-03-17T10:44:50Z

Generation testvoc with lttoolbox analyser

Unhammer: /* HFST-based testvoc of lttoolbox analyser */

2016-03-14T12:29:19Z

HFST-based testvoc of lttoolbox analyser

Unhammer: /* HFST-based testvoc of lttoolbox analyser */

2016-03-14T11:43:07Z

HFST-based testvoc of lttoolbox analyser

Unhammer: /* HFST-based testvoc of lttoolbox analyser */

2016-03-14T11:42:45Z

HFST-based testvoc of lttoolbox analyser

Unhammer: /* HFST-based testvoc of lttoolbox analyser */

2016-03-14T11:41:30Z

HFST-based testvoc of lttoolbox analyser

@@ Line 57: / Line 57: @@
 </pre>
-== Testvoc "lite" ==
+=== Testvoc "lite" ===
 A somewhat more up-to-date approach is used in [https://github.com/apertium/apertium-kaz-kir/tree/master/testvoc/lite apertium-kaz-kir/testvoc/lite], based on [https://github.com/apertium/apertium-kaz/tree/master/tests/morphotactics apertium-kaz/tests/morphotactics].  Also see [https://github.com/apertium/apertium-uzb-kaa/tree/master/testvoc/lite apertium-uzb-kaa/testvoc/lite].

@@ Line 57: / Line 57: @@
 </pre>
+== Testvoc "lite" ==
+A somewhat more up-to-date approach is used in [https://github.com/apertium/apertium-kaz-kir/tree/master/testvoc/lite apertium-kaz-kir/testvoc/lite], based on [https://github.com/apertium/apertium-kaz/tree/master/tests/morphotactics apertium-kaz/tests/morphotactics].  Also see [https://github.com/apertium/apertium-uzb-kaa/tree/master/testvoc/lite apertium-uzb-kaa/testvoc/lite].
 ==Words in bidix but not in analyser==

@@ Line 1: / Line 1: @@
 [[Test de vocabulaire|En français]]
-{{Github-migration-check}}
 {{TOCD}}
 A '''testvoc''' is literally a test of vocabulary. At the most basic level, it just expands an {{sc|sl}} dictionary, and runs each possibly analysed [[lexical form]] through all the translation stages to see that for each possible input, a sensible translation in the {{sc|tl}}, without <code>#</code>, or <code>@</code> symbols is generated.

@@ Line 22: / Line 22: @@
 ===Generation testvoc with lttoolbox analyser===
 The script generation.sh in
-https://svn.code.sf.net/p/apertium/svn/trunk/apertium-swe-dan/dev/testvoc/ should work with any pipeline that uses lttoolbox on the analysis side.
+https://github.com/apertium/apertium-swe-dan/blob/master/dev/testvoc/generation.sh should work with any pipeline that uses lttoolbox on the analysis side.
 It tests that anything the analyser can produce will go through to generation without '/' or '#'-marks (that is, there is one and only one form generated for anything the analyser can produce).
@@ Line 44: / Line 44: @@
 which may be a more "complete" testvoc.
-Running https://svn.code.sf.net/p/apertium/svn/trunk/apertium-swe-dan/dev/testvoc/generation.sh with --hfst as the first argument will make it use this method.
+Running https://github.com/apertium/apertium-swe-dan/blob/master/dev/testvoc/generation.sh with --hfst as the first argument will make it use this method.
 ===Generation testvoc with HFST analyser===
-The Tatar-Bashkir language pair has a testvoc script for use with HFST, see https://sourceforge.net/p/apertium/svn/HEAD/tree/nursery/apertium-tat-bak/dev/inconsistency.sh which contains e.g.
+The Tatar-Bashkir language pair has a testvoc script for use with HFST, see https://github.com/apertium/apertium-tat-bak/blob/master/dev/inconsistency.sh which contains e.g.
 <pre>
 hfst-fst2strings ../.deps/ba.LR-debug.hfst | sort -u |  sed 's/:/%/g' | cut -f1 -d'%' |  sed 's/^/^/g' | sed 's/$/$ ^.<sent>$/g' | tee $TMPDIR/tmp_testvoc1.txt |
@@ Line 61: / Line 61: @@
 ==Words in bidix but not in analyser==
+The script bidix-unknowns.sh in https://github.com/apertium/apertium-swe-dan/blob/master/dev/testvoc/ will look for entries in bidix that your analyser would never produce. It should work with any pipeline that uses lttoolbox on the analysis side.
-The script bidix-unknowns.sh in
-https://svn.code.sf.net/p/apertium/svn/trunk/apertium-swe-dan/dev/testvoc/ will look for entries in bidix that your analyser would never produce. It should work with any pipeline that uses lttoolbox on the analysis side.
 This is useful for making sure all your hard bidix work is actually useful. It may find lemmas that are completely missing from the analyser, or that simply have the wrong gender-tag or similar.

@@ Line 1: / Line 1: @@
 [[Test de vocabulaire|En français]]
+{{Github-migration-check}}
 {{TOCD}}
 A '''testvoc''' is literally a test of vocabulary. At the most basic level, it just expands an {{sc|sl}} dictionary, and runs each possibly analysed [[lexical form]] through all the translation stages to see that for each possible input, a sensible translation in the {{sc|tl}}, without <code>#</code>, or <code>@</code> symbols is generated.

@@ Line 39: / Line 39: @@
 If we call that command "expand", then the full testvoc pipeline would be something like
 <pre>
-expand | sed 's/.*/^&$/g' | apertium-pretransfer | apertium-transfer …bin …t1x | lt-proc -d …autogen.bin
+expand | sed 's/^/^/;s/$/$/' | apertium-pretransfer | apertium-transfer …bin …t1x | lt-proc -d …autogen.bin
 </pre>

@@ Line 33: / Line 33: @@
 Another way to testvoc a trimmed analyser, if you have [[HFST]] installed, is to replace <code>lt-expand ana.dix</code> in a simple testvoc pipeline with this sequence:
 <pre>
-lt-print trimmed-analyser.bin |sed 's/ /@_SPACE_@/g' | hfst-txt2fst -e ε | hfst-fst2strings -c0 | sed 's/^.*://'
+lt-print trimmed-analyser.bin |sed 's/ /@_SPACE_@/g' | hfst-txt2fst -e ε | hfst-project -p lower | hfst-fst2strings -c0
 </pre>
 (The -c0 says to never follow cycles; you can also follow them at most once with -c1 etc., but this can take a while depending on how many {{tag|re}}'s you use.)