Difference between revisions of "Wiki regression testing"
Popcorndude (talk | contribs) |
|||
(13 intermediate revisions by 8 users not shown) | |||
Line 1: | Line 1: | ||
{{TOCD}} |
{{TOCD}} |
||
{{Github-migration-check}} |
|||
⚫ | '''Regression testing'''<ref>See the Wikipedia article [http://en.wikipedia.org/wiki/Regression_testing Regression testing]</ref> is |
||
<br/> |
|||
'''Wiki-based regression tests have been superseded by [[Apertium-regtest]].''' |
|||
<br/> |
|||
⚫ | |||
This is most useful when you are working on a young/new language pair, where you are working with rules or dictionaries and want to change a rule or word and get an overview of the consequences, and check that your changes haven't broken anything. |
|||
Most mature pairs abandon these tests (see discussion [https://plus.google.com/u/0/114804369744409916883/posts/JvozgQxVAyT here]) and use [[testvoc]] and [[corpus test]] to test improvements in the translator, although regression tests described here have an advantage of being deterministic - in the sense that they either pass or fail - whereas it's hard to achieve such determinism in case of a corpus test (especially if the corpus is large). |
|||
The regressions tests work by downloading a wiki page with the phrases and the expected output like the page [[English_and_Esperanto/Regression_tests]] and then running the translator. |
The regressions tests work by downloading a wiki page with the phrases and the expected output like the page [[English_and_Esperanto/Regression_tests]] and then running the translator. |
||
==Installation and invocation== |
==Installation and invocation== |
||
You could simply copy the script <code>regression-tests.sh</code> from an existing language pair and change the names in it. |
|||
To start, type <code>sh regression-tests.sh</code> on a Unix box. |
|||
See https://github.com/unhammer/apertium-wiki-tests for how to set up your language pair with wiki-based regression tests. |
|||
;Troubleshooting |
|||
You should keep the test outputs (t/latest-{regression,pending}.results) on Github, that way you can tell more easily where regressions were introduced. |
|||
If you get output like: <code>regression-tests.sh: 28: [[: not found</code> then try <code>bash regression-tests.sh</code> or make the script executable (<code>chmod a+x regression-tests.sh</code>) and execute it by <code>./regression-tests.sh</code> . |
|||
==Output== |
==Output== |
||
Line 68: | Line 77: | ||
* [[Special:Search/Regression]] for some examples of regression tests. |
* [[Special:Search/Regression]] for some examples of regression tests. |
||
==References== |
|||
<references/> |
|||
[[Category:Development]] |
[[Category:Development]] |
||
[[Category:Quality control]] |
|||
[[Category:Evaluation]] |
|||
[[Category:Documentation in English]] |
Latest revision as of 20:31, 23 July 2021
This page is out of date as a result of the migration to GitHub. Please update this page with new documentation and remove this warning. If you are unsure how to proceed, please contact the GitHub migration team.
Wiki-based regression tests have been superseded by Apertium-regtest.
The term Regression testing[1] is used in Apertium to describe a way of testing a language pair, by translating a collection of "test phrases" and compare the result to the expected output.
This is most useful when you are working on a young/new language pair, where you are working with rules or dictionaries and want to change a rule or word and get an overview of the consequences, and check that your changes haven't broken anything.
Most mature pairs abandon these tests (see discussion here) and use testvoc and corpus test to test improvements in the translator, although regression tests described here have an advantage of being deterministic - in the sense that they either pass or fail - whereas it's hard to achieve such determinism in case of a corpus test (especially if the corpus is large).
The regressions tests work by downloading a wiki page with the phrases and the expected output like the page English_and_Esperanto/Regression_tests and then running the translator.
Installation and invocation[edit]
See https://github.com/unhammer/apertium-wiki-tests for how to set up your language pair with wiki-based regression tests.
You should keep the test outputs (t/latest-{regression,pending}.results) on Github, that way you can tell more easily where regressions were introduced.
Output[edit]
For each test, the scipt prints 1) source language 2) expected (correct) output 3) actual output from Apertium. In some test scripts it will only print out the "failed" test (where the expected and actual output is diverging), and in other test scripts it will print out all of the tests, and a summary.
All printing style[edit]
- Example
$ sh regression-tests.sh sv-da Jag vill gå en tur WORKS Jeg vil gå en tur sv-da Du vill gå en tur WORKS Du vil gå en tur .... sv-da Maten är äten - Maden er spist + #Mad er spist sv-da Äpplet är ätet WORKS Æblet er spist 22 / 23 ~95.65%
No-printing style[edit]
- Example
en-eo In the 'Boards' section you can change the list of activities. - En la sekcio 'Tabuloj' vi povas ŝanĝi la liston de aktivecoj. + En la 'Tabuloj' sekcio vi povas ŝanĝi la listo de aktivecoj. en-eo Just untoggle them in the treeview. - Simple malelektu ilin en la arbaspekto. + Nur *untoggle ilin en la *treeview. en-eo You can save multiple configurations, and switch between them easily. - Vi povas sekurigi multajn agordojn, kaj ŝalti inter ili facile. + Vi povas savi *multiple *configurations, kaj ŝanĝo inter ilin facile. en-eo You can add multiple profiles, with different lists of boards, and different languages. - Vi povas aldoni plurajn profilojn, kun diversaj listoj de tabuloj, kaj diversaj lingvoj. + Vi povas aldoni *multiple profiloj, kun diferencaj listoj de tabuloj, kaj diferencaj langoj.
See also[edit]
- Special:Search/Regression for some examples of regression tests.
References[edit]
- ↑ See the Wikipedia article Regression testing