Difference between revisions of "Apertium-dixtools"
|Line 31:||Line 31:|
$ svn co https://
$ svn co https://svn..net//apertium/trunk/apertium-dixtools
Revision as of 11:36, 21 November 2013
Apertium-dixtools is a shell command used to execute different tools written in java. These tools permit to do automatic process on a dictionary file (or sometimes several dictionary files).
apertium-dixtools task options parameters (expecially dictionaries filenames as parameters)
As availlable task, we can find :
- Format dictionaries for formatting word definitions/translations, une word per line.
- Sort a dictionary words are more easy to find for a human developper if they are in alphabetic order.
- Dictionary reader to get list of elements (lemmas , paradigms, definitions ... from a dictionary.
- Reverse a dictionary not necessary to be able to translate on both sides, but can be usefull for the task following.
- Crossdics more precisely à bilingual dictionary for languages A and C is built from dictionaries for A-B and B-C language pairs.
- Dictionary coverage to make statistics about how often differents word of a dictionary are used.
- Autoconcord to make bidixes concord with the monodices when there are differences (gender, number, ...) between the two languages.
- Dixtools: Equivalent paradigms to find unused paradigms and paradigm working the same as another in a dictionary.
- Dixtools: Merge dictionaries permit to merge list of words from several monodices.
$ sudo apt-get install ant sun-java6-jdk
$ svn co https://svn.code.sf.net/p/apertium/svn/trunk/apertium-dixtools
$ cd apertium-dixtools $ ant jar
You can also build and install using Maven 2 (http://maven.apache.org), by typing:
- If you update from SVN its always a good idea to do 'ant clean' first.
- 'ant jar' also attempts to do some testing of itself. This might fail, if someone made changes without ensuring that the tests runs. Just continue with installation and report the test failures to the list.
If you get an "The J2SE Platform is not correctly set up." error with property "platforms.default_platform.home" is not found, then try
$ ant -Dplatforms.default_platform.home=/usr jar
or if it f.ex. says error with property "platforms.JDK_1.6.home" is not found, and you want to point to a specific Java version, then try
$ ant -Dplatforms.JDK_1.6.home=/usr/lib/jvm/java-6-sun jar
(On Mac: if you want to put the full "/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/" (or whatever) path in there, first make a symlink from the .../1.6.0/Commands folder to .../1.6.0/bin, since ant expects javac to be in the bin-subdirectory of platforms...home)
The testing is quite verbose. The output looks like:
-do-test-run: [junit] Testsuite: dictools.CrossDictTest [junit]  Loading bilingual AB (regression_test_data/crossdict/input/apertium-es-ca.es-ca.dix) [junit] Reading file regression_test_data/crossdict/input/apertium-es-ca.es-ca.dix ... 200 lines of text [junit] ------------- ---------------- --------------- [junit] Testsuite: misc.eoen.SubstractBidixTest [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.073 sec [junit] [junit] ------------- Standard Error ----------------- [junit] checkEarlierAndRestrict [junit] ------------- ---------------- --------------- [junit] checkEarlierAndRestrict
If the test fails
If you get somethink like
[junit] ------------- ---------------- --------------- test-report: -post-test-run: BUILD FAILED /home/j/esperanto/apertium/apertium-dixtools/nbproject/build-impl.xml:595: Some tests failed; see details above.
... then the program was correctly build, but it didn't pass the tests. It can still be installed and will probably run fine.
The most probably reason (apart from someone has changed the program in a way that breaks the tests), is that your'e not using Unicode. Try writing
(or some other Unicode language installed - I use eo.UTF-8) and run the tests again. If that doesent help please report it on the mailing list.
$ sudo ant install
Usage: apertium-dixtools [task] [generic options] [task parameters] ...
Tasks: cross: cross 2 language pairs (using linguistic res. XML file - see Cross Model) cross-param: cross 2 language pairs (using command line parameters) Crossdics merge-morph: merges two morphological dictionaries (monodix) Merge dictionaries equiv-paradigms: finds equivalent paradigms and updates references list: lists entries in a dictionary - see Dictionary reader dix2trie: create a Trie from an existing bilingual dictionary dix2tiny: create data for mobile platforms (j2me, palm) from bidix reverse-bil: reverses a bilingual dictionary sort: sorts (and groups by category) a dictionary - see Sort a dictionary format: Format dictionaries (according to Generic Options) fix: fix a dictionary (remove duplicates, convert spaces)
For help on a task, invoke it without parameters
Generic options: (mostly for tasks that outputs dix files)
-debug print extra debugging information -noProcComments don't add processing comments (telling what was done) -stripEmptyLines removes empty lines (originating from original file) -alignBidix align a bidix (<p> or <i> at col 10, <r> at col 55) -alignMonodix align a monodix (pardef 10, 30, other entries 25, 45) -align [[E] P R] custom align (default <p>/<i> at col 10, <r> at col 55) -alignpardef [[E] P R] paradigm alignment (if differ from general align) Any -align option implies 'compact output style' (one dict entry per line) otherwise output is noncompact XML style (one tag per line, lots of indents) Use - as file name for piping (read/write .dix files on standard input/output)
See also Crossdics
Notes for developers
Wishlist and notes for Apertium-dixtools
There should be many more options, and ALL sub-commands should take a -fmt parameter where all could be specified:
- multiwords -- one line or many lines
- multiwords -- should they be separated
(because sometimes with complex multiwords you want to have them laid out differently and apart e.g. you have a section for verbs and it has first "simple" verbs, then it has the multiword verbs)
- multiwords -- the simple verbs are one per line
- multiwords -- and the multiword verbs are over several lines