Difference between revisions of "Java port of Apertium runtime"
Jump to navigation
Jump to search
| Line 64: | Line 64: | ||
See also this thread: [http://sourceforge.net/mailarchive/message.php?msg_name=20cf28cd1003080815x56dd1969h229c3f1c7c2e81e2%40mail.gmail.com] |
See also this thread: [http://sourceforge.net/mailarchive/message.php?msg_name=20cf28cd1003080815x56dd1969h229c3f1c7c2e81e2%40mail.gmail.com] |
||
| + | |||
| + | |||
| + | ===Debugging the C tagger=== |
||
| + | |||
| + | we should step-by-step debug the C++ version first, to see what happens |
||
| + | |||
| + | echo "^this/this<det><dem><sg>/this<prn><tn><mf><sg>$ ^is/be<vbser><pri><p3><sg>$ ^a/a<det><ind><sg>$ ^test/test<n><sg>/test<vblex><inf>/test<vblex><pres>$^./.<sent>$" | ./apertium-tagger -g ../../lttoolbox-java/testdata/en-es.prob |
||
| + | |||
| + | <pre> |
||
| + | <jacobEo> ok chy whats the problem? |
||
| + | <jacobEo> chy: you must be in apertium/apertium. |
||
| + | <chy> Sorry I did run |
||
| + | <chy> ^this<prn><tn><mf><sg> ^be<vbser><pri><p3><sg>$ $^a<det><ind><sg> ^test<n><sg>$Warning (internal): kIGNORE was returned while reading a word |
||
| + | <chy> Word being read: .<sent> |
||
| + | <chy> Debug: .<sent> |
||
| + | <chy> $.<sent> |
||
| + | <jacobEo> chy: this is the Java code |
||
| + | <jacobEo> chy: debug the C code |
||
| + | <chy> ok .. we do the same for lttoolbox not lttoolbox.java |
||
| + | <chy> ? |
||
| + | <jacobEo> no, not lttoolbox dir |
||
| + | <jacobEo> apertium dir |
||
| + | <chy> ok |
||
| + | <jacobEo> chy: this dir: http://apertium.svn.sourceforge.net/viewvc/apertium/trunk/apertium/apertium/ |
||
| + | <jacobEo> first, do 'make' in parent dir |
||
| + | <jacobEo> then run |
||
| + | <jacobEo> ./apertium-tagger -g ../../lttoolbox-java/testdata/en-es.prob |
||
| + | <jacobEo> to run the C code |
||
| + | <jacobEo> chy: ? |
||
| + | <jacobEo> chy: what's happening? |
||
| + | <chy> if i run above command ./apertium-tagger -g ../../lttoolbox-java/testdata/en-es.prob its stops |
||
| + | <jacobEo> error msg? |
||
| + | <chy> no nothing |
||
| + | <jacobEo> crash? |
||
| + | <jacobEo> is ../../lttoolbox-java/testdata/en-es.prob in place? |
||
| + | <jacobEo> ls -l ../../lttoolbox-java/testdata/en-es.prob |
||
| + | <chy> no waiting ./apertium-tagger -g /home/mhc/apertium/javaApertium/lttoolbox-java/testdata/en-es.prob |
||
| + | <jacobEo> ahhhhh!!!!!!!!!!! |
||
| + | <chy> -rw-r--r-- 1 mhc mhc 47314 2010-03-31 23:10 ../../lttoolbox-java/testdata/en-es.prob |
||
| + | <jacobEo> chy: please tell me what happens. |
||
| + | <jacobEo> we just wasted 20 mins waiting for a program that expects input before it writes anytinbg to output! |
||
| + | <jacobEo> ok |
||
| + | <jacobEo> echo "^this/this<det><dem><sg>/this<prn><tn><mf><sg>$ ^is/be<vbser><pri><p3><sg>$ ^a/a<det><ind><sg>$ ^test/test<n><sg>/test<vblex><inf>/test<vblex><pres>$^./.<sent>$" | ./apertium-tagger -g /home/mhc/apertium/javaApertium/lttoolbox-java/testdata/en-es.prob |
||
| + | <chy> ^this<prn><tn><mf><sg>$ ^be<vbser><pri><p3><sg>$ ^a<det><ind><sg>$ ^test<n><sg>$^.<sent>$ |
||
| + | <chy> sorry,,,, |
||
| + | <jacobEo> great, it works. |
||
| + | <jacobEo> chy: now, debug it :-) |
||
| + | <jacobEo> ah |
||
| + | <jacobEo> man tagger |
||
| + | <chy> Ok need a little time .. |
||
| + | <jacobEo> $ man apertium-tagger |
||
| + | <jacobEo> says |
||
| + | <jacobEo> apertium-tagger --tagger|-g [--first|-f] PROB [--debug|-d] [INPUT [OUTPUT]] |
||
| + | <jacobEo> i.e. input can be in a file |
||
| + | <jacobEo> chy: do: |
||
| + | <jacobEo> echo "^this/this<det><dem><sg>/this<prn><tn><mf><sg>$ ^is/be<vbser><pri><p3><sg>$ ^a/a<det><ind><sg>$ ^test/test<n><sg>/test<vblex><inf>/test<vblex><pres>$^./.<sent>$" > input.txt |
||
| + | <jacobEo> and then |
||
| + | <jacobEo> ./apertium-tagger -g /home/mhc/apertium/javaApertium/lttoolbox-java/testdata/en-es.prob input.txt |
||
| + | |||
| + | </pre> |
||
Revision as of 09:59, 9 April 2010
A "Java port" of Apertium would enable use on
- Windows,
- J2ME/Android phones,
- web pages (applets),
- desktop application,
- Java server applications.
The last 2 is relevant as, for example an OpenOffice.org plugin should be platform independent to be maintainable.
AFAIK we havent seen anyone embedding Apertium in a desktop application. Currently Apertium is usable in a local subdir but installation isnt trivial to an end user, but note that 'embedding' something isnt the same as 'using a locally installed version'.
Having a packaged easy-to-use version of Apertium ready for embedding MT in a larger program would be very cool. Ideally should a self-contained Apertium JAR file, only dependent on JRE and an additional JAR file per language pair.
An "embedding" approach is to use a client stub to an Apertium web service, but there can be reasons why people prefers to have things installed locally (we don't need to repeat them here).
Missing for a complete port of apertium in java is tagger, piping/modes, interchunk/postchunk and format handling.
Af I see it:
- Interchunk/postchunk I could do in very short time, so I could mentor this
- Tagger: The C++ code is there and some has already been ported to Java (lttoolbox-java parts), probably someone could port it relatively easy, if someone understanding tagger would co-mentor that part. I'd say its not necessary to port tagger training, just the core (bigram) tagging during translation.
- Piping/modes isnt too hard.
- Format handling: I'd say that it would be OK just to be able to handle normal text.
--Jacob Nordfalk 08:33, 14 March 2010 (UTC)
Contents
Source code
Source code is here: http://apertium.svn.sourceforge.net/viewvc/apertium/trunk/lttoolbox-java/
Interested people
By #apertium IRC chat name:
- chy - Danish student Moffassal Hossain - User:Chy/Gsoc 2010 Application/Java_port_of_Apertium
- Kanmuri - User:Kanmuri/GSoC_2010_Application/Java_Runtime_Port
- Zerocool1989
- keshan
- ...
Random snippets of information, that someone might find usefull but noone cared to organize yet
<Unhammer> I _think_ those days are OK with me <jacobEo> Zerocool1989: please co-ordinate with jimregan as he's also working on the code. <jacobEo> Zerocool1989: I think you first have to try to use a lang par with interchunk and postchunk * sioraiocht (~tomh@unaffiliated/sioraiocht) has joined #apertium <Zerocool1989> ok <Zerocool1989> where can i get the interchunk code <jacobEo> 1 sec <jacobEo> esperanto/apertium/apertium/apertium/interchunk.cc <jacobEo> sorry <jacobEo> apertium dir in SVN + /apertium/interchunk.cc <jacobEo> but Zerocool1989 (and Kanmuri? ) you should compare with esperanto/apertium/apertium/apertium/transfer.cc <jacobEo> and the Java transfer <jacobEo> is working very differently <jacobEo> actually, I think the best thing would be not to care about interchunk and postchunk for now, I can probably do them very fast <jacobEo> better would be to help jimregan on the tagger work. <jacobEo> and to sort out the modes/piping <jacobEo> interchunk and postchunk is also only for 3+ - stage transfer. There are language pairs that only have 1 stage. <jacobEo> so another thing that could be done is to try out an 1-stage lang pair (e.g. nn-nb) and see if you can get the parts running in Java
See also this thread: [1]
Debugging the C tagger
we should step-by-step debug the C++ version first, to see what happens
echo "^this/this<det><dem><sg>/this<prn><tn><mf><sg>$ ^is/be<vbser><pri><p3><sg>$ ^a/a<det><ind><sg>$ ^test/test<n><sg>/test<vblex><inf>/test<vblex><pres>$^./.<sent>$" | ./apertium-tagger -g ../../lttoolbox-java/testdata/en-es.prob
<jacobEo> ok chy whats the problem? <jacobEo> chy: you must be in apertium/apertium. <chy> Sorry I did run <chy> ^this<prn><tn><mf><sg> ^be<vbser><pri><p3><sg>$ $^a<det><ind><sg> ^test<n><sg>$Warning (internal): kIGNORE was returned while reading a word <chy> Word being read: .<sent> <chy> Debug: .<sent> <chy> $.<sent> <jacobEo> chy: this is the Java code <jacobEo> chy: debug the C code <chy> ok .. we do the same for lttoolbox not lttoolbox.java <chy> ? <jacobEo> no, not lttoolbox dir <jacobEo> apertium dir <chy> ok <jacobEo> chy: this dir: http://apertium.svn.sourceforge.net/viewvc/apertium/trunk/apertium/apertium/ <jacobEo> first, do 'make' in parent dir <jacobEo> then run <jacobEo> ./apertium-tagger -g ../../lttoolbox-java/testdata/en-es.prob <jacobEo> to run the C code <jacobEo> chy: ? <jacobEo> chy: what's happening? <chy> if i run above command ./apertium-tagger -g ../../lttoolbox-java/testdata/en-es.prob its stops <jacobEo> error msg? <chy> no nothing <jacobEo> crash? <jacobEo> is ../../lttoolbox-java/testdata/en-es.prob in place? <jacobEo> ls -l ../../lttoolbox-java/testdata/en-es.prob <chy> no waiting ./apertium-tagger -g /home/mhc/apertium/javaApertium/lttoolbox-java/testdata/en-es.prob <jacobEo> ahhhhh!!!!!!!!!!! <chy> -rw-r--r-- 1 mhc mhc 47314 2010-03-31 23:10 ../../lttoolbox-java/testdata/en-es.prob <jacobEo> chy: please tell me what happens. <jacobEo> we just wasted 20 mins waiting for a program that expects input before it writes anytinbg to output! <jacobEo> ok <jacobEo> echo "^this/this<det><dem><sg>/this<prn><tn><mf><sg>$ ^is/be<vbser><pri><p3><sg>$ ^a/a<det><ind><sg>$ ^test/test<n><sg>/test<vblex><inf>/test<vblex><pres>$^./.<sent>$" | ./apertium-tagger -g /home/mhc/apertium/javaApertium/lttoolbox-java/testdata/en-es.prob <chy> ^this<prn><tn><mf><sg>$ ^be<vbser><pri><p3><sg>$ ^a<det><ind><sg>$ ^test<n><sg>$^.<sent>$ <chy> sorry,,,, <jacobEo> great, it works. <jacobEo> chy: now, debug it :-) <jacobEo> ah <jacobEo> man tagger <chy> Ok need a little time .. <jacobEo> $ man apertium-tagger <jacobEo> says <jacobEo> apertium-tagger --tagger|-g [--first|-f] PROB [--debug|-d] [INPUT [OUTPUT]] <jacobEo> i.e. input can be in a file <jacobEo> chy: do: <jacobEo> echo "^this/this<det><dem><sg>/this<prn><tn><mf><sg>$ ^is/be<vbser><pri><p3><sg>$ ^a/a<det><ind><sg>$ ^test/test<n><sg>/test<vblex><inf>/test<vblex><pres>$^./.<sent>$" > input.txt <jacobEo> and then <jacobEo> ./apertium-tagger -g /home/mhc/apertium/javaApertium/lttoolbox-java/testdata/en-es.prob input.txt
