Moses - Revision history

Unhammer at 08:57, 29 April 2015

2015-04-29T08:57:42Z

Unhammer: /* Troubleshooting */

2015-04-29T08:53:03Z

Troubleshooting

Unhammer: /* Compiling */

2015-04-29T08:51:59Z

Compiling

Unhammer at 14:08, 28 April 2015

2015-04-28T14:08:23Z

Unhammer: /path/prefix

2015-04-28T11:01:45Z

/path/prefix

Unhammer: /* Requisites */

2015-04-28T10:57:15Z

Requisites

Unhammer: new version uses cmake

2015-04-28T09:13:42Z

new version uses cmake

Unhammer: the new version uses "which" (not that that's best practice either but meh)

2015-04-28T09:12:08Z

the new version uses "which" (not that that's best practice either but meh)

Unhammer at 09:11, 28 April 2015

2015-04-28T09:11:24Z

Bech: Link to French page

2014-10-06T09:19:44Z

Link to French page

@@ Line 3: / Line 3: @@
 {{TOCD}}
-==Requisites==
+==Prerequisites==
+* [[GIZA++]], see the page for how to compile that. Moses also supports [[mgiza]] as an alternative to Giza.
+* [[IRSTLM]], see the page for how to compile that, and how to make a language model.
-* GIZA++ and mkcls (<code>git clone https://github.com/moses-smt/giza-pp</code>)
-* Moses (<code>git clone git@github.com:moses-smt/mosesdecoder.git</code>)
-* IRST LM (<code>svn checkout svn://svn.code.sf.net/p/irstlm/code/trunk irstlm</code>)
 ==Compiling==
-See [[Using GIZA++]] for how to compile that. Moses also supports [[mgiza]] as an alternative to Giza.
-See [[IRSTLM]] for how to compile that.
 Do
 <pre>
@@ Line 21: / Line 15: @@
 </pre>
 The bjam part takes a long while.
-==Building language model==
-<pre>
-export IRSTLM=/path/prefix
-build-lm.sh -i cy.crp.txt -o cy.lm.gz -t /tmp
-</pre>
 ==Troubleshooting==

@@ Line 30: / Line 30: @@
 ==Troubleshooting==
+If your logs anywhere say anything about UnicodeEncodeError, you might have to do
 <pre>
-do
 export PYTHONIOENCODING=utf-8
-before running train-model.perl (or fix merge_alignments.py yourself)
 </pre>
+before running train-model.perl (or fix merge_alignments.py yourself)
 ==See also==

@@ Line 10: / Line 10: @@
 ==Compiling==
+See [[Using GIZA++]] for how to compile that. Moses also supports [[mgiza]] as an alternative to Giza.
-{{see-also|Using GIZA++}}
+See [[IRSTLM]] for how to compile that.
-;GIZA++
+Do
 <pre>
+git clone https://github.com/moses-smt/mosesdecoder
-tar -xzvf giza-pp-v1.0.2.tar.gz
+cd mosesdecoder/
-cd giza-pp
+./bjam
-make
-cp mkcls-v2/mkcls /path/prefix/bin
-cp GIZA++-v2/GIZA++ /path/prefix/bin
-cp GIZA++-v2/plain2snt.out /path/prefix/bin
-cp GIZA++-v2/snt2cooc.out /path/prefix/bin
-cp GIZA++-v2/snt2plain.out /path/prefix/bin
-cp GIZA++-v2/trainGIZA++.sh /path/prefix/bin
-cd ..
-</pre>
-;Moses
-<pre>
-cd trunk
-./regenerate-makefiles.sh
-./configure --prefix=/path/prefix
-make
-make install
-cd scripts/training/symal
-make
-cp symal giza2bal.pl /path/prefix/bin
-cd ../../../
-cd scripts/training/phrase-extract
-make
-cp extract score /path/prefix/bin
-cd ../../../
-</pre>
-Now edit the file <code>scripts/training/train-factored-phrase-model.perl</code> and change the following lines:
-<pre>
-my $SCRIPTS_ROOTDIR = "/home/fran/source/moses/trunk/scripts/";
-...
-# the following line is set installation time by 'make release'.  BEWARE!
-my $BINDIR="/path/prefix/bin";
-</pre>
-<pre>
-cp scripts/training/train-factored-phrase-model.perl /path/prefix/bin/
-cp scripts/training/symal/giza2bal.pl /path/prefix/bin/
-cd ..
-</pre>
-;IRSTLM
-<pre>
-cd irstlm
-cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=/path/prefix
-make -j4
-make install
 </pre>
+The bjam part takes a long while.
 ==Building language model==

@@ Line 75: / Line 75: @@
 export IRSTLM=/path/prefix
 build-lm.sh -i cy.crp.txt -o cy.lm.gz -t /tmp
+</pre>
+==Troubleshooting==
+<pre>
+do
+export PYTHONIOENCODING=utf-8
+before running train-model.perl (or fix merge_alignments.py yourself)
 </pre>

@@ Line 65: / Line 65: @@
 <pre>
 cd irstlm
-cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=/usr/local # or whatever
+cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=/path/prefix
 make -j4
+make install
-sudo make install # or just "make install" if prefix is under your user
 </pre>

@@ Line 65: / Line 65: @@
 <pre>
 cd irstlm
+cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=/usr/local # or whatever
-./install
+make -j4
-</pre>
+sudo make install # or just "make install" if prefix is under your user
-<pre>
-cp bin/* /path/prefix/bin/
-cp bin/x86_64-pc-linux-gnu/* /path/prefix/bin/
-mkdir -p /path/prefix/include
-cp include/* /path/prefix/include
-cp lib/x86_64-pc-linux-gnu/libirstlm.a /path/prefix/lib/
-cd ..
 </pre>

@@ Line 67: / Line 67: @@
 ./install
 </pre>
-Now edit the files in <code>scripts/build-sublm.pl</code> and <code>scripts/merge-sublm.pl</code> and check the location of gzip,
-<pre>
-my $gzip="/usr/bin/gzip";
-my $gunzip="/usr/bin/gunzip";
-</pre>
-On Debian systems, <code>gzip</code> and <code>gunzip</code> are found in <code>/bin</code>, these two scripts will fail silently if gzip is not found.
 <pre>

@@ Line 6: / Line 6: @@
 * GIZA++ and mkcls http://giza-pp.googlecode.com/files/giza-pp-v1.0.2.tar.gz
-* Moses (<code>svn co https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk</code>)
+* Moses (<code>git clone git@github.com:moses-smt/mosesdecoder.git</code>)
-* IRST LM (<code> svn co https://irstlm.svn.sourceforge.net/svnroot/irstlm</code>)
+* IRST LM (<code>svn checkout svn://svn.code.sf.net/p/irstlm/code/trunk irstlm</code>)
 ==Compiling==

← Older revision		Revision as of 09:19, 6 October 2014
Line 1:		Line 1:
			[[L'outil Moses\|En français]]

	{{TOCD}}		{{TOCD}}