Difference between revisions of "Moses"
Jump to navigation
Jump to search
| (One intermediate revision by the same user not shown) | |||
| Line 3: | Line 3: | ||
{{TOCD}} |
{{TOCD}} |
||
| − | == |
+ | ==Prerequisites== |
| ⚫ | |||
| − | |||
| ⚫ | |||
| − | * GIZA++ and mkcls (<code>git clone https://github.com/moses-smt/giza-pp</code>) |
||
| − | * Moses (<code>git clone git@github.com:moses-smt/mosesdecoder.git</code>) |
||
| − | * IRST LM (<code>svn checkout svn://svn.code.sf.net/p/irstlm/code/trunk irstlm</code>) |
||
==Compiling== |
==Compiling== |
||
| ⚫ | |||
| − | |||
| ⚫ | |||
| − | |||
Do |
Do |
||
<pre> |
<pre> |
||
| Line 21: | Line 15: | ||
</pre> |
</pre> |
||
The bjam part takes a long while. |
The bjam part takes a long while. |
||
| − | |||
| − | ==Building language model== |
||
| − | |||
| ⚫ | |||
| − | export IRSTLM=/path/prefix |
||
| − | build-lm.sh -i cy.crp.txt -o cy.lm.gz -t /tmp |
||
| − | </pre> |
||
==Troubleshooting== |
==Troubleshooting== |
||
| + | If your logs anywhere say anything about UnicodeEncodeError, you might have to do |
||
<pre> |
<pre> |
||
| − | do |
||
export PYTHONIOENCODING=utf-8 |
export PYTHONIOENCODING=utf-8 |
||
| ⚫ | |||
before running train-model.perl (or fix merge_alignments.py yourself) |
before running train-model.perl (or fix merge_alignments.py yourself) |
||
| − | </pre> |
||
==See also== |
==See also== |
||
Latest revision as of 08:57, 29 April 2015
Prerequisites[edit]
- GIZA++, see the page for how to compile that. Moses also supports mgiza as an alternative to Giza.
- IRSTLM, see the page for how to compile that, and how to make a language model.
Compiling[edit]
Do
git clone https://github.com/moses-smt/mosesdecoder cd mosesdecoder/ ./bjam
The bjam part takes a long while.
Troubleshooting[edit]
If your logs anywhere say anything about UnicodeEncodeError, you might have to do
export PYTHONIOENCODING=utf-8
before running train-model.perl (or fix merge_alignments.py yourself)
