Difference between revisions of "IRSTLM"
Jump to navigation
Jump to search
Naan Dhaan (talk | contribs) m (exporting IRSTLM) |
|||
| (10 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
''IRSTLM'' is a free and open source exact statistical language model using memory-mapping. The language models are compatible with those created with the closed-source SRILM Tooolkit. |
''IRSTLM'' is a free and open source exact statistical language model using memory-mapping. The language models are compatible with those created with the closed-source SRILM Tooolkit. |
||
See the homepage at |
See the homepage at https://hlt-mt.fbk.eu/technologies/irstlm |
||
| ⚫ | |||
| ⚫ | |||
see https://github.com/irstlm-team/irstlm |
|||
or |
|||
<pre> |
<pre> |
||
svn checkout svn://svn.code.sf.net/p/irstlm/code/trunk irstlm |
svn checkout svn://svn.code.sf.net/p/irstlm/code/trunk irstlm |
||
| Line 11: | Line 13: | ||
make -j4 |
make -j4 |
||
make install |
make install |
||
</pre> |
|||
if you get <code>stdlib.h</code> error, see https://github.com/irstlm-team/irstlm/issues/22 |
|||
== Make a language model == |
|||
<pre> |
|||
# if you specified /path/prefix previously |
|||
export IRSTLM=/path/prefix |
|||
# else |
|||
export IRSTLM=/usr/local |
|||
$IRSTLM/bin/build-lm.sh -i incorpus.txt -o out.lm.gz -t tmp/ |
|||
</pre> |
</pre> |
||
==See also== |
==See also== |
||
* [[Moses]] ( |
* [[Moses]] (includes alternative LM system KenLM) |
||
* [[Using GIZA++]] |
* [[Using GIZA++]] |
||
* [[RandLM]] - a randomised LM, based on [http://en.wikipedia.org/wiki/Bloom_filter Bloom Filters] |
* [[RandLM]] - a randomised LM, based on [http://en.wikipedia.org/wiki/Bloom_filter Bloom Filters] |
||
Latest revision as of 15:15, 5 August 2021
IRSTLM is a free and open source exact statistical language model using memory-mapping. The language models are compatible with those created with the closed-source SRILM Tooolkit.
See the homepage at https://hlt-mt.fbk.eu/technologies/irstlm
Installation[edit]
see https://github.com/irstlm-team/irstlm or
svn checkout svn://svn.code.sf.net/p/irstlm/code/trunk irstlm cd irstlm cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=/path/prefix make -j4 make install
if you get stdlib.h error, see https://github.com/irstlm-team/irstlm/issues/22
Make a language model[edit]
# if you specified /path/prefix previously export IRSTLM=/path/prefix # else export IRSTLM=/usr/local $IRSTLM/bin/build-lm.sh -i incorpus.txt -o out.lm.gz -t tmp/
See also[edit]
- Moses (includes alternative LM system KenLM)
- Using GIZA++
- RandLM - a randomised LM, based on Bloom Filters
