Difference between revisions of "IRSTLM"
Jump to navigation
Jump to search
(Created page with '''IRSTLM'' is a free and open source exact statistical language model using memory-mapping. The language models are compatible with those created with the closed-source SRILM Too…') |
Naan Dhaan (talk | contribs) m (exporting IRSTLM) |
||
(11 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
''IRSTLM'' is a free and open source exact statistical language model using memory-mapping. The language models are compatible with those created with the closed-source SRILM Tooolkit. |
''IRSTLM'' is a free and open source exact statistical language model using memory-mapping. The language models are compatible with those created with the closed-source SRILM Tooolkit. |
||
− | See the homepage at |
+ | See the homepage at https://hlt-mt.fbk.eu/technologies/irstlm |
+ | |||
+ | |||
+ | ==Installation== |
||
+ | see https://github.com/irstlm-team/irstlm |
||
+ | or |
||
+ | <pre> |
||
+ | svn checkout svn://svn.code.sf.net/p/irstlm/code/trunk irstlm |
||
+ | cd irstlm |
||
+ | cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=/path/prefix |
||
+ | make -j4 |
||
+ | make install |
||
+ | </pre> |
||
+ | |||
+ | if you get <code>stdlib.h</code> error, see https://github.com/irstlm-team/irstlm/issues/22 |
||
+ | |||
+ | == Make a language model == |
||
+ | <pre> |
||
+ | # if you specified /path/prefix previously |
||
+ | export IRSTLM=/path/prefix |
||
+ | # else |
||
+ | export IRSTLM=/usr/local |
||
+ | |||
+ | $IRSTLM/bin/build-lm.sh -i incorpus.txt -o out.lm.gz -t tmp/ |
||
+ | </pre> |
||
==See also== |
==See also== |
||
− | * [[Moses]] ( |
+ | * [[Moses]] (includes alternative LM system KenLM) |
* [[Using GIZA++]] |
* [[Using GIZA++]] |
||
* [[RandLM]] - a randomised LM, based on [http://en.wikipedia.org/wiki/Bloom_filter Bloom Filters] |
* [[RandLM]] - a randomised LM, based on [http://en.wikipedia.org/wiki/Bloom_filter Bloom Filters] |
Latest revision as of 15:15, 5 August 2021
IRSTLM is a free and open source exact statistical language model using memory-mapping. The language models are compatible with those created with the closed-source SRILM Tooolkit.
See the homepage at https://hlt-mt.fbk.eu/technologies/irstlm
Installation[edit]
see https://github.com/irstlm-team/irstlm or
svn checkout svn://svn.code.sf.net/p/irstlm/code/trunk irstlm cd irstlm cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=/path/prefix make -j4 make install
if you get stdlib.h
error, see https://github.com/irstlm-team/irstlm/issues/22
Make a language model[edit]
# if you specified /path/prefix previously export IRSTLM=/path/prefix # else export IRSTLM=/usr/local $IRSTLM/bin/build-lm.sh -i incorpus.txt -o out.lm.gz -t tmp/
See also[edit]
- Moses (includes alternative LM system KenLM)
- Using GIZA++
- RandLM - a randomised LM, based on Bloom Filters