Difference between revisions of "SFST"
| (9 intermediate revisions by 5 users not shown) | |||
| Line 6: | Line 6: | ||
On Ubuntu/Debian: |
On Ubuntu/Debian: |
||
<pre> |
<pre> |
||
sudo apt-get install |
sudo apt-get install libreadline-dev |
||
</pre> |
</pre> |
||
===Download, compile, install== |
===Download, compile, install=== |
||
<pre> |
<pre> |
||
wget ftp://ftp.ims.uni-stuttgart.de/pub/corpora/SFST/SFST-1.4. |
wget ftp://ftp.ims.uni-stuttgart.de/pub/corpora/SFST/SFST-1.4.6g.tar.gz |
||
tar -xzvf SFST-1.4. |
tar -xzvf SFST-1.4.6g.tar.gz |
||
cd SFST/src |
cd SFST/src |
||
make |
make |
||
| Line 18: | Line 19: | ||
</pre> |
</pre> |
||
Note: on Arch Linux, you may have to do <code>make CFLAGS=- |
Note: on Arch Linux, you may have to do <code>make clean && make CFLAGS=-fPIC</code> instead of make. |
||
==Packaged version with fst-proc== |
|||
A packaged version, with the <code>fst-proc</code> program for processing Apertium input streams can be downloaded from Apertium SVN: |
|||
: this is now superseded by [[HFST]] + hfst-proc, right? |
|||
<pre> |
|||
$ svn co http://apertium.svn.sourceforge.net/svnroot/apertium/branches/sfst |
|||
</pre> |
|||
;Compiling: |
|||
Follow the standard steps: |
|||
<pre> |
|||
$ sh autogen.sh |
|||
$ ./configure |
|||
$ make |
|||
$ make install |
|||
</pre> |
|||
Mac users: first do |
|||
<pre> |
|||
$ sudo port install libtool |
|||
$ sudo ln -s /opt/local/bin/glibtoolize /bin/libtoolize # or wherever |
|||
</pre> |
|||
(also, the call to <code>basename(name)</code> in src/fst-proc.C needs to be changed to eg. <code>name</code> if you want to compile without libiberty sources lying around) |
|||
==Usage== |
==Usage== |
||
| Line 92: | Line 65: | ||
**For further information see: [[Omorfi]] |
**For further information see: [[Omorfi]] |
||
* SMOR — comes in the SFST distribution (German, 1,096 lemmas, GPL) |
* SMOR — comes in the SFST distribution (German, 1,096 lemmas, GPL) |
||
* trmorph (Turkish, wide coverage) |
* [[trmorph]] (Turkish, wide coverage) |
||
**For further information see: [[Turkish]] |
**For further information see: [[Turkish]] |
||
Latest revision as of 17:48, 30 March 2012
SFST (Stuttgart Finite State Toolkit) is a set of programs that can be used for writing morphological analysers. It is also one of the possible backends for HFST.
Installation[edit]
Prerequisites[edit]
On Ubuntu/Debian:
sudo apt-get install libreadline-dev
Download, compile, install[edit]
wget ftp://ftp.ims.uni-stuttgart.de/pub/corpora/SFST/SFST-1.4.6g.tar.gz tar -xzvf SFST-1.4.6g.tar.gz cd SFST/src make sudo make install sudo make libinstall
Note: on Arch Linux, you may have to do make clean && make CFLAGS=-fPIC instead of make.
Usage[edit]
To try SFST out, you can start by compiling the German transducer, SMOR, that comes with the package:
$ cd data/SMOR $ make
Wait some time, and you will have a file called smor.a, now you need to compact this so it can be read by fst-proc,
$ fst-compact smor.a smor.ac
Now you can use it,
$ cd ../../src $ echo "Ich habe ein Bier" | fst-proc ../data/SMOR/smor.ac ^Ich/<CAP>ich<+PPRO><pers><1><Sg><NoGend><Nom>$ ^habe/haben<+V><1><Sg><Pres><Konj>/haben<+V><3><Sg><Pres><Konj>/haben<+V><Imp><Sg>/haben<+V><1><Sg><Pres><Ind>$ ^ein/ein<+ART><Indef><Masc><Nom><Sg>/ein<+ART><Indef><Neut><Nom><Sg>/ein<+ART><Indef><Neut><Akk><Sg>$ ^Bier/*Bier$
It should also work with deformatters and reformatters,
$ echo "Ich habe <em>ein</em> bier" | apertium-deshtml | ./fst-proc ../data/SMOR/smor.ac ^Ich/<CAP>ich<+PPRO><pers><1><Sg><NoGend><Nom>$ ^habe/haben<+V><1><Sg><Pres><Konj>/haben<+V><3><Sg><Pres><Konj>/haben<+V><Imp><Sg>/haben<+V><1><Sg><Pres><Ind>$[ <em>]^ein/ein<+ART><Indef><Masc><Nom><Sg>/ein<+ART><Indef><Neut><Nom><Sg>/ein<+ART><Indef><Neut><Akk><Sg>$[<\/em> ]^Bier/*Bier$.[][
Note: fst-proc currently does a rather crude tokenisation based on spaces, so multiwords currently aren't possible.
Morphologies[edit]
SFST has the following morphologies available for download:
- Morph-IT! (Italian, 34,968 lemmas, LGPL)
- Omorfi–SFST implementation of word form morphology of Finnish (Finnish, 93,510 lemmas, LGPL)
- For further information see: Omorfi
- SMOR — comes in the SFST distribution (German, 1,096 lemmas, GPL)
- trmorph (Turkish, wide coverage)
- For further information see: Turkish
Performance[edit]
The analysers produced are fast. For a 1.3Mb analyser (SMOR), it processes ~1,100 words per second. Compare with lttoolbox which processes ~5,000 words per second.
