Talk:Scandinavian MT project
Jump to navigation
Jump to search
Coverage on Wikipedia dumps ("w/o cmp" is with decompounding turned off, ie. without the -e switch to lt-proc).
E.g.
bzcat ~/corpora/nnclean2.txt.bz2 \
|tr ' ' '\n' \
|grep -m5113060 . \
|apertium-deshtml \
|lt-proc nno-dan.automorf.bin \
|apertium-cleanstream -n \
|awk 'BEGIN{OFS=FS="\t"} /^\^/{lu++} /\/\*/{u++} END{print "unk","known","tot","cov %";print u,lu-u,lu,100*(lu-u)/lu}'
| Direction | w/o cmp | regular |
|---|---|---|
| nob-nno | ||
| nno-nob | ||
| dan-nob | ||
| dan-nno | ||
| nno-dan | 87.6% | 89.0% |
| nob-dan | 89.7% | 91.4% |
| dan-swe | 80.6% | 83.0% |
| swe-dan | 83.8% | |
| swe-nno | ||
| swe-nob | ||
| nno-swe | ||
| nob-swe |
