Talk:Scandinavian MT project
Jump to navigation
Jump to search
Coverage on Wikipedia dumps ("w/o cmp" is with decompounding turned off, ie. without the -e switch to lt-proc).
E.g.
bzcat ~/corpora/nnclean2.txt.bz2 \
|tr ' ' '\n' \
|grep -m5113060 . \
|apertium-deshtml \
|lt-proc nno-dan.automorf.bin \
|apertium-cleanstream -n \
|awk 'BEGIN{OFS=FS="\t"} /^\^/{lu++} /\/\*/{u++} END{print "unk","known","tot","cov %";print u,lu-u,lu,100*(lu-u)/lu}'
| Direction | w/o cmp | regular |
|---|---|---|
| nob-nno | 90.9% | 92.6% |
| nob-dan | 89.8% | 91.5% |
| nno-nob | 89.2% | 90.6% |
| nno-dan | 87.4% | 88.8% |
| dan-nob | 85.1% | 86.4% |
| swe-dan | 80.4% | 83.7% |
| dan-nno | 82.5% | 83.5% |
| dan-swe | 80.6% | 82.9% |
| nob-swe | 74.9% | 76.2% |
| nno-swe | 73.5% | 74.6% |
| swe-nob | 69.2% | 72.1% |
| swe-nno | 69.1% | 71.9% |
