Compilation Speed

Compiling dictionaries takes time. There are some tricks for speeding it up.

Each language pair is different so you may want to time the effect yourself, but the following tricks have all had real speedups on apertium-nno-nob at least. All timings are real/wall clock:

1) Use make -j instead of just make, in order to use several processors for make goals that can be parallellised. (Note that if you're low on memory and your dictionaries are very large, you may want to cap it to make -j2 or similar.)

• make clean && make in nno-nob takes 2m33s
• make clean && make -j in nno-nob takes 1m24s

2) Put <code>export LT_JOBS=yes</code> in your ~/.bashrc (or ~/.bash_profile) – this will let lttoolbox split dictionaries into sections of at most 50k entries and minimise them in parallel (and large sections get exponentially slower to minimise). You can also tweak the threshold with <code>export LT_MAX_SECTION_ENTRIES=50000</code>.

• make clean && export LT_JOBS=no && make -j nob.automorf.bin in nob takes 41s (typical situation where only monodix has been changed)
• make clean && export LT_JOBS=yes && make -j nob.automorf.bin in nob takes 17s

3) Try another malloc, e.g. tcmalloc or jemalloc. On Debian/Ubuntu, you would sudo apt install libtcmalloc-minimal4 and then put <code>function m () { export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libtcmalloc_minimal.so.4; make "$@"; }</code> into your ~/.bashrc (or ~/.bash_profile). Now you can use m instead of make to compile with tcmalloc.

• make clean && export LT_JOBS=yes && make -j in nno-nob takes 1m24s with regular malloc (glibc)
• make clean && export LT_JOBS=yes && m -j in nno-nob takes 1m03s with tcmalloc

4) If you work mostly on one direction, you may want to make a debug goal in your Makefile.am for just that direction (example, used as m -j e)