Compounds

Some languages in Indo-European, particularly Germanic languages and Proto-Indo-Iranian languages like Sanskrit, make long compound words with low frequency that are unlikely to be found in dictionaries. Typically for any "normal" noun, there can be around 10—100 compound nouns which inflect in exactly the same way (at least for Afrikaans).

• Afrikaans: infrastruktuurontwikkelingsplan, infrastruktuur+ontwikkelings+plan ("infrastructure development plan"), (cf. personeelverminderingsprosedure, "personnel protection procedure")
• Dutch : "hulpagina" (help page), "woordbetekenis" (meaning of a word), "inwonertal" (number of inhabitants)
• German: Kontaktlinsenverträglichkeitstest, Kontakt+linsen+verträglichkeits+test ("contact-lens compatibility test")
• Danish: Kontaktlinsevæske, Kontaktlinse+væske ("contact-lens liquid")
• Sanskrit: विद्या + आतुर = विद्यातुर , vidyā + ātur = vidyātur ("eager to gain knowledge")
• Esperanto: Vikitraduko, Vikio+traduko (Wiki translation). Tradukoservo (==tradukservo), traduko+servo (translation service). Poŝtelefono, poŝa+telefono (pocket phone). Bonkvalita, bona+kvalita (good quality). Diaro, dio+aro (god collection),

There should be some method of attempting to resolve unknown compound words into their constituent parts.

See also the bug report

Epenthetics

Many compounds have (epenthetics), letters which connect the two words but perhaps only occur in compounds:

• kransekake => krans+e+kake ≈ 'ring cake'
• ungdomsfyll => ungdom+s+fyll = 'youth drunkenness'

We can solve this in the dix format in the following way:

<pardef n="krans__n">
  <e>       <p><l>ane</l>       <r><s n="compound-R"/><s n="n"/><s n="m"/><s n="pl"/><s n="def"/></r></p></e>
  <e>       <p><l>ar</l>        <r><s n="compound-R"/><s n="n"/><s n="m"/><s n="pl"/><s n="ind"/></r></p></e>
  <e>       <p><l>en</l>        <r><s n="compound-R"/><s n="n"/><s n="m"/><s n="sg"/><s n="def"/></r></p></e>
  <e>       <p><l></l>          <r><s n="compound-R"/><s n="n"/><s n="m"/><s n="sg"/><s n="ind"/></r></p></e>
  <e>       <p><l></l>          <r><s n="compound-only-L"/><s n="n"/><s n="m"/><s n="sg"/><s n="ind"/></r></p></e>
  <e>       <p><l>e</l>         <r><s n="compound-only-L"/><s n="n"/><s n="m"/><s n="sg"/><s n="ind"/></r></p></e>
</pardef>

if we interpret compound-R as meaning "this might be the right part of a compound, but might also stand alone" and compound-only-L as meaning "this might be the left part of a compound, but can't stand alone". This also lets us specify that (for this paradigm at least) only the right part may inflect.

Objection 1: why not have just compound-L and compound-R, meaning "might be the left/right part or might stand alone"? Because you're never going to see "kakee" on its own, right?

Answer: True, we won't see "kakee" on its own, but "kranse" is a verb with another meaning, and tagging the noun as "compound-L" and allowing it to be analysed on its own would lead to unnecessary ambiguity.

Objection 2: Hey, you've got compound-only-L but not compound-L and not compund-only-R. You can't analyse Klingon!

Answer: This isn't even implemented yet. Once two symbols are implemented and working, we can think about adding the rest of the possibilites.

Objection 3: You really should have some sort of compound-both symbol for your <l/> entry instead of one with compound-only-L and one with compound-R, it'll mess up your tagger.

Answer: These symbols are removed in the output.

The epenthetic has no meaning in itself, but if you can have the same word with or without an epenthetic, it might signal a difference. Compare:

• sjefsekretær = 'chief of secretaries' (Norwegian)
• sjefssekretær = 'the boss's secretary'
• gjestfri = 'hospitable'
• gjestefri = 'guest free' (as in "guest free zone")

(gjestfri would typically be listed in the dictionary, gjestefri being compositional.)

Outstanding questions

• Where would compound processing go in the pipeline? Presumably after initial analysis? e.g. in between lt-proc and apertium-tagger.

Proposed algorithms

Analysis

Baseline

input: ^*infrastruktuurontwikkelingsplan$

1. Read word from left to right.
2. Take the shortest match first from the dictionaries, e.g. infrastruktuurontwikkelingsplan,
   1. Read i-n-f-r-a-s-t-r-u-k-u-u-r (add, because no words have +o)
   2. Read o-n-t-w-i-k-k-e-l-i-n-g-s (add, because no words have +p)
   3. Read p-l-a-n
3. Output in order.

output: ^infrastruktuur<n><sg>$ ^ontwikkeling<n><pl>$ ^plan<n><sg>$

This won't work for Esperanto and other languages where the root without word ending is never seen (correct in Esperanto is infrastruktur+evolu+plano, not infrastrukturo+evoluo+plano)

This means these entries have to be in the dictionaries, we could treat it the same way as epenthetics. Say that we have a marker on (or inside) <e> that says that this is <e> may be the left side of a compound. Then for Esperanto, you'd have to add, to your pardef or whatever:

   <e (left-compoundable)>              <p><l>evolu</l> <r>evoluo</r></p></e>
   <e (non-compoundable, regular entry)><p><l>evoluo</l><r>evoluo</r></p></e>

Left-to-right longest-match

input: ^*infrastruktuurontwikkelingsplan$

1. Read word from left to right.
2. Take the longest match first from the dictionaries, e.g. infrastruktuurontwikkelingsplan,
3. While not found:
   1. Read infrastruktuurontwikkelingsplan
   2. Read infrastruktuurontwikkelingspla
   3. Read infrastruktuurontwikkelingspl
   4. ...
4. When a word is found, e.g. "infrastruktuur", remove it from the string, put it in the output queue, and start the process again with ontwikkelingsplan.
5. Output each item from the output queue in order.

output: ^infrastruktuur<n><sg>$ ^ontwikkeling<n><pl>$ ^plan<n><sg>$

Generation

If we have translated a string of input words from English to Afrikaans and now want to compound the ones that can be compounded, we have a problem that instead of:

infrastruktuurontwikkelingsplan

we have:

infrastruktuur ontwikkeling plan

One way of resolving this would be to compile a wordlist (without morphological info) into a tree, and then scan the tree as we scan the output, concatenating words where they appear in the wordlist. Wordlists are substantially more numerous than morphological resources so it would be quite cheap. In order to avoid false-positives, we could just set a length requirement of something like 9 characters or more.

Further reading

• http://en.wikipedia.org/wiki/Compound_(linguistics)
• Koehn, P. and Knight, K. (2003) "Empirical Methods for Compound Splitting". 11th Conference of the European Chapter of the Association for Computational Linguistics, (EACL2003).
• Brown, R. (2002) "Corpus-Driven Splitting of Compound Words". TMI 2002
• Larson, M., Willett, D., Köhler, J. and Rigoll, G. (2000) "Compound splitting and lexical unit recombination for improved performance of a speech recognition system for German parliamentary speeches". Conference on Spoken Language Processing, 2000.
• Moa, H. (2005) "Compounds and other oddities in machine translation". Proceedings of the 15th NODALIDA conference, Joensuu 2005.
• Johannesen, J. B. and Hauglin, H. (1996) "An automatic analysis of Norwegian compounds". In Haukioja, T. (ed.): Papers from the 16th Scandinavian Conference of Linguistics, Turku/Åbo, Finland 1996: 209-220.