Курсы машинного перевода для языков России/Session 1

From Apertium
Jump to navigation Jump to search

This session has two objectives, the first is to give an overview of the theory of morphology, how words are inflected and how new words are formed. And the second is to demonstrate how the analysis and generation of morphology is dealt with in Apertium.

Theory

The theory section is split into three subsections: The first deals with morphotactics, that is how morphemes (parts of words) occur and are joined together. The second gives some details of morphophonology, or how changes in morphemes happen as a result of them being joined together. The final section covers a theoretical description of how this is treated with computers.

Morphotactics

The morphotactics of a language is the way that morphemes in that language are joined together to form words. Morphemes are the smallest units of meaning. Morphemes can be free, or bound. They are free if they can occur on their own, and bound if they must be connected to another word. A single morpheme may have several allomorphs which mean the same thing but are written or spoken differently. For example the dative case (used to indicate movement in the direction of) in Chuvash has several allomorphs, which change depending on the vowel quality of the stem to which it attaches.

aчама ача·м·а "to my child"
ачамсене ача·м·сен·е "to my children"
ӗҫӗме ӗҫ·ӗм·е "to my work"
каҫмана каҫма·на "переходу"

Morphemes can be further split into two subtypes, inflectional and derivational. In the examples, · signifies a derivational boundary, and » signifies an inflectional boundary.

ӗҫ ӗҫ "работ·а"
ӗҫсем ӗҫ·сем "работ·ы"
ĕçчен ĕç»чен "работ»ник"
ĕçченсем ĕç»чен·сем "работ»ник·и"
ӗҫле ӗҫ»ле "работа»ть"
ӗҫле ӗҫ»ле»тер "to make (someone) work"

Inflection

Inflectional morphemes carry grammatical information, such as number, case, tense, etc. but do not change the word category (part of speech), nor do they change the basic semantic meaning. For example in Chuvash, ӗҫ and ӗҫсем have the same basic semantic meaning, but if you add the derivational affix -лЕ, ӗҫле, then the meaning changes to "do ӗҫ".

Examples of inflectional morphemes might be the -lar, -сем and -и plurals (kitaplar (tr), ачасем (cv), книги (ru)), and case endings -ран (ablative), -kor (temporal), and -eh (locative): уй+ран (cv), öt+kor(hu), vas+eh(sl))

In translation, inflection is very frequently treated as a productive process, meaning there are rules to determine how the different inflections of a word change in translation.

Derivation

Derivational morphemes change the basic semantic meaning of a word, and can also change word category. Depending on the language pair involved, derivation is usually treated less than inflectional morphology, as the semantic changes caused by derivational morphemes can be more unpredictable.

Some examples of derivations might be -DAKi in Turkish and -ja in Finnish (Makedonya'+daki, kirjoa+ja).

TODO; more details

Compounding

Compounding is a process where two or more words are joined together to form one. In the languages spoken in Europe, this happens most productively in the Germanic languages and in the non-Indo European languages.

Examples of compound words might be:

  • Tietokoneanimaatioelokuva = Tietokone+animaatio+elo+kuva (fi)

TODO; examples

  • Kontaktlinsenverträglichkeitstest = Kontakt+Linsen+Verträglichkeit(s)+Test(de)
  • Infrastruktuurontwikkelingsplan = Infrastruktuur+ontwikkelings+plan(nl)
  • Specialisthelsetjenestelov = Specialist+helse+tjeneste+lov(da)

In languages where word compounding is very productive, it is desirable for compound words to be analysed and translated automatically.

Morphophonology

Morphophonology studies the phonological changes that morphemes undergo when they are joined together. For example, when the -ksi suffix is added to the word elefantti in Finnish, the consonant tt is shortened to t to produce elefantiksi. The superessive suffix in Hungarian is -n, but when it is affixed to a stem ending in a consonant, it receives a linking vowel o, e or ö depending on the vowels in the stem of the word, so for example: asztalon, hegyen, könyvön.

TODO; examples

Computational representations of morphology

Computational models of morphology usually use tools called finite-state transducers to model both morphotactics and morphophonology. A finite-state transducer is a bit like a flowchart, where depending on the part of the word you are reading, you make different decisions as to what inflection or derivation it has. Unlike the typical flowchart however, a decision may lead to more than one conclusion!

A finite-state transducer modelling the basic nominal morphotactics (plural, possession, case) of three words in Bashkir. Note how archiphonemes (letters in { and }) are used to represent letters that can change according to the phonology.

The above transducer, once expanded is too big to easily read through, but if we remove the possessives, we can take a closer look at how it works.

A finite-state transducer modelling the case and number inflection of the Bashkir word мәктәп "school".

Consider the example of the word мәктәп "school" in Bashkir, it declines for number (singular, plural) and case (nominative, genitive, dative, accusative, locative and ablative). If we look at the transducer above, each arc in the graph has a label. The label has two parts, a left side (on the left of :) and a right side (on the right of :). If we read from left to right, we can analyse a word.

Singular Plural
Nominative мәктәп мәктәптәр
Genitive мәктәпнең мәктәптәрнең
Dative мәктәпгә мәктәптәргә
Accusative мәктәпне мәктәптәрне
Locative мәктәпдә мәктәптәрҙә
Ablative мәктәпдән мәктәптәрҙән

You can try doing this with one word from the declension table on the right. For example мәктәптәрҙән "from (the) schools". We should get the analysis мәктәп<n><pl><abl>. The process goes something like as follows:

  • read м, write м (input: м, оutput: м)
  • read ә, write ә (input: мә, оutput: мә)
  • read к, write к (input: мәк, оutput: мәк)
  • read т, write т (input: мәкт, оutput: мәкт)
  • read ә, write ә (input: мәктә, оutput: мәктә)
  • read п, write п (input: мәктәп, оutput: мәктәп)
  • read 0, write <n> (input: мәктәп0, оutput: мәктәп<n>)
  • read т, write <pl> (input: мәктәп0т, оutput: мәктәп<n><pl>)
  • read ә, write 0 (input: мәктәп0тә, оutput: мәктәп<n><pl>0)
  • read p, write 0 (input: мәктәп0тәp, оutput: мәктәп<n><pl>00)
  • read 0, write <abl> (input: мәктәп0тәp0, оutput: мәктәп<n><pl>00<abl>)
  • read ҙ, write 0 (input: мәктәп0тәp0ҙ, оutput: мәктәп<n><pl>00<abl>0)
  • read ә, write 0 (input: мәктәп0тәp0ҙә, оutput: мәктәп<n><pl>00<abl>00)
  • read н, write 0 (input: мәктәп0тәp0ҙән, оutput: мәктәп<n><pl>00<abl>000)

Note that reading or writing 0 is like reading or writing nothing.

Practice

There are two handouts for this practical,

  • Как использовать HFST, чтобы разработать новый морфологический анализатор
  • Как использовать lttoolbox, чтобы разработать новый морфологический анализатор

Further reading

Retrieved from "https://wiki.apertium.org/w/index.php?title=Курсы_машинного_перевода_для_языков_России/Session_1&oldid=30254"