Difference between revisions of "Talk:Agglutination"
Jump to navigation
Jump to search
(Removing all content from page) |
|||
(3 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
− | {{TOCD}} |
||
− | In Hungarian a word has usually 2500 forms. |
||
− | Therefore a Hungarian dictionary with all forms would contain 1 million * 2500 words, |
||
− | that is 2.5 GWords, approx 20 GBytes, that can not be handled by computers |
||
− | and handling it would make no sense. |
||
− | |||
− | Yes, hunspell handles that perfectly, it also handles vowel harmony. |
||
− | |||
− | What about Apertum to handle Hungarian? [[User:Muki987|Muki987]] 10:56, 6 April 2009 (UTC) |
||
− | |||
− | :I've played about with [[hunmorph]] — one of its limitations iirc is that it cannot do generation, only analysis. My personal preference for handling languages like Hungarian and Finnish etc. is to use something like [[SFST]] (see also [[Omorfi]]). The problem of course is then to get someone to write the actual code. - [[User:Francis Tyers|Francis Tyers]] 12:21, 6 April 2009 (UTC) |
||
− | |||
− | In Hungarian there is simply too much to generate. Not sure with Finnish&Turkish&Basque&Persian, but they also have a lot. I give you an example: |
||
− | |||
− | :Hello, there is not "simply too much to generate", there are languages much more agglutinative than Hungarian that have FST morphologies. For example see [http://www.morphologic.hu/downloads/publications/na/2008_lrec-saltmil_ural_na.pdf this paper]. - [[User:Francis Tyers|Francis Tyers]] 07:10, 7 April 2009 (UTC) |
||
− | ::The above statement is wrong. [[User:Muki987|Muki987]] 19:35, 14 May 2009 (UTC) |
||
− | |||
− | * ház (house) |
||
− | * házhoz to the .. |
||
− | * háztól from the.. |
||
− | * házig up to.. |
||
− | * háznak of the.. |
||
− | * háznál at the |
||
− | * házba into.. |
||
− | * házban in the... |
||
− | * házból from the... |
||
− | * házról about ... |
||
− | * házra on top of the... |
||
− | * házon on the .... |
||
− | * házzá become a ... |
||
− | * házat it (accusativ) - 14 |
||
− | * házam (my house - repeat all previous to this like:) -- 28 |
||
− | ** házamhoz ... |
||
− | ... |
||
− | ** házamat ... |
||
− | * házad (your house repeat all previous to this) -42 |
||
− | * háza (his, her, its house repeat all previous to this) - 56 |
||
− | * házunk (our house repeat all previous to this) -- 70 |
||
− | * házatok (your house repeat all previous to this) - 84 |
||
− | * házuk (their house repeat all previous to this) - 98 |
||
− | * házé (of the house repeat all previous to this) - 112 |
||
− | * házamé (of my house -repeat all previous to this) - 126 |
||
− | ... |
||
− | * házuké (of their house -repeat all previous to this) - 210 |
||
− | * házak (plural - repeat all previous for this up to here) 420 |
||
− | ... |
||
− | * házacska ( a little house - repeat all prevoius up to here) 840 |
||
− | * házikó ( a little house- repeat all prevoius, except last) 1260 |
||
− | * házas (married- repeat all previous for this up to here, except the last 2) 1680 |
||
− | |||
− | ... |
||
− | * you can see, it is almost trivial to get thousands of words just without a grammar book for each substantive. |
||
− | |||
− | In my opinion if we get a word, házaitokétól, (which is not unusual) we need an analysis tool, that shows: |
||
− | * házaitokétól (from something of your houses) |
||
− | * this is from ház |
||
− | * it is plural |
||
− | * it suits to the prefix "from" |
||
− | * it suits to "plural you" |
||
− | * the houses own something |
||
− | * the owned thing is singular (otherwise it would be házaitokéitól) |
||
− | |||
− | With this knowledge we can construct the English (or Spanish, German, etc...) form. [[User:Muki987|Muki987]] 20:14, 6 April 2009 (UTC) |
||
− | |||
− | ::I copied the diskussion with Jimregan onto my discussion page. We can continue there. [[User:Muki987|Muki987]] 10:14, 7 April 2009 (UTC) |
||
− | |||
− | ==Comparison of Omorfi and Hunmorph== |
||
− | ===Omorfi=== |
||
− | http://www.ling.helsinki.fi/cgi-bin/omor/omorfi-cgi-demo.py |
||
− | |||
− | Omorfi - Demo of Finnish Morphology |
||
− | |||
− | These demos are based on the HFST implementation of Finnish morphology using SFST , and Nykysuomen sanalista . A guesser is used for missing words. For more information see HFST home page |
||
− | Wordform Nykysuomen has no known analyses. The 6 best baseform and paradigm guesses were chosen: |
||
− | <pre> |
||
− | *1. Nykysuomen 32 noun sg nom |
||
− | *2. Nykysuomi 7 noun sg acc |
||
− | *3. Nykysuomi 7 noun sg gen |
||
− | *4. Nykysuomi 7 noun sg ins |
||
− | *5. Nykysuomi 25 noun sg acc |
||
− | *6. Nykysuomi 25 noun sg gen |
||
− | </pre> |
||
− | As far as I can see here: http://wiki.apertium.org/wiki/Omorfi |
||
− | <pre> |
||
− | $ echo "kaikki ihmiset syntyvät vapaina ja tasavertaisina arvoltaan ja oikeuksiltaan." | fst-proc omorfi/src/omorfi.sfstc |
||
− | |||
− | ^kaikki/kaikki<noun><7><a><sg><nom>$ ^ihmiset/ihminen<noun><38><pl><acc>/ihminen<noun><38><pl><nom>$ |
||
− | ^syntyvät/syntyä<verb><52><j><act><pcpva><pl><acc>/syntyä<verb><52><j><act><pcpva><pl><nom>/syntyä<verb><52><j><act><indv><pres><pl3>$ |
||
− | ^vapaina/vapaa<noun><17><pl><ess>$ |
||
− | ^ja/*ja$ ^tasavertaisina/*tasavertaisina$ ^arvoltaan/arvo<noun><1><sg><abl><pl3>/arvo<noun><1><sg><abl><sg3>$ ^ja/*ja$ |
||
− | ^oikeuksiltaan/oikeus<noun><40><pl><abl><pl3>/oikeus<noun><40><pl><abl><sg3>$. |
||
− | </pre> |
||
− | |||
− | Omorfi also analyses and that's it. I do not see any difference to hunmorph, do you? [[User:Muki987|Muki987]] 21:24, 6 April 2009 (UTC) |
||
− | |||
− | ===Hunmorph=== |
||
− | <pre> |
||
− | $ echo "ablakot" | ocamorph --aff lexicons/morphdb.hu/out/morphdb_hu.aff --dic lexicons/morphdb.hu/out/morphdb_hu.dic |
||
− | </pre> |
||
− | and you get |
||
− | |||
− | <pre> |
||
− | > ablakot ablak/NOUN> |
||
− | </pre> |
||
− | |||
− | This is pretty much the same IMHO, what Omorfi produces. What so you think? [[User:Muki987|Muki987]] 21:28, 6 April 2009 (UTC) |
||
− | |||
− | ::The difference is that in Omorfi, you can go the other way. From |
||
− | |||
− | :::<code>^syntyä<verb><52><j><act><pcpva><pl><acc>$</code> → <code>syntyvät</code> |
||
− | |||
− | ::Can you do that in hunmorph? It was my understanding that you couldn't. - [[User:Francis Tyers|Francis Tyers]] 07:10, 7 April 2009 (UTC) |
||
− | |||
− | :::Btw, if you want to look at an agglutinative language pair currently in SVN, check out [[Northern Sámi and Lule Sámi]] — the transducers were generated from full-form lists, which is not the ideal way to do it. A better way would have been to somehow compile the XFST source code using a free compiler (for example SFST/HFST), but unfortunately that isn't possible yet :( - [[User:Francis Tyers|Francis Tyers]] 07:17, 7 April 2009 (UTC) |
||
− | |||
− | ==Moses== |
||
− | I compared Moses to Apertium, and as far as I can see, Apertium is much better, cleaner, more usable. Moses is like google translation, not bad in certain situations, but will never have acceptable quality. Unfortunately. |
||
− | ==Matxin== |
||
− | Unfortunately I can not read Spanish. I can read English, Hungarian, German. |
||
− | If anybody translated to English (using apertium,) would be a great help for me. |
||
− | |||
− | :We have Catalan→English, Welsh→English, Spanish→English, Galician→English. - [[User:Francis Tyers|Francis Tyers]] 07:12, 8 April 2009 (UTC) |
||
− | |||
− | ::I mean, I would like to be able the user's guide and description. (Descripción del sistema de traducción es-eu Matxin) [[User:Muki987|Muki987]] 19:57, 8 April 2009 (UTC) |
||
− | |||
− | :::For Apertium, you can check out the [https://wiki.apertium.org/w/images/d/d0/Apertium2-documentation.pdf official docs] linked from the [[documentation]] page, they're a bit out of date, but for a broad overview they are valid. For Matxin, I have translated the documentation into English, but they haven't put it on their site yet. If you [[Special:Emailuser/Francis Tyers|email me]] I can send you a copy. - [[User:Francis Tyers|Francis Tyers]] 21:43, 8 April 2009 (UTC) |
||
− | |||
− | ===Hungarian-whatever (inflecting)=== |
||
− | However, I have an imagination of Hungarian-whatever-Hungarian translation. |
||
− | |||
− | A Hungarian sentence would be preprocessed, and each word would be added the morphological analysis result. |
||
− | |||
− | For example: |
||
− | Ma megyek a házba. (Today (ma) I go (megyek) into the house (a házba)) |
||
− | |||
− | I get with hunmorph for this: |
||
− | > ma |
||
− | ma/ADV |
||
− | ma/NOUN |
||
− | > megyek |
||
− | megy/VERB<PERS<1>> |
||
− | > a |
||
− | a/ART |
||
− | > házba |
||
− | ház/NOUN<CAS<ILL>> |
||
− | |||
− | I would enter this to apertium, |
||
− | I would expect a usable translation for non-agglutinative languages, and some output, that hunmorph could again translate back into more readable form for agglutinative languages. |
||
− | |||
− | :Not even any need for a pre-process, there is no reason why we cannot replace the Apertium morphological analyser with hunmorph (apart from the fact that it is written in Ocaml and slow!) ;). If you're interested in doing this I'll look at converting the output of hunmorph to apertium 'standard' (see [[Apertium stream format]]). - [[User:Francis Tyers|Francis Tyers]] 07:12, 8 April 2009 (UTC) |
||
− | |||
− | ::I wonder why ocaml is that slow. It is a compilable language, being as fast, as C/C++. Also, hunpos, that delivers also word types, works much faster (is also an ocaml project!). [[User:Muki987|Muki987]] 12:10, 8 April 2009 (UTC) |
||
− | |||
− | :::I don't know, it's strange. It might be either to do with the language, or with the program itself -- but when I tried it last it was ~212 words/second. Which is at least a power of ten or more slower than lttoolbox/sfst. - [[User:Francis Tyers|Francis Tyers]] 21:43, 8 April 2009 (UTC) |
||
− | |||
− | ::Is there somewhere a system documentation of apertium, that would speed up my understandig of it's structure and logic? I believe, apertium is the right project for me, and I should like get first a system overview. Thanks. [[User:Muki987|Muki987]] 12:10, 8 April 2009 (UTC) |
||
− | |||
− | :::See my answer above. - [[User:Francis Tyers|Francis Tyers]] 21:43, 8 April 2009 (UTC) |
||
− | |||
− | ===Whatever (inflecting) -Hungarian=== |
||
− | I go into the house |
||
− | |||
− | Apertium knows from the rules: |
||
− | I go = megyek |
||
− | the: a |
||
− | Into the house: ház/NOUN<CAS<ILL>> ->hunlex or whatever translates into: házba |
||
− | |||
− | What do you think? |
||
− | [[User:Muki987|Muki987]] 20:59, 7 April 2009 (UTC) |
||
− | |||
− | :Yep, no problem. - [[User:Francis Tyers|Francis Tyers]] 07:12, 8 April 2009 (UTC) |