Speeding up monodix creation
Jump to navigation Jump to search
This page outlines some ideas for increasing the speed at which monolingual dictionaries (analysers) can be created.
- Extract + constraints
- Extract + constraints + corpus
Try this at some point:
**Issues** -- the corpora may not be well aligned (e.g. JRC-Acquis Czech--Slovak) -- try and discard shoddy alignments. <spectie> you have an aligned corpus <spectie> polish--czech, czech--slovak, danish--swedish <spectie> and you have an analyser for polish, czech or danish <spectie> you want to make an analyser for swedish <spectie> you make templates from the paradigms in the danish analyser <spectie> tag the danish of the corpus <spectie> that you have <spectie> align it with the swedish side <spectie> then read off the alignments, taking the surface forms from the right side and the tags from the left side -- note you need to specify the lemma in a config file, e.g. for nouns n.*.sg.nom another variation of this without parallel corpora might be to use extract and then use a bilingual dictionary (or even just wordlist) and comparable corpus to disambiguate the possibilities. -- e.g. you have a surface form in language X which can be either Noun or Verb. You look up the surface form in language X in a dictionary X--Y ( you have an analyser + tagger for Y) You disambiguate the right analysis for X based on the analysis in Y. -- you could extend this to >1 languages, e.g. you want to build a Danish analyser and you have English--Danish,Swedish--Danish wordlists and analysers for Swedish,English. You can check both.
- I'd volunteer Upper and Lower Sorbian as ideal candidates for a test run of this: the grammar is almost 100% the same, I have an almost complete paradigm list (minus verbal participles) for Lower Sorbian, and a complete forms list for Upper Sorbian - but very little information mapping endings to cases. What little I have of Kashubian adjectives comes from a manual version of the above process, BTW. -- Jimregan 01:29, 5 June 2008 (BST)
- Problem is that there is little aligned text for Upper/Lower Sorbian no? - Francis Tyers 10:21, 5 June 2008 (BST)
- True, but I did say for a test run :) Jimregan 13:00, 12 June 2008 (BST)