Difference between revisions of "Specific resources per language"

From Apertium
Jump to navigation Jump to search
 
(84 intermediate revisions by 20 users not shown)
Line 1: Line 1:
{{TOCD}}
{{TOCD}}
The incubator can be found [http://apertium.svn.sourceforge.net/svnroot/apertium/trunk/incubator/ here]. It provides a place for people to put dictionaries and other stuff that is useful in constructing language pairs. On this page you can put resources which will be useful in the construction. Try and mark them for licence, or at least free/non-free.
The incubator can be found in the 'incubator' column in https://apertium.github.io/apertium-on-github/source-browser.html. It houses language pairs which haven't completely matured and are under work.


==Albanian==
:''Dictionary: [http://apertium.svn.sourceforge.net/svnroot/apertium/trunk/incubator/apertium-mk-sq.sq.dix apertium-mk-sq.sq.dix]''
;Resources


==Specific resources per language==
* http://www.albanianoverview.com/grammar.htm

Here are some links to resources that might be useful for expanding on work in the Incubator. Below you can put resources which will be useful in the construction. Try and mark them for licence, or at least free/non-free.

See also the individual language pages.

===[[Albanian]]===
:''Dictionary: [https://github.com/apertium/apertium-sqi/blob/master/apertium-sqi.sqi.dix Albanian Monodix]''

;Resources
* http://mylanguages.org/learn_albanian.php
* http://www.seelrc.org:8080/grammar/pdf/albanian_bookmarked.pdf
* http://www.idividi.com.mk/recnik/index.htm -- albanian--macedonian dictionary (non-free)
* http://www.idividi.com.mk/recnik/index.htm -- albanian--macedonian dictionary (non-free)


==Armenian==
===[[Armenian]]===
:''Dictionary: [http://apertium.svn.sourceforge.net/svnroot/apertium/trunk/incubator/apertium-hy-en.hy.dix apertium-hy-en.hy.dix]''
:''Dictionary: [https://github.com/apertium/apertium-hye/blob/master/apertium-hye.hye.dix Armenian Monodix]''


;Resources
;Resources
Line 16: Line 24:
* http://www.armeniapedia.org/index.php?title=Category:Armenian_Language_Lessons
* http://www.armeniapedia.org/index.php?title=Category:Armenian_Language_Lessons


===[[Assamese and Hindi]]===
==Breton==
:''Dictionary: [http://apertium.svn.sourceforge.net/svnroot/apertium/trunk/incubator/apertium-br-fr.br.dix apertium-br-fr.br.dix]''
:''Dictionary: [https://github.com/apertium/apertium-as-hi/blob/91f3c38b0c636deb620cbd27725d63dd763c5f0b/apertium-as-hi.hi.dix Assemese-Hindi Bidix]''


;Resources


--- Anusuya
* http://fr.wiktionary.org/wiki/Cat%C3%A9gorie:Grammaire_en_breton
* http://books.google.com/books?id=SQYPenZO6SUC&pg=PA1&dq=modern+breton&sig=9RjVmVzuA8iV5kzahLL_0sHaDmQ
* http://books.google.com/books?id=YYkCAAAAQAAJ&printsec=frontcover&dq=breton&num=100&as_brr=1#PPR5,M1 (public domain Breton-French dictionary and Grammar)
* http://www.preder.net/klask.php (non-free)


===[[Belarusian]]===
==Cornish==
:''Dictionary: [http://apertium.svn.sourceforge.net/svnroot/apertium/trunk/incubator/apertium-cy-kw.kw.dix apertium-cy-kw.kw.dix]''


* [http://www.vitba.org/fofmb/fofmb.html GFDL grammar of the language]
==Bulgarian==
:''Dictionary: [http://apertium.svn.sourceforge.net/svnroot/apertium/trunk/incubator/apertium-mk-bg.bg.dix apertium-mk-bg.bg.dix]''


===[[Bengali]]===
;Resources


* http://bengalinux.sourceforge.net/cgi-bin/anubadok/index.pl -- Free software translation for English→Bengali
* [http://www.sfs.nphil.uni-tuebingen.de/iscl/Theses/zhechev.pdf Bulgarian verbal morphology]
* http://anubadok.sf.net/ -- See above


==Cornish==
===[[Bulgarian]]===

* https://link.springer.com/article/10.1007/s11185-010-9059-2

===[[Cornish]]===

:''Dictionary: [https://sourceforge.net/projects/apertium/files/apertium-cy-en/0.1.0/ Cornish Monodix from SourceForge]''

'''This resource has not been migrated to GitHub from SVN
'''


;Resources
;Resources


* https://www.freelang.net/online/cornish.php
* [http://www.cornishtranslator.com/ Cornish Translator]
* [http://kevindonnelly.org.uk/kernewek/ Cornish-Welsh bilingual wordlist]
* [http://kevindonnelly.org.uk/kernewek/ Cornish-Welsh bilingual wordlist]


==Czech==
===[[Czech]]===
:''Dictionary: [http://apertium.svn.sourceforge.net/svnroot/apertium/trunk/incubator/apertium-pl-cs.cs.dix.xml apertium-pl-cs.cs.dix.xml]''
:''Dictionary: [http://apertium.svn.sourceforge.net/svnroot/apertium/incubator/apertium-pl-cs.cs.dix.xml apertium-pl-cs.cs.dix.xml]''
'''This resource has not been migrated to GitHub from SVN
'''

:''Dictionary: [https://github.com/apertium/apertium-eo-cs/blob/c16fa21194a285941307a68e420c194a1825ebc3/apertium-eo-cs.eo-cs.dix Czech-Esperanto Bidix]''
:''Dictionary: [https://github.com/apertium/apertium-cs-sl/tree/062fa172705e16f77302a8096df3733581079fb8 Czech-Slovenian Bidix]''
;Resources
;Resources


* [http://nlp.fi.muni.cz/nlp/aisa/NlpCz/Frekvence_slov_lemmat.html Most frequent words] Also includes a list of the most frequent bi- and tri-grams, but these are of little use as multiwords
* [http://users.ox.ac.uk/~tayl0010/links.html James Naughton's links]
* [http://users.ox.ac.uk/~tayl0010/links.html James Naughton's links]
* [http://www.czech-language.cz/alphabet/alph-krtiny.html Some complications with diacritics]
* [http://www.czech-language.cz/alphabet/alph-krtiny.html Some complications with diacritics]
* [http://ufal.mff.cuni.cz/pdt/Morphology_and_Tagging/Morphology/index.html Czech morphological guesser] - 'free', but not open source


==Greek==
===[[Faroese]]===
:''Dictionary: [http://apertium.svn.sourceforge.net/svnroot/apertium/trunk/incubator/apertium-en-el.el.dix apertium-en-el.el.dix]
:''Dictionary: [https://github.com/apertium/apertium-fao/blob/master/apertium-fao.fao.dix Faroese Monodix]''

;Resources
* [http://giellatekno.uit.no/cgi/d-fao.eng.html U. Tromsø -- Faroese analyser ]
* [https://github.com/apertium/apertium-fao-isl/blob/master/apertium-fao-isl.fao-isl.rlx Faroese Constraint Grammar]
* [http://www.archive.org/details/frskanthologi00denmgoog Faroese-Danish dictionary from 1886]

===[[Finnish]]===
{{see-also|Omorfi}}
;Resources

* http://kaino.kotus.fi/sanat/nykysuomi/ — full form list for Finnish -- LGPL
* [http://www.ling.helsinki.fi/kieliteknologia/tutkimus/hfst/ Helsinki Finite-State Transducer Technology (HFST)]
<pre>
s = lemma
hn = homonymy ref
t = inflection info
tn = inflection number (referring to table)
av = ref to consonant gradation
</pre>

===[[German and English]]===

German-English bilingual dictionary (>216,000 entries), generated from linguistic data (GPL Version 2 or later) available for [http://www-user.tu-chemnitz.de/~fri/ding/ "Ding: A Dictionary LookUp program"] (version 1.5 2007-04-09) from Frank Richter, [http://tu-chemnitz.de Technische Universität Chemnitz]

:''[https://github.com/apertium/apertium-eng-deu/blob/master/apertium-eng-deu.eng-deu.dix German-English Dictionary]''

===[[Greek]]===
:''Dictionary: [https://github.com/apertium/apertium-ell/blob/master/apertium-ell.ell.dix Greek Monodix]
:''Greek-English Dictionary: [https://github.com/apertium/apertium-ell-eng/blob/master/apertium-ell-eng.eng.dix Greek-English Dictionary]


;Resources
;Resources
Line 58: Line 104:
* Greek <-> Ukranian, Russian, Polish Grammar & Dictionary: http://ellinika.gnu.org.ua/
* Greek <-> Ukranian, Russian, Polish Grammar & Dictionary: http://ellinika.gnu.org.ua/


==Hindi==
===[[Hebrew]]===

:''Dictionary: [http://apertium.svn.sourceforge.net/svnroot/apertium/trunk/incubator/apertium-hi-en.hi.dix apertium-hi-en.hi.dix]''
;Resources

* http://www.cs.technion.ac.il/~barhaim/MorphTagger/ HMM-based part-of-speech tagger For Hebrew -- GPL
* http://hspell.ivrix.org.il/ The hspell Hebrew spell-checker has a mode for analyzing morpholocial data -- GPL
* http://www.code972.com/blog/hebmorph/ HebMorph is the analyser powering hspell's capabilities -- GPL

===[[Hindi]]===
{{see-also|Hindi}}


;Resources
;Resources


* Morphological analyser: http://www.iiit.net/ltrc/morph/index.htm (GPL)
* POS tagged English-Hindi wordlist: http://indlinux.sourceforge.net/downloads/files/hindidict.txt.bz2
* POS tagged English-Hindi wordlist: http://indlinux.sourceforge.net/downloads/files/hindidict.txt.bz2


* https://github.com/unhammer/apertium-en-hi/blob/master/apertium-en-hi.en.dix
==Iranian Persian==
:''Dictionary: [http://apertium.svn.sourceforge.net/svnroot/apertium/trunk/incubator/apertium-tg-fa.fa.dix apertium-tg-fa.fa.dix]''
* https://github.com/apertium/apertium-hin/blob/master/apertium-hin.hin.dix
* https://github.com/apertium/apertium-urd-hin/blob/master/dev/en-hi-ur.list
* https://github.com/apertium/apertium-urd-hin/blob/master/apertium-urd-hin.urd-hin.dix



===[[Iranian Persian]]===
:''Dictionary: [https://github.com/apertium/apertium-pes/blob/master/apertium-pes.pes.dix Persian Monodix]''


;Resources
;Resources
Line 73: Line 133:
* [http://books.google.com/books?vid=OCLC20216670&id=Ru1ncSqiRXkC&printsec=titlepage&hl=de#PPA24,M1 Grammar of Persian]
* [http://books.google.com/books?vid=OCLC20216670&id=Ru1ncSqiRXkC&printsec=titlepage&hl=de#PPA24,M1 Grammar of Persian]


==Portuguese==
===[[Ingush]]===


; Resources
Even if Apertium has a stable es-pt pair, the coverage of the Brazilian Portuguese Dictionary built at NILC (Universidade de Sao Paulo) for Unitex is much better, and could be used perhaps to improve it.


* [http://www.linguistics.berkeley.edu/~ingush/database.html Lexical database] (non-free)
* [http://books.google.com/books?id=J7wqVHeRWdwC&pg=PA5&lpg=PA5&dq=ingush+father&source=bl&ots=N8TDZudzGZ&sig=JO9X_Y9gio7dUhZWeyZX7j17iPw&hl=ca&ei=vfq4TM6CH86OjAfO94XaDg&sa=X&oi=book_result&ct=result&resnum=3&ved=0CB8Q6AEwAg#v=onepage&q=ingush%20father&f=false Ingush-English dict] (non-free)

===[[Latvian]]===
;Resources
;Resources
* https://github.com/PeterisP/morphology GPL full-form dictionary (https://github.com/PeterisP/morphology/blob/master/src/main/resources/Lexicon.xml)


;See also
* [http://www.nilc.icmc.usp.br/nilc/projects/unitex-pb/web/dicionarios.html Recursos Lexicais Português do Brasil]
* [[Latvian and Russian]]


===[[Lithuanian]]===
We believe it has a LGPL license.
:''Dictionary: [https://github.com/apertium/apertium-lit/blob/master/apertium-lit.lit.dix Lithuanian Monodix]''


;Resources
==Russian==


===[[Nogai]]===
:''Dictionary: [http://apertium.svn.sourceforge.net/svnroot/apertium/trunk/incubator/apertium-pl-ru.ru.dix.xml monodix]''

:''Bidix: [http://apertium.svn.sourceforge.net/svnroot/apertium/trunk/incubator/apertium-pl-ru.pl-ru.dix.xml Polish-Russian]''
'''Contents to be added'''
:''Bidix: [http://apertium.svn.sourceforge.net/svnroot/apertium/trunk/incubator/apertium-en-ru.en-ru.dix.xml English-Russian]

===[[Ossetian]]===
:''Dictionary: [https://github.com/apertium/apertium-oss/blob/master/apertium-oss.oss.dix Ossetian Monodix]''


;Resources
;Resources


* [http://www.azargoshnasp.net/languages/ossetian/grammersketchossetian.pdf Ossetian: Grammatical Sketch] &mdash; quite nice and comprehensive.
* http://www.alphadictionary.com/rusgrammar/
* http://www.seelrc.org:8080/grammar/pdf/stand_alone_russian.pdf
* [http://www.ossetic-studies.org/ Ossetic National Corpus]


===[[Piemontese]]===
==Slovakian==
:''Dictionary: [http://apertium.svn.sourceforge.net/svnroot/apertium/trunk/incubator/apertium-pl-sk.sk.dix apertium-pl-sk.sk.dix]''
:''Dictionary: [https://sourceforge.net/p/apertium/svn/HEAD/tree/incubator/apertium-it-pms.pms.dix Piemontese Monodix from SourceForge]''
'''This resource has not been migrated to GitHub from SVN
'''


;Resources
;Resources


* http://digilander.libero.it/dotor43/indexit.html -- Piemontese grammar incl. 17k word Piemontese--Italian dictionary (POS tagged and partly annotated for inflection). site suggests "© These pages can be freely used for all purposes, but not for political reasons, and not against the laws (no matter what is the country)."
* http://old.bohemica.com/slovak/slovakgrammar.pdf (Slovakian, with some English)
* http://pl.wiktionary.org/wiki/Aneks:J%C4%99zyk_s%C5%82owacki_-_tabele_koniugacji (In Polish)
* http://www.angelfire.com/sk3/quality/Slovak_declension.html


===[[Portuguese]]===
==Swedish - Danish==

:''Pair: [http://apertium.svn.sourceforge.net/svnroot/apertium/trunk/apertium-sv-da/ apertium-sv-da]''
Even if Apertium has a stable es-pt pair, the coverage of the Brazilian Portuguese Dictionary built at NILC (Universidade de Sao Paulo) for Unitex is much better, and could be used perhaps to improve it.


;Resources
;Resources


* [http://www.nilc.icmc.usp.br/nilc/projects/unitex-pb/web/dicionarios.html Recursos Lexicais Português do Brasil]
* http://w3.msi.vxu.se/~nivre/research/Talbanken05.html (A 300,000-word tree-bank: it is in XML, all words are nicely tagged with PAROLE-style tags, and it should be easy to build a morphological analyser and a PoS tagger from it; authors are likely be happy to let us use it if we cite them).
* http://www.isv.cbs.dk/~mbk/treebank/ (Danish tree bank, 100,000-word, as above, under the GPL)
* http://www.ling.su.se/staff/sofia/suc/suc.html (Stockholm Umeå Corpus: 1,000,000 Swedish words, tagged; a license has to be granted by authors - it was used for apertium-sv-da)


We believe it has a LGPL license.
==Quechua==

===[[Punjabi]]===

'''Contents to be added'''

===[[Quechua]]===


;Resources
;Resources
Line 119: Line 194:
* AVENUE Quechua-Spanish system. (ask [[User:Francis Tyers|Francis Tyers]])
* AVENUE Quechua-Spanish system. (ask [[User:Francis Tyers|Francis Tyers]])


==Norwegian==
===[[Russian]]===


:''Dictionary: [https://github.com/apertium/apertium-rus/blob/master/apertium-rus.rus.dix monodix]''
''See: [[Norsk ordbank]]''
:''Bidix: [https://github.com/apertium/apertium-pol-rus/blob/master/apertium-pol-rus.pol-rus.dix Polish-Russian]''

:''Bidix: [https://github.com/apertium/apertium-rus-eng/blob/master/apertium-ru-en.ru.dix English-Russian]
==Urdu==
:''Dictionary: [http://apertium.svn.sourceforge.net/svnroot/apertium/trunk/incubator/apertium-hi-ur.ur.dix apertium-hi-ur.ur.dix]''


;Resources
;Resources
* http://www.lama.univ-savoie.fr/~humayoun/UrduMorph/ &mdash; GPL analyser of Urdu
* http://www.crulp.org/software/langproc/E2UMachineTranslationSystem.htm -- Urdu--English MT system


* http://www.alphadictionary.com/rusgrammar/
==Lithuanian==
* http://www.seelrc.org:8080/grammar/pdf/stand_alone_russian.pdf
:''Dictionary: [http://apertium.svn.sourceforge.net/svnroot/apertium/trunk/incubator/apertium-en-lt.lt.dix apertium-en-lt.lt.dix]''
* [http://www.cic.ipn.mx/~sidorov/rmorph/index.html Russian analyser] - non-free, Windows only
* [http://citeseer.ist.psu.edu/cache/papers/cs2/433/http:zSzzSzwww.ling.ohio-state.eduzSz~hanazSzbibliozSzHanaFeldmanBrew2004-RusMorphLite.pdf/hana04resourcelight.pdf Using Czech resources for the morphological analysis of Russian]
*[http://sourceforge.net/projects/pere/ Pere] - free translator, including Russian<->Ukranian<->English dictionaries. Built from alignments, low quality.
* [http://www.revdanica.com/xdxf/tmp/Muzafarov/inXDXF/rus2taj.xdxf Russian--Tajik phrase dictionary, 41k entries].

===[[Sanskrit]] '''संस्कृतम्'''===
:''Dictionary: [https://github.com/apertium/apertium-san/blob/master/apertium-san.san.dix Sanskrit Monodix]


;Resources
;Resources
* [http://www.sanskrit-lexicon.uni-koeln.de/ Sanskrit Lexicon at Uni-Koeln]
* [http://www.sanskrit-lexicon.uni-koeln.de/aequery/index.html Apte's En-Sa] dictionary


==Finnish==
===[[Slovakian]]===
:''Dictionary: [https://github.com/apertium/apertium-slk/blob/master/apertium-slk.slk.dix Slovak Monodix]''


;Resources
;Resources


* http://pl.wiktionary.org/wiki/Aneks:J%C4%99zyk_s%C5%82owacki_-_tabele_koniugacji (In Polish)
* http://kaino.kotus.fi/sanat/nykysuomi/ &mdash; full form list for Finnish -- LGPL
* http://www.angelfire.com/sk3/quality/Slovak_declension.html
* [https://kitwiki.csc.fi/twiki/bin/view/KitWiki/OMorFiSFSTVersion#Installation Omorfi–Open Morphology for Finnish language]
* [http://www.ling.helsinki.fi/kieliteknologia/tutkimus/hfst/ Helsinki Finite-State Transducer Technology (HFST)]
<pre>
s = lemma
hn = homonymy ref
t = inflection info
tn = inflection number (referring to table)
av = ref to consonant gradation
</pre>


==Hebrew==
===[[Thai]]===
* https://github.com/veer66/Yaitron Yaitron English-Thai and Thai-English XML dictionary, license seems standard 4-clause


===[[Urdu]]===
;Resources
:''Dictionary: [https://github.com/apertium/apertium-urd/blob/master/apertium-urd.urd.dix Urdu Monodix]''
:''Bidix: [https://github.com/apertium/apertium-urd-hin/blob/master/apertium-urd-hin.urd-hin.dix Hindi-Urdu Monodix]''


==Github Migration==
* http://www.mila.cs.technion.ac.il/english/resources/lexicons/ lexicons for Hebrew, in weird XLS format -- GPL


For languages whose resources are not yet on Github, you can use [[apertium-init]] to make their corresponding repository and add the files from SVN to that repositiry.
==Piemontese==


;Resources


* http://members.fortunecity.it/dotorcarlo/vocen1.html Piemontese--English -- public domain
* http://digilander.libero.it/dotor43/indexit.html -- Piemontese grammar incl. 17k word Piemontese--Italian dictionary (POS tagged and partly annotated for inflection). site suggests "© These pages can be freely used for all purposes, but not for political reasons, and not against the laws (no matter what is the country)."


==Bengali==

* http://bengalinux.sourceforge.net/cgi-bin/anubadok/index.pl -- Free software translation for English→Bengali
* http://anubadok.sf.net/ -- See above


[[Category:Development]]
[[Category:Development]]
[[Category:Repository]]
[[Category:Documentation in English]]

Latest revision as of 20:40, 11 December 2019

The incubator can be found in the 'incubator' column in https://apertium.github.io/apertium-on-github/source-browser.html. It houses language pairs which haven't completely matured and are under work.


Specific resources per language[edit]

Here are some links to resources that might be useful for expanding on work in the Incubator. Below you can put resources which will be useful in the construction. Try and mark them for licence, or at least free/non-free.

See also the individual language pages.

Albanian[edit]

Dictionary: Albanian Monodix
Resources

Armenian[edit]

Dictionary: Armenian Monodix
Resources

Assamese and Hindi[edit]

Dictionary: Assemese-Hindi Bidix


--- Anusuya

Belarusian[edit]

Bengali[edit]

Bulgarian[edit]

Cornish[edit]

Dictionary: Cornish Monodix from SourceForge

This resource has not been migrated to GitHub from SVN

Resources

Czech[edit]

Dictionary: apertium-pl-cs.cs.dix.xml

This resource has not been migrated to GitHub from SVN

Dictionary: Czech-Esperanto Bidix
Dictionary: Czech-Slovenian Bidix
Resources

Faroese[edit]

Dictionary: Faroese Monodix
Resources

Finnish[edit]

See also: Omorfi
Resources
s = lemma
hn = homonymy ref
t = inflection info
tn = inflection number (referring to table)
av = ref to consonant gradation

German and English[edit]

German-English bilingual dictionary (>216,000 entries), generated from linguistic data (GPL Version 2 or later) available for "Ding: A Dictionary LookUp program" (version 1.5 2007-04-09) from Frank Richter, Technische Universität Chemnitz

German-English Dictionary

Greek[edit]

Dictionary: Greek Monodix
Greek-English Dictionary: Greek-English Dictionary
Resources

Hebrew[edit]

Resources

Hindi[edit]

See also: Hindi
Resources


Iranian Persian[edit]

Dictionary: Persian Monodix
Resources

Ingush[edit]

Resources

Latvian[edit]

Resources
See also

Lithuanian[edit]

Dictionary: Lithuanian Monodix
Resources

Nogai[edit]

Contents to be added

Ossetian[edit]

Dictionary: Ossetian Monodix
Resources

Piemontese[edit]

Dictionary: Piemontese Monodix from SourceForge

This resource has not been migrated to GitHub from SVN

Resources
  • http://digilander.libero.it/dotor43/indexit.html -- Piemontese grammar incl. 17k word Piemontese--Italian dictionary (POS tagged and partly annotated for inflection). site suggests "© These pages can be freely used for all purposes, but not for political reasons, and not against the laws (no matter what is the country)."

Portuguese[edit]

Even if Apertium has a stable es-pt pair, the coverage of the Brazilian Portuguese Dictionary built at NILC (Universidade de Sao Paulo) for Unitex is much better, and could be used perhaps to improve it.

Resources

We believe it has a LGPL license.

Punjabi[edit]

Contents to be added

Quechua[edit]

Resources

Russian[edit]

Dictionary: monodix
Bidix: Polish-Russian
Bidix: English-Russian
Resources

Sanskrit संस्कृतम्[edit]

Dictionary: Sanskrit Monodix
Resources

Slovakian[edit]

Dictionary: Slovak Monodix
Resources

Thai[edit]

Urdu[edit]

Dictionary: Urdu Monodix
Bidix: Hindi-Urdu Monodix

Github Migration[edit]

For languages whose resources are not yet on Github, you can use apertium-init to make their corresponding repository and add the files from SVN to that repositiry.